ST Alpha - "I am running a tool that extracts the data to my DB2 server where I will use it to load our new database." (Reading a Post)

SlickerTalk
Search Archives
The Leaderboard
The FAQ

Down to Post

Backboards:

Posts: 152

In response to "and what do you want to do with it? -- nm" by Beryllium

I am running a tool that extracts the data to my DB2 server where I will use it to load our new database.

Posted by GregW (aka not2fast) Aug 23 '12, 15:14

Since it has about 200 million rows, it takes over 5 hours to extract the entire thing. My management wants me to try to split this into smaller groups of data that can be extracted on multiple threads to see if I can get the extract done quicker.

They want to shorten the downtime for when we switch from MySQL to DB2 in production.

It seems to work, but it does not accept the LIMIT keyword for some reason. I need to check with IBM on that one.

Responses:

Really? It doesn't use disk files for sorting? -- nm - GregW Aug 23, 15:21 1
- mysql might - but since you're invoking it from java, I think it's trying to create an in-memory result set that is too large -- nm - Beryllium Aug 23, 15:22
You might want to forget java and just use mysqldump ... or even the mysql command itself to make a CSV -- (link) - Beryllium Aug 23, 15:19
well, it sounds like what you're telling it to do is "select 200 million rows into RAM", which ... isn't useful. -- nm - Beryllium Aug 23, 15:17

Post a message top

I am running a tool that extracts the data to my DB2 server where I will use it to load our new database.

Replies are disabled on threads older than 7 days.