ST Alpha - "I keep running out of sort work space when using the Order By. So what I really need is a way to run 10 simultaneous queries against a 200 million " (Reading a Post)

SlickerTalk
Search Archives
The Leaderboard
The FAQ

Login
Create Account

Search Dr. S. Talk
TT/ST Wiki
How Well Do You Know ...

RSS Feed

Hosting by DigitalOcean

Support ST on Ko-Fi

Down to Post

Backboards:

Posts: 155

In response to "why not just import them into a temp table and sort them in mysql directly? -- nm" by oblique

I keep running out of sort work space when using the Order By. So what I really need is a way to run 10 simultaneous queries against a 200 million

Posted by GregW (aka not2fast) Aug 23 '12, 13:30

row table where each query will return 20 million rows each.

Responses:

have you looked into views? -- nm - Beryllium Aug 23, 13:31 8
- No, what would that look like? -- nm - GregW Aug 23, 13:33 7
  - I forget the exact syntax, but it's like a table that represents a query - so you could make a query that selects the subset, - Beryllium Aug 23, 13:36 6
    - I don't know what you mean by 'selects the subset'. Right now, I am selecting the whole table into a file and sorting the file. - GregW Aug 23, 13:47 5
      - SELECT * FROM table LIMIT 40000000,60000000 ORDER BY whatever. What is your requirement for sorting? - oblique Aug 23, 13:52 3
        
        Thanks, I will try that. It isn't the time needed, but it apparently keeps running out of space even though I have over 100G free. - GregW Aug 23, 13:57 1
        
        very strange. How big is the field you are sorting on, or what datatype is it? -- nm - Beryllium Aug 23, 14:09
        
        Yes, indexes can speed up sorting (and even just selecting) -- nm - Beryllium Aug 23, 13:53
      - using LIMIT and OFFSET, yes -- nm - Beryllium Aug 23, 13:50

Post a message top

Replies are disabled on threads older than 7 days.