← Zákazky

Merging large public data - millions of records

Rozpočet: $5.0 - $12.0 HOURLY / PART_TIME ⭐ 4.93 (95) United States

python, data-extraction

I need to merge several large public data files together, they are provided via csv or api. In total they have millions of rows of data - more than i can open in excel. I have instructions on how to merge it from ChatGPT and sample scripting but need someone who execute it. We are going to do it in a few parts The first part is taking the public files, and merging each sheet together, deleting a lot of data we don't need (columns/rows) and sending me back the completed file in parts that can open each in excel and then send a few segments out to me separately. Once that is complete we discuss part 2 which is merging many more files together to create a large database Part 3 would be to use the API and store the data on a cloud big data server so we can build something with it. Please tell about experience handling and merging together files with millions of rows. This is like 20mm rows and tell how you would go about it and store it. Then there are instructions we have for how to match the files and merge them together to get the exact file we need.
Otvoriť na Upwork