Set Up BigQuery Database + Monthly Auto-Ingestion for Amazon Shipment Data
Bütçe: $500.0
FIXED /
⭐ 5.00 (32)
United States
etl-pipelines, sql, python, bigquery, google-cloud-platform
We're an e-commerce brand that's been exporting Amazon Fulfilled Shipment reports monthly since January 2020. We now have 6.5 years of historical shipment data (888K rows across 78 monthly files) that we want to centralize in Google BigQuery for analysis.
The data prep work is already done. We have a single combined, deduplicated CSV (507MB, ~888K rows) with a standardized schema, plus a schema document describing every column. You won't need to clean, merge, or transform raw files. That part is handled.
We need two things.
First, load the existing combined CSV into a BigQuery dataset with appropriate column types (timestamps parsed, price fields as numeric, etc.).
Second, set up a process so that each month when we export a new report from Amazon (CSV, ~20K-30K rows), it gets added to the same BigQuery table without manual work beyond dropping the file somewhere. Amazon has used two different header formats over the years, which the schema doc explains. The monthly process needs to handle both formats and deduplicate against existing data.
The schema document was generated with AI assistance, so we'd welcome your feedback on it. If you see a better way to structure the data or the column types, tell us.
Please include in your proposal how you'd approach the monthly automation specifically, and a rough hour estimate for the full scope.
Upwork'te aç