Processing a CSV File in Batch With Sidekiq
04-Apr-2019 3059
Sidekiq Pro comes with a great feature to process a collection of jobs as a batch, allowing them to be monitored as a group and executing a callback function when all the jobs are finished. This is useful when you need to load a lot of spreadsheet files into your database.Recently, that was the case of one of Ombu Labs' clients. They needed to upload a CSV file with over 10 thousand rows of loans data, which makes processing the file synchronously impossible because the browser will time out after a few seconds. Breaking the file into smaller ones wasn't a good idea either, because it would take an unacceptable amount of time to finish. So we decided to use the Sidekiq's batch logic.
Processing a CSV File in Batch With Sidekiq #ruby #rubydeveloper #rubyonrails #Processing #Batch #Sidekiq #sidekiq https://www.rubyonrails.ba/link/processing-a-csv-file-in-batch-with-sidekiq