Rails 导出大容量 csv
2019/10/08
有时候我们可能需要从数据库查询大量的结果然后导出 csv,如果直接 send_data,可能需要占用大量内存或者导致超时,这时候可以用 stream 方式导出,这样用分页的方式把数据分批输出到浏览器,直到数据传输完毕。同时可以调整 limit 的值,来决定导出的快慢,但是要注意内存占用情况。
def export_action
respond_to do |format|
format.csv do
filename = "large_file.csv"
headers.delete("Content-Length")
headers["Cache-Control"] = "no-cache"
headers["Content-Type"] = "text/csv"
headers["Content-Disposition"] = "attachment; filename=\"#{filename}\""
headers["X-Accel-Buffering"] = "no"
self.response_body = export_csv
response.status = 200
end
end
end
def export_csv
Enumerator.new do |yielder|
# header: your csv header
yielder << CSV.generate_line(header)
# iterate your data to export
limit = 5000
pages = (total_count.to_i / limit.to_f).ceil
pages.times do |offset_idx|
Model.limit(limit).offset(offset_idx * limit).each do |row_result|
yielder << CSV.generate_line(cvs_report_row(row_result))
end
end
end
end
# format your csv row
def csv_report_row(row_result)
csv_row = []
csv_row << row_result.column
csv_row
end
参考:
https://medium.com/table-xi/stream-csv-files-in-rails-because-you-can-46c212159ab7
https://coderwall.com/p/kad56a/streaming-large-data-responses-with-rails