<-- home

Rails 导出大容量 csv

有时候我们可能需要从数据库查询大量的结果然后导出 csv,如果直接 send_data,可能需要占用大量内存或者导致超时,这时候可以用 stream 方式导出,这样用分页的方式把数据分批输出到浏览器,直到数据传输完毕。同时可以调整 limit 的值,来决定导出的快慢,但是要注意内存占用情况。

def export_action
  respond_to do |format|  
    format.csv do
      filename = "large_file.csv"

      headers.delete("Content-Length")
      headers["Cache-Control"] = "no-cache"
      headers["Content-Type"] = "text/csv"
      headers["Content-Disposition"] = "attachment; filename=\"#{filename}\""
      headers["X-Accel-Buffering"] = "no"

      self.response_body = export_csv

      response.status = 200
    end
  end
end

def export_csv
  Enumerator.new do |yielder|
    # header: your csv header
    yielder << CSV.generate_line(header)

    # iterate your data to export
    limit = 5000
    pages = (total_count.to_i / limit.to_f).ceil

    pages.times do |offset_idx|
      Model.limit(limit).offset(offset_idx * limit).each do |row_result|
        yielder << CSV.generate_line(cvs_report_row(row_result))
      end
    end
  end
end
      
# format your csv row
def csv_report_row(row_result)
  csv_row = []
  csv_row << row_result.column
  csv_row
end

参考:

https://medium.com/table-xi/stream-csv-files-in-rails-because-you-can-46c212159ab7

https://coderwall.com/p/kad56a/streaming-large-data-responses-with-rails