Tagpython

Gzip and upload crawled files to S3 with Scrapy

Just wanted to drop a quick note if any one lands here and is struggling with the problem of using Scrapy’s build-in S3 feed storage and wants to upload the feed/file created by scrapy to S3 as gzipped file.

Continue reading

Mode Analytics: finding the perfect Analysis tools

EDIT 2017-10-22: The team at Mode has been updating the tool almost every single month, the below is based on what our team needed in 2016 and what Mode Analytics could provide at the time of writing.

Lots of the work we do, and one of the many things we use our SQL tool for, ¬†is for small little investigations (wouldn’t want to call it growth hacking but something similar).

This means writing quick, single queries with an output that only needs to be looked and stared at for 5 minutes and then never again. Mode¬†Analytics in this instance doesn’t feel like the right tool for the job but we go it to work for part of the business. Especially A/B Experiments in combination with the Python environment.

Continue reading

© 2022 Koen Kamphuis

Theme by Anders NorénUp ↑