CategoryDeveloping

Gzip and upload crawled files to S3 with Scrapy

Just wanted to drop a quick note if any one lands here and is struggling with the problem of using Scrapy’s build-in S3 feed storage and wants to upload the feed/file created by scrapy to S3 as gzipped file.

Continue reading

Re:dash: finding the perfect Analysis tools

Re:dash is an open source data collaboration and visualisation tool that allows you to share dashboards across your business. Designed because of the bloated other tools that data teams had to use when connecting to Redshift.

We’ve trailed the open-source version (release of late 2016: 0.12) and you can see what I think about the tool in the review below.

Continue reading

Finding the perfect Data (SQL) Analyses tool

This is probably the most generic title I can come up with but at my current company  we are looking into new ways of visualising and sharing our data within the company.

This sounds like it should be very easy to find but after the guys from WagonHQ got bought by Box there seems to be quite a gap in the SQL dashboarding landscape.

We used Wagon for quickly pumping out some shared chart we can embed into a page, which we can then send across the office. Nothing complicated. We use Redshift as a data warehouse.

Keep in mind that when I’m saying the tools below are “bad” or “not working” it’s because they aren’t doing the thing I want them to do. Doesn’t mean its a crap platform or wouldn’t work for you.

Continue reading

Mode Analytics: finding the perfect Analysis tools

EDIT 2017-10-22: The team at Mode has been updating the tool almost every single month, the below is based on what our team needed in 2016 and what Mode Analytics could provide at the time of writing.

Lots of the work we do, and one of the many things we use our SQL tool for,  is for small little investigations (wouldn’t want to call it growth hacking but something similar).

This means writing quick, single queries with an output that only needs to be looked and stared at for 5 minutes and then never again. Mode Analytics in this instance doesn’t feel like the right tool for the job but we go it to work for part of the business. Especially A/B Experiments in combination with the Python environment.

Continue reading

Howto Bulk Unfollow Playlists on Spotify

I’ve been trying to use Spotify the last couple of weeks as a substitute for Google Music. And at the moment I’m quite loving it. The account I’ve created about 3 years ago was still active but had all my old playlists on it. The problem was that this was a import from iTunes and it imported all my 283(!) playlists and unfortunately there is no easy way of deleting playlists inside Spotify.

Continue reading

© 2024 Koen Kamphuis

Theme by Anders NorénUp ↑