Build yet another customer success/support/CMS internal tool Over the period of the few last years, I have built several back-facing panels used by customer success teams, marketing folks, and other developers as well. Various plug and play rails options like administrate or active admin are attractive options but in my experience quickly you will find them limiting and instead of speeding up do development process it will slow you down.

Fight massive technical debt on all fronts Once I took over a project that was in a really bad shape. And it was not a joke - starting from useless tests that tested implementation itself, ridiculously complex codebase (everything interconnected with everything), and finishing with fragmented infrastructure (outdated Linode servers + stuff randomly deployed to Heroku). Who would even pick up a project like this you ask? Probably not somebody sane.

Empower the team by providing tooling I wanted to provide some tooling for the team and at the same time, I didn’t want to burden our devops person (yup, a single one, talk about efficiency and bus factor, right?) too much. After some research I discovered Rancher - and oh boy it was a crazy good call to use it! We used it for hosting our Sentry instance, we spin up Grafana and pushed a lot of data into InfluxDB to create some better observability around our apps, we spin up our CI instance using Drone - and even used some gaming-dedicated boxes from hosting provider to speed up our test suites (as we could control the hardware underneath).

Account Importer after the merger - hacks on hacks with postgres_fwd Once the merger was done we immediately had a business need to start moving accounts from our old product to the new one. As most of the data structure was rather similar I have decided to absolutely abuse postgres_fwd. The postgres_fdw module basically allows you to define remote servers and access them in your queries transparently. So you can have server_1 (localhost), server_2 (remote), and server_3 (other remote) and construct an insert query by selecting a bunch of stuff from server_2 or/and server_3 - and even do that in a single transaction - which is crazy powerful.

Company merger - impossible 3-months rewrite It was a fun time, our company acquired two other companies and we were told that we have 3 months to integrate all the products into a single, unified platform. So you would sign in and could use one, two, or three products in any configuration seamlessly. We estimated the effort for around 1 year of work and we got 3 months to deliver it.

Introduce Go (that had great adaptation) We got to a point where we had to start deploying some apps on low-end machines and ruby stopped being the right tool for this problem. High memory footprint, tricky to track memory leaks pushed us to the corner. Once we were in that corner I started looking into compiled languages that could be potentially easily adapted at our company. Go seems really attractive in that department - really simple syntax, solid tooling, ridiculously simple deployment (single binary) and it was gaining more and more attention (for example it had great netlink library that we could just effortlessly consume).

Weird Microservices Redis setup that worked nicely It was a follow-up on that resque/sidekiq-based application - it was growing and growing, along with the business logic complexity. In 2016 I suggested splitting it into a bunch of smaller ruby-sidekiq apps that would communicate via Redis pipeline - after all, we already used Redis for communication and were relying on sidekiq heavily - so why reinvent the wheel? We ended up with a bunch of small apps and two internal gems.

Optimize database for fun & profit We got to a point where we ran out of int32 range on our biggest table - remember when Basecamp years later made a big blog post after the same issue caused some downtime for them? We were overflowing int32 before it was cool! Once you get to that point you have to think if you’re using that datetime field and if maybe date won’t do the job here (4 bytes less for each row!

Disaster recovery - ~8h rewrite from MongoDB to Postgres I recall once we had a huge problem with our MongoDB cluster - without going into too many details - it was like a 3TB+ beast that we have used mostly as some sort of cache layer. Unfortunately one of the replica servers decided to die on us - plugging a new server into the cluster as you can imagine slowed down everything to the point everything almost stopped - new server tried to pull the data from the main server that was under heavy load already and we ended up in a weird spot.

Start hiring from the ground up after moving to another city I remember my first hr-talk - it was so awkward. Nobody prepared me for it. Nobody walked me through anything, gave me any tips, I was just left alone to ask some questions for a person who was applying for a senior role. It went so bad and I was so stressed! I cherished that memory in order not to make the same mistake with my peers :).