I run a business. I also code every day.
The developer in me loves to experiment with shiny new tech. It's a ton of fun to grab the most talked about tech of the moment and build something cool with it.
But I have to admit that while there's a lot to learn from such experiments, they usually don't result in new tech that can be deployed to production.
You see, while the developer in me may be excited by what could be accomplished with the new tech, the business owner in me knows that there's a lot more to consider before any new tech can be deployed successfully.
In this post, I'll describe the decision making process we've used in selecting and evolving the tech stack of Enchant.
Development doesn't happen in a vacuum
Once a system is built, it's going to live for years in test and production environments.
It will need to meet legal and compliance checklists.
If it stores data, we'll need some way to back it up and restore it.
We'll need experienced people on dev and ops teams to maintain it.
We'll likely also need to scale it at some point.
... and in process of getting through all of that, I'm sure we're going to hit edge cases and we'll need help from the community in solving those problems.
All in all, there's a lot of effort and people involved to successfully deploy new technology into production. Far beyond the initial development effort.
What we need to do is choose the "right" technology, the one that gets out of the way and helps streamline all those efforts.
This is not about using the right tool for the job
We're taught to use the right tool for the job.
Need real time browser communications? Use websockets.
Need in memory caching? Use memcached.
Need to add search capability? Use elasticsearch.
This line of thinking isn't wrong.
But it's not quite right, either.
Yes, we do want to use the right tool for the job. However, the right tool for the job may not be the most right tool for the company.
It may be unreasonably expensive.
It may require more maintenance than the team can commit to.
It may not integrate well with our existing tech.
... and so on.
So, instead, we need to figure out what technology gives the right balance of pros and cons for our company.
This is also not about using boring technology either
Some advocate using boring technology.
Boring technology is a concise and interesting phrase that makes sense on the surface. Avoid the new and shiny because it may not be around in 10 years.
But boring technology doesn't honestly represent the full picture. There's lots of technology that's pretty boring and is still in use after 20 years, but nobody really cares about it anymore... and you probably shouldn't either.
Just because it's boring and still around doesn't meant it's good.
When we were just starting out
Years ago when we started building Enchant, we used some tech that we understood well and some tech that was new and shiny.
Although that seemed like a good idea at the time, most of the new and shiny choices eventually caused more pain than we had ever anticipated. We ended up either ripping them out or working around the problems they introduced.
When starting out, we had countless tech and non-tech challenges to solve. Any time spent struggling with unfamiliar tech was precious time wasted.
The tech we understood well, however, enabled us to move fast and focus on things that mattered more.
If we were starting over, I'd strongly vote to stick to what we know and understand well.
At least in the beginning.
How we choose the right tech today
Today our approach is much more methodical.
We start by creating a proof of concept using the right tool for the job.
When the right tool for the job isn't clear, we make multiple proof of concepts to try out the most right looking tools for the job.
Creating these proof of concepts is an important step.
It helps us get a gauge for the tech - implementation/operational complexity, documentation, community, and so on.
Once we have a good feel for the tech and how it solves the problem, we're in a much better position to think about how we actually want to solve the problem.
Then it comes down to finding balance.
Balance between what we already manage and what new stuff we're willing to take responsibility for.
The right technology is already there
Sometimes the problem can be solved by tech we already have as part of our stack.
Need real time browser communications? Use fast polling.
Need in memory caching? Use a caching library.
Need to add search capability? Use full text search in the database.
These may not be best options.
But they're pretty awesome options when they fit our current scale and let us move forward without adding additional operational complexity.
These type of decisions are usually a huge win since we already have the people and processes in place for a production deployment.
The right technology is proven and widely deployed
It's usually a bad idea to start using really new technology even if the world is very excited about it and it sounds like it'll solve all the problems.
We just don't know what will happen to it in the next 3, 5 or 10 years.
Will we be able to continue to hire good people for it?
Will it get updated when security issues are found?
Will we find ourselves stuck with tech that's difficult to migrate away from?
The solution to all of this? Just wait.
Wait until the technology matures.
Wait until it's widely deployed.
Wait until it's the obvious solution.
If the tech turns out to be short lived, it will be gone before we even start to care.
More importantly, if it survives, it's pros and cons (and there are always cons) will be well known and documented.
The right technology is robust
When a technology evolves to meet the complex requirements of business deployments, it becomes pretty flexible and awesome.
Processes are in place to handle security releases.
Tooling and documentation exists to guide users through upgrades.
High availability and backup options are documented.
Paths to scale the technology are obvious with published success stories.
There's a strong community of users who share best practices based on their experiences.
Most importantly, the creators of the technology have better understood the flaws of their initial design. The tech is now stable, reliable, adaptable and robust.
Sometimes, even the most right tech is no longer right
It's important to be periodically re-evaluating the choices we've made.
What was right 3 years ago may not make the most sense anymore.
The signs are usually obvious:
Major supporters and vocal users migrate away.
The community starts to crumble away.
Improvements are no longer happening at the pace they once did.
When the time comes, we need to be ready to assess alternate options and create a plan of action.
Creating and evolving your tech stack is more of an art than science.
If you're starting out fresh, go with what you know.
If you're growing and expanding, be conscious of what you already use and the complexity that any new tech would introduce.
Evolving a tech stack comes down to finding balance between what you already manage and what you're willing to take responsibility for.