Software development has transformed massively in the last decade. We have tons of tools now, which we didn’t have before. Methods have also evolved, things like CI / CD have changed the way we think about software development. There are even things like low-code or no-code systems, which have simplified software development.
But the holy grail of software engineering has still remained building long-lasting software and that is still insanely difficult!
Read The Entire ArticlePlanning to launch a new website? A new blog? Sounds like a good idea but do you really know what you should think about first? No, not the website design, not the website look & feel and colour scheme, not your logo and branding, not even the content. The first thing you should be clear about, is the SEO, i.e. Search Engine Optimization.
I know it is counter-intuitive. Also trust me when I say that I was definitely not intending to create a dramatic beginning. Neither was I trying to bait you into reading this entire article. I was just stating facts. I was, in other words, saying that SEO trumps everything.
Read The Entire Article
In the lifecycle of any software, change is often costlier than anticipated. Because more often than not, it accumulates technical debt. When you buy something from somebody and don’t pay them back, you are in debt. Essentially, when you owe something to somebody, you are in their debt. As long as you are capable of paying them back in the near future, the debt is considered manageable. But if the accumulated debt exceeds your paying capacity, the debt becomes unmanageable.
Read The Entire Article
Have you ever wondered why in this era of automation and mechanisation, engineers still get to retain their jobs? Many job profiles have come and gone away. But even with so many advances in technologies across industries and the emergence of artificial intelligence and machine learning, why haven’t engineers been replaced yet?
The answer is quite simple, really. But before that, let us talk theatre! I am sure you have had the pleasure of watching a nice play or two so far in your life. Don’t say you haven’t, because OMG then you have definitely missed a thing! Moving on, by any teeny tiny chance, have you ever had the good fortune of watching two performances of the same amazing play twice? Yes? Then what was the one thing that struck you? I’ll tell you what struck me the most. I once saw two back to back performances of a play within a span of hours. The day, the cast and the venue was the same. The only variable was the audience. What struck me most, was that both the performances or if I may call them renditions, felt so different! The scenes were same, characters were the same, dialogues didn’t change at all, still, I experienced the difference. If you allow me some liberty to get a little philosophical, I would say, such is life! Right? If somebody asks me to describe life, I’d say life is not like a movie, new and different is every single day, just like every different performance of a play!
Looping back to the current topic of deliberation, every engineering project is essentially a fresh endeavour. No matter how many similar projects you may have done in the past, the next project will always throw something unexpected at you. If you are into construction, no matter how many buildings you build, how many bridges you construct, the next project will still be a challenge! Because something or the other will change, maybe the soil conditions, maybe the water table, maybe the wind conditions, maybe availability of material; something will change that will catch you off-guard. If you are a software engineer and are tasked to develop an e-commerce application, no matter how many e-commerce apps you’ve successfully developed in the past, this one is going to be different! The products being sold may be different, the user profiles of customers may be different, the tech-stack or the deployment environment may vary.
Life is chaotic and dynamic. With knowledge, training and experience, engineers are the ones, who can bring some order to this chaos. No matter how much progress we achieve in life, the chaos will remain and we will keep needing the skilled folk who’ll manage it, with methods, systems and innovations. We’ll keep needing engineering minds!
Every big endeavour begins with one or more problems that need to be solved. When the daring Sheikh from UAE challenged his team to build the greatest hotel ever built, the first problem they had to solve was creating an artificial island that would withstand the tremendous pressure from the sea waves and at the same time serve as the foundation of a 300 meter tall structure. Then there was also this not so little problem about the wind. The shape of this structure was that of a sail; just imagine the amount of wind a 300 meter tall sail would have to withstand! Needless to say, with such a humungous ask at hand, the team had to start somewhere. The engineers began their work. First they figured out how to build the island that met the brief. They created a rock-bed and then topped it up with a mesh of perforated concrete blocks. This artificial sponge prevented flooding and absorbed the pressure of waves. Then they addressed the problem of the structure stability. This hotel in Dubai, called Burj al Arab, is usually known for its opulence and luxury, but in actuality, is an engineering feat!
Another lesser known example comes from India, where the Government wished to build a 750 Km long railway line covering the Western coast of India. The problem was that the entire coastal strip was a mountainous region with lots or rivers, ravines and valleys sprinkled in between. If this wasn’t enough, the region was also prone to landslides, especially during the monsoon season. Moreover, the track had to be near flat and a minimum radial curvature of 1.25 Km had to be maintained to enable faster train speeds. After commissioning, the engineers began their work and completed it in record time! They ended up creating one of the most scenic train routes in the world comprising of 2000 plus bridges and 91 tunnels! It included the longest railway tunnel in Asia.
Examples are aplenty. Truth be told, in reality, very few problems get “absolutely” solved. You don’t always need an absolute solution, you need a realistic or a workable solution to get the job done. Engineers are the ones who have an eye for the right approximations and the know-how to achieve the required trade-offs!
A problem, which is complex from one perspective, suddenly appears to be simple from another perspective, and then the world is disrupted. In the early days of the Internet, ranking the results of search engines was a complex problem, which nobody really had the solution to. Two engineers from Stanford University, simply saw this problem from a different perspective and then an elegant solution unraveled, known as PageRank. They figured that simply the number of backlinks to a webpage was a good enough approximation of its popularity. This became the basis of the Google search algorithm and we all know, the world has never been the same since.
The ability to bring predictability in chaos, the skill of conjuring up practical solutions to seemingly impossible challenges and the knack of seeing things differently, have made engineers invaluable since time immemorial. The contribution of the engineering profession to humankind is unparalleled and yes that is exactly what differentiates engineers from the rest! Do you agree?
Rather than bigger and spaced out software releases, companies have lately gotten better results with quicker incremental software releases. Shorter release cycles help get user feedback earlier. Effects of bugs & erroneous decisions also stay contained. In short, good features get milked faster and the damage from bad things gets rectified sooner. Overall, resulting in better software delivery. CI / CD, or the continuous integration / continuous deployment, takes this approach to its extreme.
This article discusses the possibility of using the CI / CD pipeline to develop NPM packages. There isn’t a generic YES / NO answer to this question. The answer varies case to case. However, it is important to understand what goes in making this decision, such as the various factors and trade-offs involved.
CI / CD takes the agile software methodology to its extreme. Agile enabled us to release weekly, bi-weekly or monthly. With CI / CD, we can now have daily software releases.
The base of CI / CD, is automation. In a typical CI / CD project — (1) before writing new code, the developer ensures that all the prior code is covered under automated unit tests, (2) tester relies on automated integration tests and (3) the devops engineer automates the deployment mechanism.
(1) and (2) ensures that the newly code written has no unintended consequences. (3) eliminates manual deployment errors, enables canary releases and graceful rollbacks.
Faster User Feedback — CI / CD release cycles are extremely short, hence user feedback is received earlier.
Lesser & Isolated Errors — Because of automation in testing and deployment, possibility of errors reduces. Faults, if any, can be quickly isolated and resolved.
Quality delivery — CI / CD brings control, efficiency, transparency and accountability to software development, ultimately resulting in a good quality delivery.
Increased Complexity — CI / CD is automation-driven, hence requires highly skilled resources and sophisticated instrumentation. In a project that is based on multiple tech-stacks, things can get complicated quickly.
Operational Difficulties With Larger Teams — CI / CD works well with smaller teams (less than 10 people). For larger projects with larger teams, work needs to be efficiently broken down and handed over to smaller teams for execution. This presents operational difficulties.
Let us take an example of a React package, called react-upload-to-s3. It is an all-in-one react-only component for uploading images, documents and videos to AWS S3.
CI / CD pipeline for such a project would look like the following
There are several factors, which need to be considered to decide whether CI / CD is the right approach for your npm project.
New project or on-going project? — If your project is an on-going project, then setting up CI / CD is going to be a relatively difficult task. It is not impossible, but the migration from a traditional deployment pipeline to a CI / CD pipeline, will be slow and your interim costs would be higher. If you are starting from scratch on a new project, then you can definitely go for a CI / CD approach!
Resourcing — CI / CD requires highly skilled developers, testers and devops engineers. Do you have such people on your team? Your developers should be able to write unit tests, ideally covering 100 percent of the code. Your testers should be experienced in writing automated tests for integration and regression. Your devops team should be able to automate the deployment process under varying load conditions, for varying scalability requirements and against different business scenarios & asks. If you have such a team, then go for it! If you don’t, first build this capability by getting such people on-board, and then considering CI / CD.
Size of the project — How big is your project? In terms on resourcing, how many developers and engineers would you need to complete the project? It is observed that CI / CD gives good results with smaller teams. Typically less than ten people per team is perfect for CI / CD. Is your team size going to be bigger than this? If yes, you’ll have to break down the work in smaller sized teams. But mind you, breaking down would then further give rise to operational difficulties. What could they be? Do you have a grasp? How do you plan to iron them out? You will need to thoroughly answer these questions before deciding to take the CI / CD approach with larger projects. For smaller project, CI / CD is a no brainer.
Software architecture — Automation testing is the foundational principle of the CI / CD approach. Is your code testable? Is the separation of concerns done properly? Do the unit tests cover almost 100 percent of the code? How have you architected the client and the services? Are the interfaces clearly defined and documented? Do you have a test suite ready for the backend services as well?
There are many other factors as well, but most of them are the derivatives of these afore-mentioned important factors.
If you are starting off a new project with a skilled team of 10 people or lesser, then CI / CD is a no brainer for you. Just got for it! Not only will it improve your software delivery, it will also help you optimize costs long term! In all other cases, understand the above decision making factors from the context of your project requirements and make an informed judgement.