My Experience Contributing To The Cartography Open Source Project

Open-source contributions are a great way to help the tech community. They are also one of the best ways out there to learn and build your skills. Open-source software are publicly available source codes of a product that you can use and modify for free. Chromium, Mozilla Firefox, the Linux Operating system are some of the most popular open-source software.

Recently, we at Cloudanix made open source contributions to the Cartography project by Lyft. We contributed to the pre-existing AWS module by adding support for KMS and API gateway. Since the Cartography project did not have support for Microsoft Azure, we started from scratch to add that capability to Cartography. We added support for Azure services like CosmosDB, SQL, Storage, and Virtual Machines.

After 6 pull requests and dozens of review comments, here is what we thought the benefits of contributing to open source are and what we learnt.

Benefits Of Contributing to Open Source

• Improves Coding Skills: From a personal point of view, I can most definitely say that my Python coding skills have improved drastically after working on the Cartography project. I have got the habit of writing docstrings for every function. I have learned to write valuable comments in the code wherever necessary, adhering to the PEP8 style and unit testing.
• Exposed to new trends and treasures in technology: Personally, before contributing to Cartography, I had no idea tools like that existed. Furthermore, I had never worked on anything related to AWS or Azure. After contributing to Cartography, I have started getting quite comfortable with these tools and more, like Terraform.
• Networking and Peer Recognition: Working on an open-source project provides recognition. Contributing to the same project often makes you recognizable in the community. This is very important in terms of your career and your learning curve.
• In-depth knowledge of the project: You can make contributions like opening issues, adding fixes, adding features, and updating documentation. This will give you an in-depth knowledge of the project you are working on. So, suppose in the future, people ask questions about some functionality of that project on StackOverflow. In that case, you can be their go-to person.
• Improving software on a user and business level: I will talk thoroughly about this in the "Why We Chose The Above Modules to Contribute?" section.
• Valuable experience in collaboration and teamwork: Contributing to open source is a good crash course to collaboration in technology. You are working on a code written by someone else and trying to contribute to it. In this process, you will end up communicating with the original contributors of that project. Knowing what their expectations are, understanding the project's roadmaps, fixing open issues, and then communicating over Slack/Discord is an excellent step towards embedding a sense of teamwork in yourself.

How Did We Approach Contributing to Lyft's Cartography?

So before we contributed to Lyft, we knew our contribution would be adding AWS support for KMS and API Gateway and adding Azure support for CosmosDB, SQL, Storage, and Virtual Machines. However, before we began writing the code for this, there were some steps that we took.

• Learning about Source Control: The first step to any open source contribution is to learn about version control systems. The majority of the open-source projects use a version control system. Version control systems are tools that help with merging new code into the project or the main repository. The Cartography repository is hosted on GitHub. So, in this case, we had to get a good command over GitHub. This included knowing operations like -
  1. Forking the project,
  2. Making commits,
  3. Mastering markdown,
  4. Committing your code changes, and
  5. Opening pull requests.
• Assessing My Skills: Before moving ahead with the contribution, we read the Lyft Cartography project's code. This exercise was done to explore the skills we would need directly related to writing the code. The project needs you to have a strong command over the Python programming language as the entire codebase is in Python. Since we would be adding modules related to AWS and Azure cloud services, we needed a basic understanding of these cloud services. Furthermore, we had to get familiar with Python's AWS SDK documentation, known as Boto3, and Azure documentation for CosmosDB, SQL, Storage, and Virtual Machines.
• Understanding the existing code: Next, it was time for us to explore the existing codes. We used the AWS intel module codes as references to build the KMS and API Gateway modules. We preferred not to stray too far from the already existing code style and workflow so that all the AWS module codes are streamlined in a particular manner. This approach helped when we added the Azure module and support for Azure services like CosmosDB, SQL, Storage, and Virtual Machines.
• Writing the documentation: Contributing to open source doesn't just involve adding code. In our case, we were adding support for entire modules of the cloud services to the project. Hence we had to be comfortable with markdown and writing good and efficient documentation.

Why Did We Choose The Above Modules to Contribute?

Initially, we came across the Cartography project as users. While Cartography supported most of the services we needed, we needed Cartography's functionalities for AWS KMS and API Gateway and resources of Microsoft Azure's CosmosDB, SQL, Storage, and Virtual Machines. This is one of the key benefits of open-source. If we want to add new functionality as per our need to an already existing project, we can do so. Since Cartography is an open-source project, we get to contribute features that meet our business needs. At the same time, Cartography gets those features that other users can use. We assessed our business needs and concluded that we would like feature support for the services I mentioned earlier. Hence, we went ahead and contributed that to the Cartography project!

What Did I Learn?

For me, the PRs I opened for Cartography were my first open-source contributions. The code reviews helped me tremendously to be a better developer.

• Writing Clean and Readable Code: My college assignments would always end up being spaghetti. It would not be very readable and barely clean. One of the things I had to adhere to while writing code for Cartography was the PEP8 style. Removing unnecessary spaces, adding trailing commas, not having more than 120 characters in one line of code, and so on. And while I started adhering to it, I could also see the benefits of it. It actually made my code readable - even for me. I found it easier to navigate through the code and be more efficient at writing and debugging the code in general. Another practice followed for the Cartography codes was writing type hints for every function. This is again one of the methods to make your code understandable.
• Writing Efficient Code: In the Cartography code, I would gather the resources in one list and then iterate over them and ingest them into neo4j. Little did I know this was not the efficient way to do it. I got a code review by Alex Chantavy saying that I should re-write those parts of the code by using UNWIND for the list of resources inside the Cypher queries instead of looping through them. He further explained to me the reason behind this - "UNWIND makes it a lot faster and helps with potential memory issues." Another feedback from Ramon Petgrave was using list comprehension rather than a list(map(...)). I had used list(map(...)) for the entirety of the code. And this is a big thing - for small codes, you don't care much about efficiency, but for big projects like Cartography, the code's efficiency is vital. I was not very well aware that list comprehension is a faster alternative to list(map(...)). This is one of the benefits of open source as well - you get constructive feedback on your style of code from the best in the industry, and you learn new things at the same time.
• Handling Exceptions: Handling exceptions was something I was familiar with and a practice that I did follow while writing my Cartography codes. However, for bigger projects, I had to be specific about what the exception was. Handling exceptions in big projects like Cartography is very important. Ramon Petgrave's review of my Azure code made me realize the importance of being specific during exception handling.
• Testing: Lastly, writing tests for the code I wrote was a big task. I am not so familiar with testing. In college, we barely practiced test-driven development. However, I realized it is a necessity to do that in the real world. Writing integration tests for the code I wrote was a good way to assure myself that I am not breaking anything in the already existing codebase.

Conclusion

Lastly, I learned a lot while contributing to the Cartography open source project. Open source contribution really does give you a feeling of satisfaction, and I highly recommend you to contribute to open-source projects.

A huge thank you to the entire team at Cloudanix for providing us this opportunity to contribute to open-source. Thanks are in order to my mentor - Purusottom Mupunu, who contributed to this project with me by laying an initial foundation and then guiding me with my PRs and the thousands of questions I would bug him with. We also want to extend our sincere gratitude to Alex Chantavy and Ramon Petgrave. Their code reviews helped us become better developers than we were before we contributed to Cartography.

Note: This post was originally written by me and was published on Medium on the Cloudanix page.