The Challenges of Open Source Software Dependencies on Your Code When Porting to Linux on Z

Photo by Ethan Sykes on Unsplash

Navigating the Challenges of Upstream Dependencies

In my job at IBM, my team works to help ISVs who are porting software to run on Linux on Z. On any given day, a team taking on new platforms finds a unique set of challenges, particularly when it involves open-source software projects. When they also take on porting to Linux on Z, they are also having to look at those upstream dependencies.

For those not familiar, a quick look at upstream and downstream open source dependencies: Upstream open-source dependencies are software components or modules that a particular open-source project relies on or integrates into its codebase. Downstream open-source dependencies are software components or modules that rely on or utilize a particular open-source project. So, the things you use from open source in your software (a charting package, for example) upstream is what you need to put in your code instead of writing your own. Downstream projects from your code would use your code in their own in lieu of building it themselves. So the up or down is whether your open source code is using something else or being used. Another way is to literally think about a stream. Is the water coming from somewhere else to you (upstream) or already flowed past you (downstream)?

When considering the porting of software to run on Linux on Z, a robust and reliable platform, one must address the intricacies of upstream dependencies. This can be a huge part of the effort required if these open source communities did not design with z in mind and their code is not able to be easily coded to run on z. When taking on this challenge, you might have to be prepared to fork the code, take on the challenge of porting, and be willing to help the community adopt the code and take on some role in keeping it current. In some cases, you may be forced to do it all yourself if you cannot find any traction in getting it adopted by the open source community. Thankfully, the IBM zSystems open source community is very robust, very large, and very skilled. There are people out there (many who are IBMers) who actively participate in open source projects and do so with z specifically in mind.

In this blog post, we will explore the significance of upstream dependencies in the context of porting open-source software to Linux on Z and discuss the associated issues and strategies for managing them effectively.

Understanding Upstream Dependencies: Upstream dependencies are external components or modules that an open-source project relies on. These dependencies, such as libraries, dlls, frameworks, or APIs, are developed and maintained separately by external teams or open-source communities. Porting open-source software to Linux on Z introduces challenges associated with these upstream dependencies. At IBM, we have a team who takes on some of these challenges to help our ISVs with open source dependencies.

1. Compatibility and Platform-Specific Considerations: One of the primary issues when porting open-source software is ensuring compatibility with upstream dependencies. These dependencies may have been primarily developed for and tested on other platforms, requiring adjustments to work seamlessly on Linux on Z. Architecture-specific differences, endianness, or alignment concerns must be addressed to ensure compatibility and optimal performance. Modifications may be needed in the codebase, build scripts, or configuration files to handle platform-specific intricacies. Another option may be — especially when you cannot port the dependency — to replace the dependency in your product with something else. We recently have started a porting project that has had to do just that and actually replaced the upstream dependency for all platforms with what they now view as a better solution.
2. Availability and Support: Another challenge is the availability and support of upstream dependencies for Linux on Z. While many popular open-source projects have made efforts to support diverse platforms, some dependencies may not have undergone extensive testing or received updates for Linux on Z. This may lead to limitations in terms of feature availability, bug fixes, or security patches. Collaboration with upstream projects becomes crucial to encourage support for Linux on Z and address any compatibility issues specific to the platform. Your company will definitely have a vested interest in becoming a part of these open source projects and collaborating especially to get the zSystems support you desire.
3. Community Collaboration and Communication: Engaging with the upstream open-source community is paramount for successful porting. Collaborating with the developers, maintainers, and contributors of upstream dependencies fosters a better understanding of their roadmap, plans for platform support, and any ongoing efforts related to Linux on Z. Communicating the porting objectives, challenges faced, and feedback regarding compatibility issues can promote collaboration, community-driven solutions, and timely updates for Linux on Z support. Your goal to push for is parity across the board. The project supports z just like any other platform.
4. Rigorous Testing and Validation: Thorough testing and validation are essential when dealing with upstream dependencies during the porting process. Modifying or adapting the open-source software to work on Linux on Z can introduce unintended side effects or compatibility issues with upstream dependencies. Regression testing, integration testing, and performance benchmarking become critical to ensure that the software and its dependencies work harmoniously on the new platform. Collaboration with the upstream projects and leveraging their testing frameworks or documentation can aid in this effort. In our case, I have been told by our team that does open source porting that having a community that identifies problems early and reports it to them is something that helps them to participate in open source porting projects effectively to know what needs fixing without spending all kinds of time trying to identify IF there is any problem which doesn’t scale well at all.

Porting open-source software to run on Linux on Z offers tremendous opportunities for leveraging the platform’s power and reliability as well as positioning to leverage the other strengths of the platform (security, resiliency, scalability, etc.) However, managing upstream dependencies is a critical aspect of this process. Addressing compatibility challenges, engaging with the open-source community, and conducting rigorous testing are key strategies to overcome the issues associated with upstream dependencies. By fostering collaboration and actively participating in the open-source ecosystem, developers can navigate the challenges, ensure compatibility, and contribute to the growth of open-source software on Linux on Z and in turn, ensure their own company’s product will have a strong foundation to build upon for years to come.