Category Archives: opensource

How to make your code on downstream ?

Leave a reply

Nowadays most of my projects have been using opensource or worked based on opensource. If we just needs to use it, I think it would be a good situation for you. However we usually have projects that keep the opensource over years for your products. So if you have to hack a lot of modules inside of opensource code, it can be a nightmare when you rebase current source base against the latest opensource after months or years. In this article I would like to share some of my experiences on how to make a downstream patch when we work using opensource.

  1. Try to contribute your patch to the opensource project as much as possible

    • I think this is the best way to reduce our heavy burden that we should maintain in your downstream source code. Even if the opensource project you’re using is being developed fast, you will often face many conflicts during the rebase because it’s likely that original code or architecture have changed frequently in the meantime. To avoid the conflict, it would be best if you contribute your patch to the opensource project as much as possible.
    • Below documents are a good example to explain how to contribute your code to the opensource project.

  2. Make your downstream port

    • We know well that #1 is the best way to reduce our downstream patches though, it is often hard to keep because downstream patches are often too hacky or unstable. Even if you submit a downstream patch to upstream, you might get many review comments or objections from the opensource maintainers. In such case, what else can you do ? In my experience, it was important to separate our downstream implementation from the original code. For example, we can make new TriangleFoo.h/cpp files instead of original Triangle.h/cpp files, then we can use them through the modification of few build scripts. 
      1. Figure class
 class Figure {
  public:
      virtual int calculateSize();
 }
 
2. Triangle class
 class Triangle : public Figure {
  public:
      virtual int calculateSize() override;
 }

 Triangle::calculateSize() {
     return width * height / 2;
 }

3. TriangleFoo class
 class TriangleFoo : public Figure {
     public: virtual int calculateSize() override;
 }
 
 TriangleFoo::calculateSize() {
     return new_width * height / 2;
 }

4. Build script. In this example we use cmake,
 list(APPEND Figure_SOURCES
     Figure.cpp
     Triangle.cpp
     TriangleFoo.cpp
 )
 
 list(REMOVE_ITEM Figure_SOURCES
     Triangle.cpp
 )

      We can avoid some conflicts in the next rebase with the latest opensource in the Triangle.h/cpp files. However we still need to modify the TriangleFoo.h/cpp if Figure.h is changed. For example, when a new parameter is added or a return type is changed.

  3. Use #if ~ #endif guard

    • When we only need to modify few lines or just to change logic inside a function, we can use #if ~ #else ~ #endif guard. The guard can help us to know what codes were added by us or modified by us. Besides it might be help us to check easily if the downstream patch generated side effects through turning it off. In my previous projects, most of issues have come from downstream patches because they lacked code review, missed test cases, or were too hacky against original architecture. In such cases, you can check the issue just by turning the guard off. However, if you use #if ~ #endif guard in many places, the usages can mess your code up. So I’d like to recommend you use it only when you really need. 
      Triangle::calculateSize()
{
#if defined(DOWNSTREAM_ENABLED)
    return new_width * height / 2;
#else
    return width * height / 2;
#endif
}

  4. Try to make a patch per a feature

    • As you may have experienced before, it is very hard to implement a feature with a commit. Even though you succeed in implementing a new feature with a commit perfectly, you may face to touch the implementation again in order to fix a bug or apply new requirements again. In such case your git history will get messier and messier. It will make it difficult to rebase based on the latest opensource. To avoid it, you may have manually merged original implementation with the fixup commits reflected later. But there are two useful git commands for this case – git commit fixup and git rebase –autosquash.
        • git commit –fixup : Automatically marks your commit as a fix of a previous commit. Construct a commit message for use with git rebase –autosquash.
        • git rebase -i –autosquash : Automatically organize merging of these fixup commits and associated normal commits.

    • Example
      There is a good article to explain that explains method [1]. If you need to understand further, it would be good if you visit the URL. Let’s assume that we have 3 commits on your local repository.

      $ git log --oneline
  new commit1 (7ae79f6)
  new commit2 (9e4c1de)
  new commit3 (480ee07)
  previous commit (19c8abf)

      But if we just noticed that we missed to add a comment in commit2, it’s time to use --fixup option.

      $ git add [modified file]
$ git commit --fixup [new commit2's commit-id]
  (i.e. git commit --fixup 9e4c1de)

      Then you can clean your branch before merging it using - autosquash option.

      $ git rebase -i --autosquash [previous commit id]
  (i.e. git rebase -i autosquash 19c8abf)

 

Reference
[1] http://fle.github.io/git-tip-keep-your-branch-clean-with-fixup-and-autosquash.html