Should You Commit the Vendor Folder in Go?

Should You Commit Your Vendor Folder in Go

The post Should You Commit the Vendor Folder in Go? first appeared on Qvault.

The answer to the age-old question of, “should I commit the vendor dependencies in my Go project to source control?” is “almost always“. As an FYI, we here at Qvault use Go for all of our backend work, and we always commit our vendor folders. Let’s take a look at the reasoning behind my claim that committing dependencies is ideal.

What Is the Vendor Folder?

If you are coming from Node.js land, Golang’s vendor folder is basically the same as Node’s node_modules. It is a folder found at the root of a module that stores a copy of all the code the module depends on. The code is used to compile the final executable when the go build command is run. As you can imagine, at the heart of the “should we commit vendor?” discussion is the problem of repo size.

node_modules is infamous for its large size.

node_modules is the heaviest object in the universe

As a result, conventional wisdom in the Node community is to add node_modules to the .gitignore file in order to save space. After all, a quick npm install (or in Go’s case go get) will grab everything right?

Yeah. Yeah it will. Most of the time.

Reproducible Builds

npm Err! 404 'left-pad' is not in the npm registry

The error code above famously plagued the developer world because developers were too lazy, or perhaps too sloppy, to write a few simple lines of code. Had a copy of the dependency been committed to all the projects that depended on leftpad then nothing would have been broken when the package was removed from NPM.

module.exports = leftpad;
function leftpad (str, len, ch) {
  str = String(str);
  var i = -1;
  if (!ch && ch !== 0) ch = ' ';
  len = len - str.length;
  while (++i < len) {
    str = ch + str;
  }
  return str;
}

Luckily, up to this point, the Go community has been much more rigorous about upholding the virtues of keeping dependencies to a minimum. When dependencies are kept to a minimum, it’s easy to commit the entire vendor folder without incurring the huge data cost that the average node_modules folder would demand.

Final Verdict

Just like the flow chart at the beginning of the post outlines, if you don’t have an insane amount of dependencies, just commit those dependencies! You save yourself the headache of worrying about their source repos being deleted or not having network access when you build your project. On the other hand, if you do have a metric shitload of external code, maybe you should work on cutting the fat.

Thanks For Reading!

Take computer science courses on our new platform

Follow and hit us up on Twitter @q_vault if you have any questions or comments

Subscribe to our Newsletter for more programming articles



source https://qvault.io/2020/11/16/should-you-commit-the-vendor-folder-in-go/

Comments

Post a Comment

Popular posts from this blog

Why is Exclusive Or (XOR) Important in Cryptography?

Image
If you are getting into cryptography, or just trying to understand the fundamentals, you may have noticed that the exclusive-or operation is used quite often, especially in ciphers. What is XOR ( ⊕ )? XOR, or “exclusive or” operates on binary data. It returns true if both of its inputs are opposites (one false and one true), otherwise, it returns false. An example in go code would be something like: func exclusiveOr(a bool, b bool) bool { return a != b } XOR Cipher – The Perfect Cipher The XOR operation can be used as a simple cipher for encrypting and decrypting messages with a single key. This is known as symmetric encryption. It is interesting to note that if: The key is the same size as the message The key is kept secret and generated truly randomly Then the cipher is impossible to crack. This is known as a one time pad . However, a simple XOR shouldn’t be used in production due to the key length needing to be too long to be practical. Cipher Example A...
Read more

Base64 vs Base58 Encoding

Image
By Lane Wagner –  @wagslane  on Twitter Base64 is one of the most popular encoding formats for representing data. Have some binary data? Base64 encode it for convenient readability and parsing. Base58 is just another encoding format (with 58 characters instead of 64, and has gained popularity largely due to Bitcoin and other cryptocurrencies. When it comes to data encoding, there is typically a trade-off made between: Human Readability: Do humans have a good idea of what is being represented at a glance? Efficient data compression: How many bytes are used to represent the same data? Let’s rank Base58, Base64, and ASCII encoding against each other using these metrics! First: A Note on Binary All data is stored in a raw binary format on computers. These encoding formats (Base64, Base58, and ASCII) are just different ways of reading and writing binary data. For example, in Base64 the binary code 000000 represents the letter A , but in ASCII the binary 00000000 repres...
Read more

(Very) Basic Intro to Hash Functions (SHA-256, MD-5, etc)

Image
This is going to be a basic introduction to hash functions. I will assume most of my audience is here to gain an understanding of why hash functions are used and the basic idea of why they work. My goal is to explain it in a general sense, I will be omitting proofs and implementation details and instead focusing on the high-level principles. Why Use A Hash Function? Hash functions are used all over the internet in order to securely store passwords, find duplicate records, quickly store and retrieve data, and more. For example,  Qvault  uses hashes to extend master passwords into private encryption keys. A longer list of uses here:  https://en.wikipedia.org/wiki/Hash_function#Uses I want to focus on several important features of hash functions, arguably the most important features. Hash functions scramble data deterministically No matter the input, the output of a hash function always has the same size The original data can not be retrieved from the scrambled d...
Read more