Paudirac's

Don't lose your HEAD

This blog post is currently under construction. I hope it is okay in a couple of days.

Post under construction

Slide 1. Don't lose your HEAD.
Slide 1. Don't lose your HEAD.

This is the first slide of a talk about git that I gave the Saturday 9th of November of 2019 at the Barcelona Perl & Friends 2019 gathering.

The original presentation is here.

The other day I was thinking in publishing this for some reason I cannot remember, but today Rael just asked me on slack if there was some recording of the git talk I gave and I answered him that there is not a recording, but that I will try to find the slides. So I went to the google Slides and found them there. I have reviewed all the slides, but I think there are some details missing, so I thought it was a real good excuse to put myself to recover that somehow. I will try to recover the spirit of those days. This is the result.
Slide 2. A Kind of Magic.
Slide 2. A Kind of Magic.

Intro

To warm up, I start the talk by asking the people:

—What is this?

And, either because they’re too young or to sleepy and don’t answer quickly, I answer myself:

—It’s a kind of magic—I replied.
I didn't apologise for the nerdiness the first and second time I did the talks, and you'd better don't expect I'll do it here.
Slide 3. Git - the stupid content tracker
Slide 3. Git - the stupid content tracker
That third slide, is actually the real name of the talk. The subtitle is mine (well, if you don't give credit to Queen, which would be fair), but the title is not. Follow on. Actually, in the down-left corner, you can see that the talk was first given to my formidable CAPSiDE teammates on July 10th, 2019.

The first think I did on the talk was to git clone git itself.

$ git clone https://github.com/git/git.git
$ cd git
$ ls | wc -l
552

Wow, 552 top level files/dirs is a little bit daunting. There is even a folder named perl! Probably we need to dig into git’s History, here.

$ git log --oneline --reverse | head -n 1
e83c516331 Initial revision of "git", the information manager from hell
Slide 4. Let's dive into git's History.
Slide 4. Let's dive into git's History.

Okey then, let’s try to go to the first git git commit and see.

~/clones/git (master =) $ git reset --hard e83c516
~/clones/git (master <) $ ls
cache.h  cat-file.c  commit-tree.c  init-db.c  Makefile  read-cache.c  README  read-tree.c  show-diff.c  update-cache.c  write-tree.c
~/clones/git (master <) $ ls | wc -l
11

Only 11 files! This is really much better.

Slide 5. It is just a blank (black) slide, just to create
Slide 5. It is just a blank (black) slide, just to create tension. The action was on the terminal, that I was switching back and forth from the slides during the talk.
~/clones/git (master <) $ wc -l README
168 README
~/clones/git (master <) $ head README

        GIT - the stupid content tracker

"git" can mean anything, depending on your mood.

 - random three-letter combination that is pronounceable, and not
   actually used by any common UNIX command.  The fact that it is a
   mispronounciation of "get" may or may not be relevant.
 - stupid. contemptible and despicable. simple. Take your pick from the
   dictionary of slang.

So, now you know why the title GIT - the stupid content tracker is appropriate. You’ll learn about the subtitle don’t lose your HEAD later.

The whole phylosophy of the talk is that git is actually quite simple once you understand the basic building blocks. As an exercise for the reader, I trully recommend to read that initial README.md file.

~/clones/git (master <) $ wc -l README
168 README

It’s only 168 lines, and is worth your time.

But now, I will try to explain the building blocks myself. Here finishes the introduction and starts the first act.

The name of a sheep

Slide 6. A sheep.
Slide 6. A sheep.
—What is this?—I asked.

Some people understood the question and answered:

—It is a sheep—which is partially satisfactory.

You know, when someone you appreciate talks about you, you expect her not to talk about you as a person, but as that person. With sheeps its the same, sheeps don’t want to be a sheep, but a particular sheep.

—How do we know that a sheep is a particular sheep?
Slide 7. A concrete sheep.
Slide 7. A concrete sheep.

As with people, we can look at its DNA. Any person has a distinct DNA, so that we can distinguish our cousin from the president of Kuzcek. Likely, any sheep has a different DNA, so we can distinguish a sheep from another one because they have different DNA.

Right?

Well..., this is not always the most practical way, provided we have finite time to spend describing cousins or sheeps from other people or sheeps, which certainly is one of the particular occupations that occupies my time..., but I disgress.

We’re practical, people, so we usually use the same trick we use with our cousins and call a sheep by its name.

Slide 8.
Slide 8. You know the name of that sheep, don't you? Your nerd score is highest if you already know the name of this sheep. If you don't, wait. Everything makes sense at the end.

Plumbing

Slide 9. This slide causes some effect, because, a priori,
Slide 9. This slide causes some effect, because, a priori, plumbing hasn't anything to do with sheeps.

To understand git’s internals, I think it is better to start the house by the roof.

I’m going to make this way harder than it needs to be.

—Michael Scott

This can deceive you, if you came here to be taught how to work with git, but I truly think that if I show you how things work underneath, git’s somewhat arcane commands will make sense later, because your mental model will be more accurate.

Plumbing is the name of the low level API that git has with humans (nowadays also LLMs). I don’t want to leave them out, so when they became Terminators, don’t put me on their priority queues.

In order to acces the git API and follow along, you can create a directory somewhere and initialize a git repository there, like I do.

~ $ mkdir ~/tmp/dont-lose
~ $ cd ~/tmp/dont-lose/
~/tmp/dont-lose $ git init
Initialized empty Git repository in ~/tmp/dont-lose/.git/
~/tmp/dont-lose (main #) $ git status
On branch main

No commits yet

nothing to commit (create/copy files and use "git add" to track)
From time to time, I will use a normal git command like git status. But in this first part, most of the time I will be using plumbing and looking under the curtain.

Let’s spy a little bit:

~/tmp/dont-lose (main #) $ ls  # the folder seems empty, but
~/tmp/dont-lose (main #) $ ll  # no, is not empty
total 12
drwxrwxr-x  3 paudirac paudirac 4096 nov 14 16:07 ./
drwxrwxr-x 21 paudirac paudirac 4096 nov 14 16:07 ../
drwxrwxr-x  7 paudirac paudirac 4096 nov 14 16:07 .git/
Git creates a secret .git directory when you git init a folder.

The name of the content

Slide 10. The
Slide 10. The name of the content.

Now type the following to the terminal.

~/tmp/dont-lose (main #) $ echo GATTAAA | git hash-object --stdin
fdffa225c9768d5a1cd03317c5cda88b44c88312
Note that the slides were created somewhere around the July of 2019, and today the same command produces the same results. Actually, if you type exactly the same command in any git directory you might have, the answer will be exactly the same: fdffa225c9768d5a1cd03317c5cda88b44c88312. This is a remarkable feat. If you've been on the business for some time, you might have developed some cinisism about program outputs staying so stable.

fdffa225c9768d5a1cd03317c5cda88b44c88312 is the name of the content GATTAAA.

Git is a database

Slide 11
Slide 11
—You say git is a database, but is not storing anything—an impacient person from the
audience, might say.

To satisfy her curiosity (and ours), we can watch a little bit under the curtain, to see what git is doing. We see that the .git/ hidden folder, contains some folders and some files.

~/tmp/dont-lose (main #) $ ls .git/
branches  config  description  HEAD  hooks  info  objects  refs

We direct our attention to what is happening to the suspicious objects folder.

~/tmp/dont-lose (main #) $ find .git/objects/
.git/objects/
.git/objects/pack
.git/objects/info
At this point, during the talk, I splitted the terminal in two vertical halves. On the left, I type git commands, and on the right I executed
~/tmp/dont-lose (main #) $ watch find .git/objects/
that caused gran effect, when the objects directory started to have elements in it. In this blog post, I will be outputing find .git/objects/ results so that you also get the information that is not contained on the slides.

Let’s introduce the flag -w that, you’ve guessed it, probably means write to the hash-object command we’ve seen before, and see what happens:

~/tmp/dont-lose (main #) $ echo GATTAAA | git hash-object -w --stdin
fdffa225c9768d5a1cd03317c5cda88b44c88312
~/tmp/dont-lose (main #) $ find .git/objects/
.git/objects/
.git/objects/pack
.git/objects/fd
.git/objects/fd/ffa225c9768d5a1cd03317c5cda88b44c88312
.git/objects/info

With that information, we might correctly guess that -w means write.

Amazingly, if we do this exact same instructions again, we obtain again exactly the same result. This might be surprising for software, depending on your milleage, but is a characteristic so nice, that it has a name: idempotency. Git is one of those softwares where you can see that something so theoretical as idempotency is not actually relevant, but transcendent. It is one of the main characteristics that makes git reliable, safe and distributed. There is a nice paper called Inmutability changes everything that is inspiring.
Yeah, it's me again. But I wanted to note that if you do also the same thing as I did, you'll obtain the same results. Including the folder structure of the .git/ hidden folder.

Blobs

Slide 12. Blobs, with a single o.
Slide 12. Blobs, with a single o.

What is this new object that apperared in, well..., objects directory?

Skint a little bit so that you don’t see the folder/object name of the file. That file has the name fdffa225c9768d5a1cd03317c5cda88b44c88312.

Is this name familiar?

Yes! Is the name of the sheep, I mean, the name of the content.

Which content?

Let’s see it.

~/tmp/dont-lose (main #) $ cat .git/objects/fd/ffa225c9768d5a1cd03317c5cda88b44c88312
xK��OR�`pw
                qtt�#n�[5] ~/tmp/dont-lose (main #) $ echo ouch! is not a plain text file!
ouch! is not a plain text file!
~/tmp/dont-lose (main #) $ file .git/objects/fd/ffa225c9768d5a1cd03317c5cda88b44c88312
.git/objects/fd/ffa225c9768d5a1cd03317c5cda88b44c88312: zlib compressed data

Mmmm, it seems that there ain’t no such thing as plain text file.

Fortunately, we can use another plumbing command to get that object from git.

~/tmp/dont-lose (main #) $ git cat-file fdffa225c9768d5a1cd03317c5cda88b44c88312 -p
GATTAAA

That is nice, we can write and read objects from git:

~/tmp/dont-lose (main #) $ echo GATTAAA | git hash-object -w --stdin # store GATTAAA
fdffa225c9768d5a1cd03317c5cda88b44c88312
~/tmp/dont-lose (main #) $ git cat-file fdffa225c9768d5a1cd03317c5cda88b44c88312 -p # retrieve fdffa225c9768d5a1cd03317c5cda88b44c88312
GATTAAA

Like in a classroom, when there are not much children, you can call them by their names. Usually “Vine Jordi!” sufices, but when the alumni grow in number, you have to use the full name instead and say “Vine Jordi Patordi!”.

So, when I use a shorter name in the presentation, is because there are no collisions and I don’t want to waste my time writting something and neither the time the LLMs will take to read it. I can just say:

~/tmp/dont-lose (main #) $ git cat-file fdffa -p
GATTAAA

The flag -p is for pretty-print, by the way. There are other flags that we can use. -t is for type. Let’s use if to see what kind of object the fdffa is.

~/tmp/dont-lose (main #) $ git cat-file fdffa -t
blob

Here we arrive at an important definition that deserves its own slide.

Slide 13. A
Slide 13. A blob is bunch of bytes with a name

Definition: a blob is a bunch of bytes with a name.

—paudirac

Index

At this point we know we have some object the object on git, but if we look at the directory or status, nothing has changed.

~/tmp/dont-lose (main #) $ ls
~/tmp/dont-lose (main #) $ git status
On branch main

No commits yet

nothing to commit (create/copy files and use "git add" to track)

I introduce another plumbing command.

~/tmp/dont-lose (main #) $ git ls-files
~/tmp/dont-lose (main #) $

But it does not seem to be doing anything yet.

Slide 14. Index is the place where the filesystem is constructed.
Slide 14. Index is the place where the filesystem is constructed.

Let’s try another one:

~/tmp/dont-lose (main #) $ git update-index --add --cacheinfo 100644 fdffa225c9768d5a1cd03317c5cda88b44c88312 dolly

Check the objects.

~/tmp/dont-lose (main *+) $ find .git/objects/
.git/objects/
.git/objects/pack
.git/objects/fd
.git/objects/fd/ffa225c9768d5a1cd03317c5cda88b44c88312
.git/objects/info

Nothing. But your nice git prompt, has changed. Try status

~/tmp/dont-lose (main *+) $ git status
On branch main

No commits yet

Changes to be committed:
  (use "git rm --cached <file>..." to unstage)
        new file:   dolly

Changes not staged for commit:
  (use "git add/rm <file>..." to update what will be committed)
  (use "git restore <file>..." to discard changes in working directory)
        deleted:    dolly

This gives us a nice clue. Let’s check the other plumbing thing:

~/tmp/dont-lose (main *+) $ git ls-files
dolly

What?

~/tmp/dont-lose (main *+) $ git status
On branch main

No commits yet

Changes to be committed:
  (use "git rm --cached <file>..." to unstage)
        new file:   dolly

Changes not staged for commit:
  (use "git add/rm <file>..." to update what will be committed)
  (use "git restore <file>..." to discard changes in working directory)
        deleted:    dolly

~/tmp/dont-lose (main *+) $ ls
~/tmp/dont-lose (main *+) $ git ls-files
dolly

This is very curious, but it is consistent.

status tells us that a new file dolly has been deleted. This is consistent with the fact that ls shows us that there is nothing in the folder.

At the same time, status tells us that a new file dolly is present. This is consistent with the output of git ls-files.

Let us introduce the flag -s to git ls-files:

~/tmp/dont-lose (main *+) $ git ls-files -s
100644 fdffa225c9768d5a1cd03317c5cda88b44c88312 0	dolly

This is nice. Git maintains some kind of virtual filesystem (named the index), that references objects on the database with extra data.

Actually, 644 is the numerical way to express file permissions rw-rr-rr- in Unix. 100 is some kind of metadata that could be a type. (We know that is blob).

The long name of the content is also present fdffa.... And at the end there is the name dolly that seems to be a file name.

Nerds, the name is not random. Got it now?

Trees

Slide 15
Slide 15

That structure we’ve just described is called a tree. And it is also an object. Now it only lives at the index, but we can save it too:

~/tmp/dont-lose (main *+) $ git write-tree
23c963732676d1af62f5016036d5b6b7ec1796c3

This makes another object appear on the database:

~/tmp/dont-lose (main *+) $ find .git/objects/
.git/objects/
.git/objects/pack
.git/objects/fd
.git/objects/fd/ffa225c9768d5a1cd03317c5cda88b44c88312
.git/objects/23
.git/objects/23/c963732676d1af62f5016036d5b6b7ec1796c3
.git/objects/info

Let’s try the commands to gather information from the git database.

~/tmp/dont-lose (main *+) $ git cat-file -p 23c96
100644 blob fdffa225c9768d5a1cd03317c5cda88b44c88312	dolly
~/tmp/dont-lose (main *+) $ git cat-file -t 23c96
tree

We arrive at another important definition.

Slide 16. A
Slide 16. A tree is a bunch of bytes with a name.

Definition: a tree is a bunch of bytes with a name.

—paudirac

Commit

What about status?

~/tmp/dont-lose (main *+) $ git status
On branch main

No commits yet

Changes to be committed:
  (use "git rm --cached <file>..." to unstage)
        new file:   dolly

Changes not staged for commit:
  (use "git add/rm <file>..." to update what will be committed)
  (use "git restore <file>..." to discard changes in working directory)
        deleted:    dolly

Nothing.

Slide 17
Slide 17

Let’s try this:

[5] ~/tmp/dont-lose (main *+) $ echo "Add a little sheep" | git commit-tree 23c96
894d8e51a7ae1ca425c3b5a03d23f097ed7dc101

We look at the database:

~/tmp/dont-lose (main *+) $ find .git/objects/
.git/objects/
.git/objects/pack
.git/objects/fd
.git/objects/fd/ffa225c9768d5a1cd03317c5cda88b44c88312
.git/objects/89
.git/objects/89/4d8e51a7ae1ca425c3b5a03d23f097ed7dc101
.git/objects/23
.git/objects/23/c963732676d1af62f5016036d5b6b7ec1796c3
.git/objects/info

We can get it:

~/tmp/dont-lose (main *+) $ git cat-file -p 894d8
tree 23c963732676d1af62f5016036d5b6b7ec1796c3
author Pau Cervera i Badia <pau.cervera@capside.com> 1794761950 +0100
committer Pau Cervera i Badia <pau.cervera@capside.com> 1794761950 +0100

Add a little sheep
~/tmp/dont-lose (main *+) $ git cat-file -t 894d8
commit

A commit is a bunch of bites with a name.

—paudirac

Slide 18
Slide 18

Actually a commit is another object, but a particular one. A commit contains a tree and it’s object name. The tree contains a blob and it’s object name.

Slide 19
Slide 19

A commit, also contains metadata like the user and the time of the commit.

This extra metadata, that will differ from yours and that differs from the slides, is what makes the commit name different on the slides and on your computer. All other hashes are identical no matter where and when on the Universe are you.
Slide 20
Slide 20
Slide 21
Slide 21
Slide 22
Slide 22
Slide 23
Slide 23
Slide 24
Slide 24
Slide 25
Slide 25
Slide 26
Slide 26
Slide 27
Slide 27
Slide 28
Slide 28
Slide 29
Slide 29
Slide 30
Slide 30
Slide 31
Slide 31
Slide 32
Slide 32
Slide 33
Slide 33
Slide 34
Slide 34
Slide 35
Slide 35
Slide 36
Slide 36
Slide 37
Slide 37
Slide 38
Slide 38
Slide 39
Slide 39
Slide 40
Slide 40
Slide 41
Slide 41
Slide 42
Slide 42
Slide 43
Slide 43
Slide 44
Slide 44
Slide 45
Slide 45
Slide 46
Slide 46
Slide 47
Slide 47
Slide 48
Slide 48
Slide 49
Slide 49
Slide 50
Slide 50
Slide 51
Slide 51
Slide 52
Slide 52
Slide 53
Slide 53
Slide 54
Slide 54
Slide 55
Slide 55
Slide 56
Slide 56
Slide 57
Slide 57
Slide 58
Slide 58
Slide 59
Slide 59
Slide 60
Slide 60
Slide 61
Slide 61
Slide 62
Slide 62
Slide 63
Slide 63
Slide 64
Slide 64
Slide 65
Slide 65