My first series of casino classes recently ended, just in time for the holidays. However, along with some other outlier students, I didn’t want to lose momentum over Christmas break, so I started venturing into the wild trying to find figures within reach to self-teach. Along the way, I started to see patterns between all the figures I saw appear most, and I realised that I could double the amount of figures I knew just by understanding three new phrases: one phrase to learn vacilalas, two phrases to learn sencillas. And it worked.

In the last 2 months, I’ve begun taking Cuban salsa classes at the local designated club. I’ve gotten so hooked that I genuinely can’t remember what occupied my thoughts and free time in the 23.75 years prior to this, and of course, like all excellent students, I’ve been taking notes.

Here’s a fun fact about me: I graduated with the joint title of engineer (ir.) and master of science (MSc) in computer science, without anyone ever teaching me about sorting algorithms. It was only a couple months after I was hired as a PhD student that I taught myself sorting algorithms, and that was because I had to teach them to bachelor students in turn the week after.
Inevitably, while studying other data structures, I also encountered the concept of amortisation. I vaguely remembered coming across its Wikipedia page once and taking away that it meant something like “proving that something is fast via black magic”. And yet, it really is not that deep.

“Why would you even care about my future well-being? Why would you not just block me everywhere and pretend I no longer exist? My well-being has no benefit to you. I could die and it wouldn’t affect you.” This is an approximation of what my first girlfriend conveyed to me when I parted ways with her. It’d take me a year to unpack that response.

There are (at least) two claims that have never made sense to me ever since I heard them. One is that double-entry bookkeeping, where you track the capital inside a company by keeping two lists of money amounts that add up to the same total, is actually useful and not a formality that is 50% redundant. The other is that fractional-reserve banking (FRB), whereby a bank is not forced to have at least as much cash on hand as the combined total of all account balances open at that bank, allows banks to create cycles of infinite money. The former is true but hard to believe at first. The latter is false but hard to disbelieve at first. Both become apparent when considering the typical claim about FBR through the lens of double-entry bookkeeping.

The promise of DeBERTa is that it does away with absolute positional embeddings added at the start of the transformer encoder, and replaces it instead with an attention mechanism that takes into account the relative distance between tokens when doing attention, in every head of every layer. Since absolute positional embeddings are the only part of a transformer whose dimensionality is related to the length of the input, would taking it out mean DeBERTa can process an infinite amount of tokens at once? Let’s analyse.

The famous roberta-base HuggingFace checkpoint is a serialised version of a RobertaForMaskedLM model, consisting of a roberta field and an lm_head field. Yet, despite this, you can still call .from_pretrained("roberta-base") on RobertaForTokenClassification and get an object that has a roberta field with exactly the checkpoint’s roberta weights, but a head with a different architecture and randomly initialised weights. Even more strikingly, you can call .from_pretrained("roberta-base") on RobertaModel, which is what normally sits inside that roberta field and consists of the fields embeddings and encoder, and somehow it can still match all the relevant checkpoint weights to the correct fields. Ever wondered how that’s done? Here’s how.

I recently watched a video online of someone complaining about the existence of millionaires and billionaires. I commented with the realistic and optimistic message that anyone has the ability to become a multi-millionaire, even if just through supplying a silly backend commodity like toilet rolls. Then, all hell broke loose.

Knowing how to build mobile apps is likely going to be a skill that will stay relevant for the foreseeable future. I’ve had some ideas for mobile apps in the past, and wanted to make sure that if I was going to learn to develop them, it had better be in a framework that allows having one code base and yet runs on both iOS and Android. We’re in luck: Google’s Flutter framework, built on top of the Dart language, does exactly that. Here are my notes learning it.

At the time of writing, I’ve publicly released a handful of Python packages (five, to be precise: Fiject, BPE-knockout, TkTkT, LaMoTO, and MoDeST) covering various parts of the NLP research workflow, with more on the way. It took me a bit to learn how to set up Python code as a package (also inaccurately called “library” or more accurately called a “module”), and as I later discovered, it’s not so trivial to have one custom package be installed automatically upon installing another custom package, especially when you are the author of both and are already using a working version. Let’s dive straight in!