I replaced five indexes* on the relationships table with two**, improved query performance in at least one case, and cut the size of the database down by 11.4% (98MB).

Lessons (finally) learned:

1. You can have too many indexes. At best, this makes the database larger. In at least one case, however, this caused the query planner to pick a less effective index, which resulted in worse performance.
2. Unless you understand the data well, it is hard to know what indexes you are going to need up front. For example, on the relationships table, it's clear in retrospect that an index on the to_iri column has better selectivity than an index on the from_iri column—and, in fact, no index is even necessary on the from_iri column. For reasons of symmetry, I created both when I created the table. I'll go so far as to say, don't even create indexes until/unless you can analyze actual data. (Aside: the SQLite3 function likelihood is an excellent way to hint about that data to the query planner.)
3. Ordering results using the automatically assigned, monotonically increasing id primary key behaves identically to ordering by something like created_at, so order by id and save yourself an index on created_at.

#ktistec #sqlite #optimization

* The original five:

CREATE INDEX idx_relationships_type_from_iri_created_at
    ON relationships (type ASC, from_iri ASC, created_at DESC);
CREATE INDEX idx_relationships_from_iri_created_at_type
    ON relationships (from_iri ASC, created_at DESC, type ASC);
CREATE INDEX idx_relationships_type_to_iri
    ON relationships (type ASC, to_iri ASC);
CREATE INDEX idx_relationships_to_iri_type
    ON relationships (to_iri ASC, type ASC);
CREATE INDEX idx_relationships_type_id
    ON relationships (type ASC, id ASC);

* The final two:

CREATE INDEX idx_relationships_type
    ON relationships (type ASC);
CREATE INDEX idx_relationships_to_iri
    ON relationships (to_iri ASC);

i'm listening to the soundtrack for the legend of zelda: breath of the wild and some of the tracks (the battle themes, of course) still trigger me! it's amazing the strength of the associations we have to sounds (and smells, too, i've heard).

TIL org-indent-mode

#emacs

before emacs lisp package archives were a thing, i learned a lot about emacs and elisp because reading the code was a natural part of installing, configuring and using a package. i like the automation and ease of installation, but i feel like something intimate is lost...

#emacs

i'm always happy when i see a new release of #crystallang

https://github.com/crystal-lang/crystal/releases/tag/1.11.0

awesome work!

i'm on hold on my cell phone and the on hold music is distorting and clipping, and in my mind there's a small, physical speaker in the loop somewhere, which is being over-driven, and a little telephone handset on its side that's picking up and transmitting the on hold music... vs. you know, technology from the 21st century...

sometimes i wish crystal had type based control of i/o. (not necessarily non-strict evaluation and the io monad a la haskell, though...) i'm looking at a boatload of code and wondering, "does this all just compute a result, or do i have to scan for the side-effects..."