one criticism of activitypub is that it's not a specification, and as a consequence you have to test extensively to ensure interoperability between implementations.

this is a valid criticism.

  1. minimally you need to implement a few things beyond activitypub to get even basic interoperability with popular implementations—http signatures at a minimum (and a draft version of that) and probably webfinger. (i guess this really isn't a problem with activitypub per se, but it sure feels like it is.)
  2. the recommendation doesn't tell you what to do with the various object types, so even if you get the basic transport working, you don't really have an application. what's the right way to implement an interoperable "comment", for example?
  3. and then there are interoperability problems in practice—just scan the issues for popular implementations. these have to be identified and fixed multiple times, every time someone spins up a new implementation.

to some degree i guess i don't care. i'm building a free/open source application not a commercial product, and it's aimed at relatively tech-savvy users, and i kind of enjoy spelunking issues like these anyway, but for anyone whose goal is mass adoption of the "fediverse" concept, it's a problem.

#activitypub #ktistec

a major milestone in the life of ktistec... with the release of blocking, the database schema for 1.0 is now fixed. this means no more migrations. i'm working on a few interoperability issues, and then i'll tag 1.0.

#ktistec

commits a2f3fad to 485dbf2 add support for blocking actors and objects. like many things, blocking is built on the back on a lot of earlier cleanup and refactoring.

as expected, blocking an actor removes the actor and its content from your timelines. blocking an object removes only that object.

finishing this is the last blocker to a 1.0 release. 🎉

#ktistec

before it was called "agile programming" it was called "extreme programming". one of the tenets of extreme programming is "merciless refactoring". another is "edgy adjectives".

an often overlooked "freedom" in a free software project is the freedom to refactor, mercilessly. why turn a free software project into a feature factory? you can get that in an office job, right?

anyway, commits 14d22ca to 14fb9af do away with 1860 lines of code, while only adding 880 lines. more importantly, they do away with quite a lot of distracting boilerplate—like explicit identifiers (IRIs) that usually have no bearing on the outcome of a test.

the one big addition was spec_helper/factory.cr (349 lines), which adds lightweight factories that take care of the details of instantiating a valid model instance.  i picked up the build/create paradigm from the factory_bot gem, and extended the let/let! pattern used in spectator.

#ktistec #agile #extreme #refactoring

in ktistec news, commits 4d2f699 through 4d2f699 include several improvements and bug fixes. highlights:

  • at runtime, the KTISTEC_DB environment variable overrides the default location for the SQLite database file. i use this during testing. it may be useful for some deployments.
  • the background TaskWorker now kicks off as soon as a new task is scheduled. the previous implementation polled for new work every five seconds.
  • i collected all inbox/outbox side-effects in one location. this makes mailbox processing easier to reason about, and also made it easier to prevent a race condition that resulted in the same object being added to the timeline more than once†.
  • images are lazy loaded in browsers that support loading="lazy".

i'm also now only one feature away—blocking actors and objects—from tagging the 1.0.0 release.

† the race condition occurs when an activity, like a create, arrives that takes a long time to validate, maybe because ids need to be dereferenced. while it's being validated, an announce activity arrives for the same object. now two activities for the same object are being processed concurrently. being added to a timeline is a side-effect, and collecting all side-effects in one location made it easier to put the duplicate check after validation but before any side-effects, which saved me from having to unwind side-effects if a duplicate was detected.

#ktistec

what a difference one line of code makes!

commit fe18dc5 moved the line @saved_record : self | Nil = nil from the module Ktistec::Model::InstanceMethods into the macro Ktistec::Model.included.

that single change cut the build time in half and reduced the memory required to build by about a third. you can be sure the improvement came as a surprise!

but really, it wasn't just this change itself—although it introduced the biggest improvement—because after all of the subsequent refactoring, simply moving the line back didn't increase the build time again.

the build time also improved slightly after other subsequent commits. and several of those commits introduced changes that reduced the amount of macro generated code. and that's the key insight: macros reduce visible boilderplate but don't reduce actual code that needs to be compiled.

i'm still investigating, but clearly the original placement of the line resulted in the generation of a lot of redundant/unnecessary code!

TL;DR if you're planning on using macros in crystal, check your build times before and after your changes—i'm sure there's a corresponding commit somewhere in the past that introduced this problem and i didn't catch it!

#ktistec #crystal

building ktistec is still an endeavor—both in time and in space. i have work ahead of me to reduce the build footprint—since starting this project, i've learned that some combinations of the crystal language features i used to recreate rails magic (macros, named arguments and inheritance) are hard on the compiler.

in any case, having exceeded the capacity of the tiny linux vps that hosts epiktistes, i set out to build a statically linked executable on my laptop (macos mojave) that i could deploy to my vps (linux centos).

the official instructions, using docker and alpine linux, are here. to get it to build, i had to install the static libraries for sqlite, and to get it to run on my vps i had to specify the location of the openssl cert file, which is loaded at runtime and doesn't match the location in alpine linux.

tl;dr the steps

  • build (i haven't automated any of this yet)
    1. docker run --rm -it -v $(pwd):/workspace -w /workspace crystallang/crystal:latest-alpine sh
    2. apk update && apk upgrade
    3. apk add sqlite-static
    4. crystal build src/ktistec/server.cr --static
    5. exit
  • run (after copying the executable to the vps)
    • SSL_CERT_FILE=/etc/ssl/certs/ca-bundle.crt ./server

#ktistec #crystal

commits 99a0fd6 to 70bcf3f implement metrics and charting.

inbox and outbox activity

an instance of the Point model records an occurrence of an event—for example, the arrival of an activity in my inbox. points have a timestamp and a value and can be aggregated by day, week, etc. aggregation takes into account the timezone of the account that's viewing the chart so you'll need to specify a timezone during setup or in your account settings. i use chart.js for the charts.

i'm only collecting inbox and outbox metrics right now, but this will expand to include other types of metrics—social events, errors, etc.

looking at the chart above... i clearly need to post more...

#ktistec

i spent most of the summer cleaning up the codebase of my covid project (aka ktistec).

in theory, ruthlessly refactoring should be a pleasure. truth be told, i don't enjoy it as much as building new features. nonetheless, everyone else—including my future self—should find things more consistent.

things i did include replacing ecr templates with slang templates, using form helpers to make forms behave more consistently and to reduce boilerplate, etc. etc. etc. i also moved from turbolinks to turbo, which allowed me to remove my custom code for making updates to parts of a page. i know it's not cool but i'm a fan of html over the wire. i also fixed/improved timeline handling and refactored population/management of timelines and notifications. i really need to refactor handling of all side-effects—i just found another place in the code where out-of-order messages result in duplicates in the inbox.

(interesting aside about that... it's possible to get a forwarded create activity from a source before receiving the create activity from the origin itself. it's not surprising in retrospect. the origin was busy sending to followers and i was in the queue. meanwhile, another follower who was mentioned in the related object broadcast the activity to their followers...)

#ktistec