i added code to log slow queries in ktistec and it's already paying dividends. most are obviously missing indexes and it's great to fix them, but the latest example—which is missing an index—is querying a table that only has one row (in my single user instance). should that table need an index on that column? i mean, just return that row...

fwiw, a slow query is currently anything that takes longer than 50msec. i wonder if that is tight enough...?

#ktistec

one thing ktistec related that i haven't had the time for is working on build and deployment tools. there are a bunch of outstanding requests—and a few PRs—for docker builds, packaged deployments for various hosting environments, etc.

if you're interested in contributing, let me know. you only have to agree to maintain them—i won't be able to.

#ktistec

well, i've run epiktistes long enough without restarting to have some confidence about memory performance. while there is an extended period of growing memory usage, the server does settle down after about 15 days.

#ktistec

i've been following hashtags for a while now. i turn off shares (boosts) and replies so they don't appear in my timeline (there's too much sharing going on out there), but then follow a handful of hashtags (like #woodworking and #crystallang and #boardgame) to see more of what i like!

#ktistec

I've been tracking epiktistes inbox/outbox traffic and memory statistics (as reported by the Boehm garbage collector) for a while. There's always a consistent increase in both heap size and free memory—to the point where reported free memory is greater than the originally allocated heap—though the difference between the two doesn't appear constant over time. At the moment, heap seems to have plateaued but (pessimistically) I don't expect it to remain flat.

Given relatively flat traffic, the growth in free memory is surprising. I haven't had a chance to investigate, but based on what I understand about Boehm the increase in free memory could be due to increased fragmentation, and the growing difference between heap and free memory could be due to the conservative nature of the garbage collection algorithm.

Or there could be legitimate memory leaks. I did find one, months ago, which was the result of caching SQL prepared statements (in a Hash) and poor practice constructing queries in a couple places, which led to nearly linear growth in cached statements. The difference then was that heap growth was much more consistent, which is not what I see here, now.

#ktistec #crystallang

commits 984e28b7 through a367c215 include changes that support following threads, hashtags and mentions:

• index pages for threads, hashtags and mentions include a follow/unfollow button
• following a thread, hashtag or mention results in notifications when new posts arrive that are part of the thread or include the hashtag or mention
• to support navigation and discovery, the details pages for posts include labels with links to the index pages for hashtags and mentions

next up is support for actively finding and fetching followed content from elsewhere in the fediverse.

also included are a bunch of bug fixes, some internal enhancements, and greatly reduced build times and executable sizes.

#ktistec

the crystal programming language always inlines blocks, which is great for performance but trades off space for speed. using blocks effectively means keeping this in mind.

somewhere along the line, i learned the habit of passing a block to a function as a means of customizing the behavior of the function. if the function that takes the block is large, it's important to remember that the body of the function is inlined where the function is called, which may not be what you are expecting. if you call the function multiple times, you even get multiple copies.

i just committed code that fixes an egregious example of this problem. in this case these ~30 lines of code replace the blocks with procs (which aren't inlined) and cut ~24mb (that's megabytes) off the executable (over a third of its size).

i regularly shoot myself in the foot trying to be clever, so i don' t know how prevalent this problem is in practice, but it's definitely something to keep in mind, especially if you see compile times and executable sizes growing!

#crystallang #ktistec

one of the crystal programming language’s greatest strengths is the ease with which you can work with c apis and external libraries without leaving crystal. you can even implement callbacks in crystal!

in sqlite you can define new functions—callable from sql—with sqlite3_create_function by providing a name and a callback that implements the function. i created a new function called strip, entirely in crystal, that removes html markup and leaves text. it can be used in a sql query like so:

select * from objects
where strip(content) like "%term%"

it’s part of the recently released content filtering code.

#crystallang #ktistec

I just released v2.0.0-8 of ktistec. The most impactful changes are:

1. No more dependencies on externally hosted assets (and fewer dependencies, overall)
2. Basic support for timeline filters (no shares and no replies).
3. Support for content filtering by keyword.

Volume has dropped off in my timeline, for the most part, now that the surge of people who signed up for Mastodon accounts a couple months ago have gone back to posting on Twitter, or have stopped posted about the transition, or whatever. But fine grained control is nice, and filtering allows me to tune my experience—better late than never!

Read the changelog for all of the details.

#ktistec

i released 2.0.0-7 just in time for the new year.  it includes contributions from @relistan and @rahul, the introduction of CI (the build is successful), and bug fixes.

i am slowly working my way toward more flexibility for reading and managing federated content.

#ktistec (as always, it's pronounced "tiz-tek")