big, heavy, and wood

justin mason flagged this article about "The log/event processing pipeline you can't have" a while back, and it has been on my mind ever since. our digital infrastructure is split across a few machines (virtual and not) and i often wish that i had a more cohesive way of collecting logs and doing even minimally interesting things with them.

i think the setup there is probably overkill for what i want, but i love the philosophy behind it. small, simple tools that fit together in an old-school unix way.

i set up an instance of graylog to play with a state-of-the-art log management tool, and it is actually pretty nice. the documentation around it is kind of terrible right now because the latest big release broke a lot of the recipes for processing logs.

right now, the path i am using for getting logs from nginx in a docker container to graylog involves nginx outputting JSON that gets double-encoded. it’s all very gross.

i think i am having a hard time finding the correct tooling for the gap between “i run everything on a single box” and “i have a lot of VC money to throw at an exponentially scalable system”. (while also avoiding AWS.)

(the very first post to this blog was the same ren & stimpy reference as the title of this post.)

the state of things

just over seven years ago, i mentioned that i had decided to switch over to using scat, the point of sale software that i had been knocking together in my spare time. it happened, and we have been using it while i continue to work on it in that copious spare time. the project page says “it is currently a very rough work-in-progress and not suitable for use by anyone.” and that's still true. perhaps even more true. (and absolutely true if you include the online store component.)

it is currently a frankenstein monster as i am (slowly) transforming it from an old-school php application to being built on the slim framework. i am using twig for templating, and using idiorm and paris as a database abstraction thing.

i am using docker containers for some things, but i have very mixed emotions about it. i started doing that because i was doing development on my macbook pro (15-inch early 2008) which is stuck on el capitan, but found it convenient to keep using them once i transitioned to doing development on the same local server where i'm running our production instance.

the way that docker gets stdout and stderr wrong constantly vexes me. (there may be reasonable technical reasons for it.)

i have been reading jamie zawinski’s blog for a very long time. all the way back to when it was on live journal, and also the blog for dna lounge, the nightclub he owns. the most recent post about writing his own user account system to the club website sounded very familiar.

character encoding is still hard?

email2webhook is nice in theory, but fell flat in handling basic character encoding.

that i still have to fight the same sort of issues that i was dealing with about 16(!) years ago is somehow not at all surprising.

i’ve switched to using zapier’s email to webhook features. it will probably bring a different set of challenges.


now i’ve done it

i noodled around a little more, and came up with an ugly way to supply and extract tags in my postings via email. the next step will be extracting attachments so i can post images again.


i wanted to play with email2webhook, so that gave me an excuse to knock together a quick and dirty way to post here via email. i didn't add a way to tag posts, but it's a start. more to come.


yes, this is still on. and now it is on a new server with new code, even though it looks the same.

i still need to knock together the "writing a post" bit of code so that i can post without using manual SQL queries. details.


it turns out that this thing is still on. it is kind of funny to me that it still just chugs along, but such is the joy of writing your own software running on your own (virtual) server. it is long past time to rebuild the infrastructure here, which may or may not happen. but it is something that i am thinking about again.

let’s encrypt

this domain is now using a certificate from the let’s encrypt project. they are scheduled for general availability in a couple of weeks.

it’s a great idea. getting free domain certificates has always been more annoying than i was ready to deal with, and once this is all fully baked, it will make it easier to just make everything encrypted by default.


it turns out that the county of los angeles regulates the use of point-of-sale systems that use barcode scanners. one requirement is that you have a customer-facing display that shows prices as they are scanned.

after investing in a lcd pole display (for about $175), i had to bang together a way for scat to feed data to the display. and thus was born lcdpoled, a very simple daemon that listens for stuff to display.

i need to do some more work to make it as seamless as i would really like, but i think i have done enough to get us through an inspection if and when they stop back in to do that.

siamang: a web-based command-line client for mysql

Siamang Main Screen

because i have many more important things to be doing, i procrastinated by whipping up a little web-based command-line client for mysql. what does that mean? you load the webpage, start typing in sql commands, and it presents the results to you. it is just a weekend hack at this point, and has a lot of rough edges. it does have some cool features, though, like a persistent command-line history using Web Storage.

probably the biggest limitation is because it not maintaining a persistent connection on the backend, you can’t use variables, temporary tables or transactions.

it was really born because i was getting frustrated running queries using the command-line client and having the ASCII table look all wonky because it was too big for my terminal screen. html makes that pretty much a non-issue. it is also tablet-friendly.

Siamang Example Table Result

it was also an excuse to play with a few things i was interested in, like knockout. the whole thing is under 400 lines of code/html right now, but by leveraging bootstrap and knockout, it actually looks pretty polished and functional.

the name comes from a type of gibbon, which seemed to be unused in the software world.

you can find the source on github.

the old samurai

the old samurai is a film that celia produced and i helped out on a little bit. enjoy!

grand central market redux

grand central market

i’m excited to hear that the grand central market is going to see some upgrades. it was almost twelve years ago when i was serving jury duty and stumbled on the grand central market for lunch, which helped the plant the seed for me to move downtown a couple of years later and set the stage for all the awesome things that have happened since.

the market seemed to be having a fairly rough go of it ever since i moved downtown, even before competition like the new ralphs market opened. the hours didn’t really accommodate the new residential influx, and few of the vendors seemed to notice what was happening in the neighborhood around them. i am optimistic that with many of the existing vendors having long-term leases already, they won’t be priced out, and their presence should help keep the market grounded and accessible to the diverse downtown community.