I wrote a better browser

2023-03-30

Yes, I did it. And it lives here, in the Bopher-NG repo. And it still is under 300 SLOC of pure Bash code. And it now supports mouse, history, proper statusbar, link stashing, force downloading, standard gopher:// URIs and is overall much faster than the prototype. But that’s not what my today’s post will be about.

I won’t describe all the changes I made to the original Bopher prototype in detail. If you’re interested, you can compare the code yourself, I deliberately left the old version “as is”, only updating the file descriptor from 3 to 4 to make the old version usable on macOS as well. Here, however, I’m going to list all the points I’ve learned so far during this improvement process.

  • ANSI codes are tricky, and ANSI literals are even trickier. Switching from "\x1b" to $'\x1b' to define the ESC character actually saved me from a lot of intermediate escaping trouble. Same for "\n" to $'\x0a', actually. Suddenly, everything got much more robust as I didn’t have to use the %b specifier in my printf calls anymore.
  • The printf command, by the way, really is a wonderful Bash builtin, but first, you need to be careful with string format specifiers, second, terminal output itself is actually quite computationally expensive. That’s why all the rendering should be done off-screen, and then only a single printf call should be made.
  • By the way, don’t be afraid to use single quotes by default in Bash. Only change them to double quotes when you do need to expand variables inside. This matters not only for security, but for performance too.
  • Also, don’t be afraid to use the arithmetic version of Bash’s for cycle to only process the array elements you actually need.
  • Mouse support is no black magic once you know which sequence turns on/off the mouse event reporting from the terminal in the proper format. Because there are, like, at least 4 of them, and only one works virtually everywhere and reports only three plain numbers that don’t need decoding. You need to read these events properly though, as mouse press and release control sequences end with different modifiers.
  • For bare Linux console though, you need to use GPM + LCXTerm combo to get the mouse working in Bopher-NG, because GPM maintainers, in 23 years, still have implemented neither X10 nor X11 mouse reporting format support.
  • As I have already mentioned, macOS is a bitch. Not only is it shipped with a really outdated Bash version (3.2 or something) so you have to explicitly pass the script to the bash command installed from Homebrew, but also has some system service messing with the file descriptor 3 that can easily kill -9 any process that tries to access it. This is why you need to use at least 4 as the descriptor number in your /dev/tcp interaction routines in order to make your scripts cross-platform.
  • The link stashing concept really is much simpler and much more elegant than both clipboard and bookmarks combined. Say, you’re browsing some Gopherspace resources and think “I might want to come back to this later”. Instead of copying the resource link and pasting it somewhere else, or making a bookmark (which you also need to name), you just press S (shift+s) and the link (in the standard gopher:// URL format) is saved to your stash file along with the UTC timestamp. You can view or edit this file anytime later when you need to recall the links, and you’ll also be able to see when you saved them, or export them to another application since they have the standard format. Using some simple scripting, you can even convert the stash file to another Gophermap and share it on a server. I’m not sure if I’ll ever implement a local Gophermap file opening mode in Bopher-NG, but that would be a logical move to convert the stash file to it as well. Until then, let it be just plain text with timestamps and URLs.

And yes, I’m actively using it myself. That might be a bold statement, but my DIY browser in pure Bash turned out to be more comfortable to use than Lagrange and even Lynx. For the same reason, I won’t abandon Kopher (as long as I have a single KaiOS phone to test it on): because it might be the best thing that happened to browsing experience on KaiOS since the release of Nokia 8110 4G. Not because my design is something extraodrinary, but because the stock one is as terrible as it can get. Same thing about Android: there are several different Gopher clients for this OS, and even the most usable of them, Pocket Gopher, wraps text by default and offers no possibility to unwrap it, but for some reason offers a possibility to change the default monospace font to a proportional one. And it also includes huge and totally unnecessary icons for every Gophermap entry type. Holy moly, is it some fucking rocket science to disable text wrapping in Android widgets? Overall, it doesn’t look like the authors of these clients (and I mean every one of them existing for Android, including the (in)famous Overbite) actually are using them on a daily basis, or even were planning to.

Oh, did I already mention that Gopher protocol itself is backwards compatible with Finger protocol? Any Finger query is the same single CRLF-terminated request, just on a different TCP port (usually 79), so you can just specify a 0-type entry in your Gophermap (e.g. 0Fingerblog\t[username]\t[host]\t79) and the client will run it accordingly. And in fact, for instance, gopher://finger.farm:79/0info is a perfectly valid Gopher URL that makes a perfectly valid Finger query to info@finger.farm (yes, I made that a link for you to try out). By the way, I doubt the usefulness of Finger Farm‘s decision to only allow signups via BigTech-owned GitHub or Twitter, but at least they offer something for the laziest of us. On the other hand, there are plan.cat and Happy Net Box that handle all signup things as they should be. By the way, I’m also somewhere there - again, no one prevents you from creating a full Gophermap served via port 79. So, as you can see, the Finger protocol is also alive and well, and has become its own thing over time, but it’s nice to see you get its support basically for free once you get a working Gopher client or implement it yourself.

So, let’s get back to Gopher itself and finally talk about the primary subject of this post: what can you find in the Gopherspace as of right now?

Note: all of the following links in this post will be gopher:// URLs. If you’re too broke (or bloke) to use a second browser like Lynx, Gophie or Lagrange (for PC), Overbite (an extension for Firefox) or Pocket Gopher (for Android or even J2ME), you can use Floodgap proxy or Muffinlabs proxy although both are not very efficient and might block some resources.

There is a common misconception that since the advent of Web (especially Web 2.0 or whatever the buzzword was), Gopherspace became almost dead and only supported by a handful of enthusiasts around the world. A simple look at the Floodgap portal will already convince you that it’s not true. First, you get three great search engines: Veronica-2 hosted at Floodgap, Contrition hosted at ForthWorks and Quarry hosted at gopher.icu. Then, you also get proxies/gateways to some popular Web resources you’re probably already familiar with:

Well, if you know you can find torrents and NSFW content on this network, even if you don’t need to, you know it’s a real deal and far from dead.

Now, here are some great Gopher-based services, catalogs and aggregators:

  • Gopher Lawn - a Bitreich-curated catalog of links to various categorized content (including most of the stuff mentioned here);
  • gopher.icu - hosts Quarry and Project Gutenberg searches, amongst other things;
  • Magical Fish - an amazing and (kinda) Kopher-friendly general purpose Gopher portal that, besides a Wolfram proxy, an IP/domain lookup service and several translation services, even contains a live chat running directly over Gopher;
  • Port 70 News - community-filled link and news aggregator;
  • SDF - one of the earliest pubnix systems still alive, the father of all tildeverses;
  • Tildeverse - the entry point to various other “tilde” communities hosting ~users’ personal pages, blogs (or “phlogs”, as they are called in Gopherspace) and projects (some of them, like tilde.pink, even have CGI enabled for Gopher only);
  • graph.no - weather forecasts in the form of meteograms or one-liners: actually it’s a Finger-based service, but as I told you before, every Finger query is a valid Gopher one so you can shape a corresponding URL for any city of interest;
  • FIGlet gateway - ASCII art text generator using FIGlet (more on that later), hosted at Floodgap;
  • mozz.us - ASCII art, text-based adventure games, drink recipes and more stuff;
  • World of Solitaire - 79 different solitaire-based card games in ASCII art (requires at least 70 character wide screen);
  • Interactive Fiction Archive - more text-based adventure games (for download, not direct play);
  • Bongusta - a phlog aggregator;
  • 1436chan - a chan-like text board (“turn on the bullshit filter” ©);
  • GopherPoll - a poll creation service on Gopher;
  • Happy Net Box itself, unlike plan.cat, also lives in Gopher and lists the finger profiles in a proper Gophermap, serving them as type 1 entries - probably the best showcase of how the two protocols are related.

I think this is a good set of resources to start with. I could continue infinitely but this writeup needs to end somewhere, and I also have some more important things to tell here than to just post a long list of links. For some further exploration though, I suggest you visit Daily Gopher servers still alive, Known servers or The Observable Gopherspace catalogs which, while incomplete, contain most popular and active Gopher servers out there.

While browsing all this, don’t be surprised to find some really weird stuff, where a PDF of the Unabomber Manifesto or someone’s trip report are some of the mildest examples of what you can encounter there. For me, one of the main beauties of Gopher is that it, after all those years, still remains a Wild West of the Internet where people themselves still rule the place, as all the corporate interests shifted to the big Web long ago. And mind you, all this is happening over a plain unencrypted TCP connection, although I really doubt that most ISPs monitor, let alone filter, port 70 these days. But of course, as a simple TCP-based protocol, Gopher (and Finger too) can be served over any secure network that can encapsulate TCP in general. That’s why, for privacy-obsessed people like myself, there also are plenty of Onion-based Gopherholes (and we obviously can make Bash connect to them as well by running it via the torsocks wrapper), some of which are listed here (not to mention the Bitreich portal itself has an .onion mirror) but this list, of course, is in no way exhaustive. For example, the Magical Fish portal is also available via Tor, and even serves a bit different content when accessed over its .onion address.

As you browse the Gopherspace and discover more and more interesting places, you may notice something else, something that the Web might have lost at a large scale long ago: aesthetics. Yes, if you are a webmaster/content creator, HTML+CSS alone are powerful tools to make your pages look exactly like you want to (especially if you know what you’re doing, and that’s why, unlike many Gopher/Gemini fans, I never was antagonistic to the modern Web, only to those who misuse and/or misunderstand its capabilities), but the aforementioned corporate interests dominating the Web have already changed your own preference of how you would want it to look. You wouldn’t want it to look like this very blog, like my txties page with IMEI editor codes or even like this wonderful website. Why so? Because you want it to look like GitHub, like Zuckerbook, like Twitter, like Medium, like Google/Apple apps… Because they are popular and they are setting the trends. The trends change, and you unwillingly follow them for your website to not look outdated, that is, if you want it to be commercially successful. They invested huge money in the UI design you’re trying to copy because they have thought for you already. But why, given every technical opportunity to express your individuality in the most creative ways possible, did everyone of you, webmasters, choose to be a clone and turn into webslaves of ever-changing trends? It’s not me you should ask. Gopherspace, on the other hand, looks like it has little to offer in terms of self-expression by decorating your content, but because it is nearly impossible to commercialize, individual pages aesthetics actually show much more freedom of expression than the uniform app-like Web of today. For instance, it’s a good tradition in Gopher to publish your text preformatted to some fixed width, with headings and optionally footers decorated with some ASCII art, or decorate your Gophermaps headings with ANSI art (using post-128 block characters and/or terminal escape codes). It’s absolutely unnecessary, it’s just a tradition (that comes from even earlier BBS dial-up or Telnet resources) but this is exactly what makes every plaintext resource individual in its presentation and not only content. Moreover, ASCII art signatures inspired the FIGlet program creation at around the same time Gopher began gaining traction, and figlets became de-facto standard way of captioning personal homepages in Gopherspace for those who didn’t want to draw these captions by hand. This is your aesthetics in the world of plain text. And it is beautiful.

Gopher’s plaintextness (is that even a word? Well, now it is…) also inspires individual authors to seek more creative ways to deliver the same services. For example, the aforementioned Google Directions gateway offers you a series of 7-type queries to shape the full navigational URL. Or the Magical Fish portal offers a way to measure your screen width in characters just with a… simple text file. That’s really ingenious. This file doesn’t need to be on Gopher, but I found it there. It probably can be found somewhere on the “big” Web too, or even generated locally with a shell one-liner, but I can hardly imagine where I could encounter it or even an idea to generate it. But now, I can use it to test every mobile Gopher browser I get my hands on. On top of that, Gophermap structure itself doesn’t make any significant difference between links to the resources on the same server and foreign servers, interactive (type 7) or not, there’s no “same-origin policy”, referrer headers, cookies or other ways of tracking where the visitor actually came from because requests are completely decoupled from the presentation layer, and this encourages cross-linking even more than it was encouraged on the early Web. And it is totally fine for most Gopherholes to group “domestic” and “foreign” interactive services together and to let the user choose between them while staying on the same Gophermap. This allows to create rather flexible document structures using seemingly rigid and straightforward syntax. And the whole secret of this flexibility lies not in the format itself but in how many different ways one can walk in Gopherspace from the same point A (provided it’s a Gophermap) to end up in the same point B on a completely different server. This was only a thing in the earliest stages of Web and entirely contradicts the dominant ideology of the modern (commercialized) Web where almost every website does everything to make users spend as much time as possible on its pages only and not navigate anywhere away from them. Gopher (and early Web that was created as a sort of fancier alternative to it) is about letting the user get to the desired content as quickly as possible. Modern Web is all about grabbing human attention, one of the most precious resources that can be monetized. Everything else that has happened to the Web during this 30-year period, was just a byproduct of this paradigm shift.

And for Gopher, which is still as usable in 2023 as it was in 1993, I’m really happy this shift never happened.

#philosophy #gopher #bash #development #browser #minimalism
_