The Subtle Oppression of the Code Library
Where have I been? Well, it was a three-day weekend and I had a couple of things I wanted to write here, but they were Long Things and I just didn't quite ever feel up to it. Also, all day today and all day Thursday, during working hours, I did nothing but download and install Perl libraries. I have a whole lot of pent-up rant about that process, and here I am unpenting it. This is a long entry and it is probably not of interest to most people. Some of you will enjoy it for the Position Statements, which I think you can probably get pretty clearly even if you skim the code-ish bits.
The deal is, for reasons I'm going to go into in a separate entry later, I needed to install a particular Perl library on my server. I will even name it: The library is Net::SAML2, and it provides functions to do SAML authentication. And if you don't know what that means, don't worry, I'll cover it in that later entry. Just assume I needed this thing for a vital task, so I had to install it.
It is worth noting -- I'll cover this in that later entry too -- that I had deliberately tried to avoid installing this library. For weeks. Because I knew in advance what hell it was going to be. I could tell from its list of dependencies. So I tried other solutions, smaller, easier ones, first. They didn't work. Eventually I got to where I was running out of time (I must finish this before 1 June), and I had to bite the bullet and install this library. I still don't know, as I write this, if the library will actually work! I haven't gotten that far yet, because it has taken me two solid days of work -- maybe not sixteen hours, but a good twelve or so -- to get the fucking library installed.
And that's just goddamned ridiculous. It should not be that way. You should not have to take two days just to get one library in place.
Now, there are services that you can use to automatically grab Perl libraries from a repository. I do not have those services installed on my server. For one thing, their operation is somewhat opaque and I like to know exactly what is in my Perl build, and to know because I built it. And I have to tell you: I wonder just how much of this situation is because everyone else is grabbing libraries automatically from CPAN (the Perl library repository) or using some other package manager. I wonder if they don't see the bloat, because they never do it by hand.
I do use a package manager for system changes -- I mean operating-system level stuff -- because I'm not that crazy. Also, frankly, because the operating system on that server is already using package management, you get fewer issues (like version clashes and so forth) if you just go with the flow. Besides, my system is partially managed by admins above my level (I'm root, I'm the sysadmin, but they do things like patch the OS routinely) and they use that package management and they'll just stomp on my stuff anyhow if I don't do it their way.
There are two places, though, where I refuse to do it their way and insist on complete control. That's my Perl build and my Apache build. The core of my web stack. And in both cases this has caused me no end of problems. Because I cannot successfully tell the system "NO, do not use your httpd that I don't know how you built it, always use mine instead," I can't let them autostart their httpd when the system boots, and despite two-plus years of trying, I have been unable to get the service manager to autostart mine instead. So I've disabled theirs forcibly (by deleting files) and I wrote a script which checks every ten minutes to see if my Apache is running and if not, start it. (This is how I get it back up when the IT admins power-cycle my server at three o'clock in the morning, which they inexplicably do once a month, and thus save myself the pain of getting to my email at nine-thirty and finding that people have screaming since five a.m. about the server being down.)
With Perl, the issue is actually that I can't mess with the system Perl. There are system internals that use it. So everywhere in my code, the code that matters, I have to make sure I'm always pointing to /usr/local/bin/perl -- which has its own set of include paths, etc, and different versions of a lot of its libraries -- rather than /bin/perl, which is what the system thinks is correct. This complicates my life, but not as much as you might think. Bear it in mind, though. It'll come into the story again below.
I am not religious about this, and I'm far less strident about it than I was twenty or thirty years ago, but I still cast a somewhat jaundiced eye over library inclusions. There are three reasons for this.
1. I feel they're often unnecessary. In the later entry, I'll have a few words to say about Digging Down To The Guts. The ratio can be incredibly bad. I've seen libraries that were five thousand lines where maybe fifty lines of that were doing the actual work. The rest is all overhead.
There's a library below -- I'll call it out when I get there -- which was 363 lines. I know, because I was so outraged at the time that I stopped and counted. Now, a third of that was POD (in-place documentation) and I can't knock POD, I have needed to use it too many times. But honestly, the library is to encode and decode URLs (you have to replace special characters with hex entities), and I do an awful lot of that, and I do it with code I ganked from someone else's example years ago.
sub urlencode
{
my $url = shift;
$url =~ s/([\W])/"%" . uc(sprintf("%2.2x",ord($1)))/eg;
return $url;
}
{
my $url = shift;
$url =~ s/([\W])/"%" . uc(sprintf("%2.2x",ord($1)))/eg;
return $url;
}
sub urldecode
{
my $url = shift;
$url =~ tr/+/ /;
$url =~ s/%([a-fA-F0-9][a-fA-F0-9])/pack("C", hex($1))/eg;
return $url;
}
{
my $url = shift;
$url =~ tr/+/ /;
$url =~ s/%([a-fA-F0-9][a-fA-F0-9])/pack("C", hex($1))/eg;
return $url;
}
You can tell these functions are a little crusty. For example, I'm not sure that the 2.2 in the sprintf format is necessary. It could just be 2, I think. You're printing the hexadecimal equivalent of the character's ordinal value, but %x as a specifier means it wouldn't have a decimal portion anyway. Also, you'll notice I decode pluses for spaces, but I don't encode spaces as pluses. The correct way to do it is to encode them as entities; I have to decode pluses, though, in case I get URLs to decode from people who did it the lazy way.
(If you're deciding to give up on this entry because your head's already reeling, OK, look: If you have a space in your URL, which is illegal, you take its ordinal (decimal) value, which is 32, and translate that to hex, which is 20. Hex is base sixteen; two sixteens is thirty-two. Then you put a percent sign in front, because that's what URLs like. Now "this url has spaces" has become "this%20url%20has%20spaces". All right? To decode, you look for anything in the URL that's percent followed by two characters. The hex() function will translate it back into a decimal number, and the pack() function with a "C" format (C for "character") will make it back into the character it was supposed to be e.g. 32 becomes a space again.)
The point is, that's thirteen lines of code. I just saved you three hundred and fifty lines of inclusion.
2. They obscure the process. I grant that if you're one of those people who has absolutely no programmer brain, you're just as confused as when you came in. But if you have even a little bit of programmer brain, in the section above, I just explained to you how the URL encoding and decoding works, and if you were having to write code of your own, that could be very important.
Code where the programmer does not actually understand what is happening is bad code. You might get away with it, you might get away with it for years -- I admit I've got a couple of places in code where I had to use someone else's algorithm and I just rope it off with comments that say HAZARDOUS VOLTAGE DO NOT TOUCH. But I try not to have any of these. Even in places where I do use someone else's library simply because it's so much of a pain in the ass that I don't want to do it myself, I try always to have at least a functional understanding of what it's doing, down in the guts.
But not everyone will take the trouble to do that. And saying "Oh, just use a library for that" is a far-too-convenient way for you, the programmer, to default on your responsibility to understand what the fuck you're actually doing.
In these trying days when programmers would apparently rather use generative tools to glue together bits of code ganked from the web, code they could gank from the web themselves and get results which actually worked, this may be a lost cause. I admit it. There are simply too many young programmers right now for whom understanding how their code works is simply not a priority.
In fairness to these kids, it may not be entirely because they're idiots or assholes. The software industry has become a soul-crushing mess; the days of the Guru Wizard Developer Who Sits In A Backroom and Answers Only To His Own Schedule are long over. Some of these kids are being squeezed so hard by impossible deadlines and a work-until-you-drop shark-tank mentality that it's unsurprising they immediately seize upon anything that might save them. (I discuss this a little bit in my story "Sweet Eliza," which you can find wherever you find my quality fiction output.)
But the thing is, if you don't understand the code, then you can't fix it. And you will be called upon to fix it, especially if you used Claude to glue it together (remember, Claude does not comprehend what it is doing) and it's broken as hell right out of the box. By picking speed and ease at the beginning, you are making technical debt for yourself that'll come back to bite you on the ass later, when your boss storms in and yells, "That code you released doesn't work! Our lawyers are in a panic! Fix it by tomorrow or you're fired!" Guess what? You're fired. Unless you are a very, very fast learner. But wouldn't it have been better to learn at the beginning?
3. They frequently carry an assumption of other tools that you may not be using. This is the real gist of today's sermon. You may be making intelligent choices about when to use a library, when it's worth that space and time, when the savings are a false economy, and so on. But when you install a big library, you're suddenly at the mercy of the choices the person who wrote that library made. And they may not be as careful or as rational about it as you are.
3. They frequently carry an assumption of other tools that you may not be using. This is the real gist of today's sermon. You may be making intelligent choices about when to use a library, when it's worth that space and time, when the savings are a false economy, and so on. But when you install a big library, you're suddenly at the mercy of the choices the person who wrote that library made. And they may not be as careful or as rational about it as you are.
As I say, I needed one library. That library had dependencies -- other libraries that it relies upon, that must be present for it to run. This is normal. Those dependencies may also have dependencies -- things they rely upon. This, too, is normal. This nesting of dependencies, as you'll see in a moment, may go pretty deep.
Over two days of work, counting the library I needed, I had to download and install one hundred and four Perl libraries. That's not counting some dependencies I already had installed for other reasons. I figure, if we count only the stuff that doesn't come in "stock Perl" -- stuff I would have had to install at some time or another -- the total count of dependencies here is over a hundred and fifty. Possibly well over.
I said it above and I'll say it again: That's goddamned ridiculous.
When I am installing a library, I've learned from years of hard experience that I can't trust what CPAN says its dependencies are. So I don't. Almost all Perl libraries -- and we will come to that "almost" in a bit -- use the "MakeMaker" system. That is, they use the standard Unix/C "make" tool to compile and build code. With Perl libraries there is often no actual compilation, but that's OK, you use the tool the same way each time. That's one of its great strengths.
It goes like this:
/your/perl Makefile.PL
make
make test
make install
make
make test
make install
That is, use Perl itself to execute a Perl script called Makefile.PL, which checks for various things and spits out a makefile in a format that the "make" tool knows how to use. Then you use the "make" tool. Then you use it again with the target "test," which runs a suite of tests that the person who wrote the Perl library (ideally) built in to make sure everything went OK. Then you run "make install," which copies the library you just built into your Perl locations/paths so that you can get at it from anywhere.
The first step of that will check for required dependencies. That's how I do it. I let the build process tell me what I need. In practice that means: I try to build the library I really want, knowing it won't work the first time. I get a bunch of errors saying "You need this, and this and this." I go out and grab all those, and one by one I build each of those. Most of those will also get errors saying "you need this and this and this." So I work my way down. It's a nested tree. When I've got everything at one level of the tree built OK, I go up and try again to buld the thing that was depending on those. And so on back up.
YES, I am going to work my way through this tree, because we're going to have some object lessons along the way, including one which I have been ranting about off and on for nearly thirty years.
So, to refresh your memory, the ultimate goal here is to build Net::SAML2, version 0.85, for handling SAML authentication.
It had a whole lot of dependencies, including many I had installed. The first set of its dependencies has to do with encryption. SAML uses SSL, which means it uses certificates and all that crap, and though I am already forced to do a great deal of that on my server, it needed a few things I wasn't already set up for.
These three are installed in this order for dependency reasons (e.g. Verify requires X509):
Crypt::OpenSSL::X509 2.01
-> Crypt::OpenSSL::Guess 0.15
-> Convert::ASN1 (0.33) - upgrade 0.31 -> 0.34
Crypt::OpenSSL::Verify 0.38
-> File::Slurper (0.012) 0.014 HOLY WAR HOLY WAR
-> -> Test::Warnings 0.038
-> Test::Exception 0.40 -> 0.43
not sure why it said not found, gonna remake and reinstall
Crypt::OpenSSL::Random 0.17
-> Crypt::OpenSSL::Guess 0.15
-> Convert::ASN1 (0.33) - upgrade 0.31 -> 0.34
Crypt::OpenSSL::Verify 0.38
-> File::Slurper (0.012) 0.014 HOLY WAR HOLY WAR
-> -> Test::Warnings 0.038
-> Test::Exception 0.40 -> 0.43
not sure why it said not found, gonna remake and reinstall
Crypt::OpenSSL::Random 0.17
Those are direct from my notes -- I have to keep notes as I go or I'll lose my place in the tree. The numbers in parentheses are requirements. Sometimes the build will just report "I need any version of this." Other times it'll say "I need a version of this either (number) or later." So for Convert::ASN1, it demanded at least 0.33; I already had 0.31 installed, which wasn't good enough, so I upgraded that to the most recent version, which was 0.34. The arrows are nesting; that is, Crypt::OpenSSL::Verify needed File::Slurper, which needed Test::Warnings.
The HOLY WAR HOLY WAR comment ... oy. OK. The Perl community is not monolithic. In fact, it's about as far from that as you can get. It's a bunch of extremely idiosyncratic humans, each of whom has their own ideas of how to do things, each doing them a different way, and it's actually kind of amazing that Perl works at all. It's a volunteer project -- no one is getting paid to maintain Perl -- has been for a very long time, and the majority of these people are extremely quirky. I can say that with some authority, having met several of them, and let the record show I do not exempt myself from that characterization.
Anyway, there are two libraries. They are called File::Slurp and File::Slurper. They do more or less the same thing. The person who wrote File::Slurper feels so strongly that you shouldn't use File::Slurp that he wrote a blog entry about it. I have not seen any rejoinder from the author of File::Slurp. Since he has posted a new version of File::Slurp at least once in the intervening years since that article was written, it's possible he is unaware of the whole thing. Or, more likely, he's ignoring it.
The amusing/annoying thing is that I have to have both of them installed, due to dependencies ... and left to my own devices, I would neither install nor use either. I slurp files all the time. (That means: Do ONE read and grab the the entire file contents at once, rather than reading it line-by-line or record-by-record.) I do it the old-fashioned way, as set forth in the Gospel of Wall back in the days of the Pink Camel Book: I temporarily disable Perl's internal record delimiter.
$/ = undef; # no record delim, so file is all one record now
$bigass_scalar = <INPUTFILE>; # yoink
$/ = "\n"; # reset it to what it normally is: newline
$bigass_scalar = <INPUTFILE>; # yoink
$/ = "\n"; # reset it to what it normally is: newline
You need a library for that?
I don't actually have a problem with the encryption dependencies; it stood to reason those were coming, and they weren't unreasonable. They are definitely not something I could or would do myself (except for the damned file-slurping).
It's this next set that really chapped my ass. And, not coincidentally, this is where the largest chunk of those two days was spent.
DateTime 1.66
-> CPAN::Meta::Check (0.011) 0.018
-> DateTime::Locale (1.06) 1.45 2+ MB TGZ
-> -> File::ShareDir::Install 0.14
-> -> Dist::CheckConflicts (0.02) 0.11
-> -> -> Module::Runtime (0.009) 0.018
-> -> File::ShareDir 1.118
-> -> -> Class::Inspector (1.12) 1.36
-> -> IPC::System::Simple 1.30
-> -> Params::ValidationCompiler (0.13) 0.31
-> -> -> Eval::Closure 0.14
-> -> -> -> Test::Requires 0.11
-> -> -> Exception::Class 1.45
-> -> -> -> Class::Data::Inheritable (0.02) 0.10
-> -> -> -> Devel::StackTrace (2.00) 2.05
-> -> -> Specio (0.14) 0.53
includes Specio::Declare, Specio::Exporter, Specio::Library::___, Specio::Subs
-> -> -> -> Clone 0.50
-> -> -> -> -> B::COW (0.004) 0.007 MOO
-> -> -> -> Clone::PP 1.08
-> -> -> -> MRO::Compat 0.15
-> -> -> -> Module::Implementation 0.09
-> -> -> -> Role::Tiny (1.003003) 2.002005
includes Role::Tiny::With
-> -> -> -> Sub::Quote 2.006009 dumbest inclusion ever
-> -> -> -> XString 0.005
-> -> -> Test2::Plugin::NoWarnings 0.10
-> -> -> -> IPC::Run3 0.049
-> -> -> -> Module::Pluggable 6.3
-> -> -> Test::Without::Module 0.23
-> -> Test::File::ShareDir::Dist 1.001002 (in Test::File::ShareDir)
-> -> -> Class::Tiny 1.008
-> -> -> File::Copy::Recursive 0.45
-> -> -> -> Test::File 1.995
-> -> -> Scope::Guard 0.21
-> -> namespace::autoclean (0.19) 0.31
-> -> -> B::Hooks::EndOfScope (0.12) 0.28
-> -> -> -> Sub::Exporter::Progressive (0.001006) 0.001013
-> -> -> -> Variable::Magic (0.48) 0.64
-> -> -> namespace::clean (0.20) 0.27
-> -> -> -> Package::Stash (0.23) 0.40
-> -> -> -> Package::Stash::XS (0.26) 0.30
-> DateTime::TimeZone (2.44) 2.68
-> -> Class::Singleton (1.03) 1.6
DateTime::Format::XSD 0.4
-> DateTime::Format::ISO8601 0.19
-> -> DateTime::Format::Builder (0.77) 0.83
-> -> -> DateTime::Format::Strptime (1.04) 1.80
-> -> -> Params::Validate (0.72) 1.31
Note difference between this and Params::ValidationCompiler, same author [NOT ON MAKE FUCK YOU]
DateTime::HiRes 0.04
-> CPAN::Meta::Check (0.011) 0.018
-> DateTime::Locale (1.06) 1.45 2+ MB TGZ
-> -> File::ShareDir::Install 0.14
-> -> Dist::CheckConflicts (0.02) 0.11
-> -> -> Module::Runtime (0.009) 0.018
-> -> File::ShareDir 1.118
-> -> -> Class::Inspector (1.12) 1.36
-> -> IPC::System::Simple 1.30
-> -> Params::ValidationCompiler (0.13) 0.31
-> -> -> Eval::Closure 0.14
-> -> -> -> Test::Requires 0.11
-> -> -> Exception::Class 1.45
-> -> -> -> Class::Data::Inheritable (0.02) 0.10
-> -> -> -> Devel::StackTrace (2.00) 2.05
-> -> -> Specio (0.14) 0.53
includes Specio::Declare, Specio::Exporter, Specio::Library::___, Specio::Subs
-> -> -> -> Clone 0.50
-> -> -> -> -> B::COW (0.004) 0.007 MOO
-> -> -> -> Clone::PP 1.08
-> -> -> -> MRO::Compat 0.15
-> -> -> -> Module::Implementation 0.09
-> -> -> -> Role::Tiny (1.003003) 2.002005
includes Role::Tiny::With
-> -> -> -> Sub::Quote 2.006009 dumbest inclusion ever
-> -> -> -> XString 0.005
-> -> -> Test2::Plugin::NoWarnings 0.10
-> -> -> -> IPC::Run3 0.049
-> -> -> -> Module::Pluggable 6.3
-> -> -> Test::Without::Module 0.23
-> -> Test::File::ShareDir::Dist 1.001002 (in Test::File::ShareDir)
-> -> -> Class::Tiny 1.008
-> -> -> File::Copy::Recursive 0.45
-> -> -> -> Test::File 1.995
-> -> -> Scope::Guard 0.21
-> -> namespace::autoclean (0.19) 0.31
-> -> -> B::Hooks::EndOfScope (0.12) 0.28
-> -> -> -> Sub::Exporter::Progressive (0.001006) 0.001013
-> -> -> -> Variable::Magic (0.48) 0.64
-> -> -> namespace::clean (0.20) 0.27
-> -> -> -> Package::Stash (0.23) 0.40
-> -> -> -> Package::Stash::XS (0.26) 0.30
-> DateTime::TimeZone (2.44) 2.68
-> -> Class::Singleton (1.03) 1.6
DateTime::Format::XSD 0.4
-> DateTime::Format::ISO8601 0.19
-> -> DateTime::Format::Builder (0.77) 0.83
-> -> -> DateTime::Format::Strptime (1.04) 1.80
-> -> -> Params::Validate (0.72) 1.31
Note difference between this and Params::ValidationCompiler, same author [NOT ON MAKE FUCK YOU]
DateTime::HiRes 0.04
Now, I'll grant that many of these dependencies were needed at later upper levels, including by other libraries below. I'd probably have had to install about a quarter of these anyway, even if I wasn't using DateTime. However, that doesn't change the fact that I resent having to use DateTime in the first place. DateTime and I have a history.
When I first began writing code for this employer, which was getting on toward twenty years ago, I inherited someone else's codebase. And he was an extremely quirky human being. I deduce that by implication (I never met him) because of some of his, uh, interesting code decisions. One of those decisions I didn't have a problem with at the time was his use of DateTime. But I got tired of dealing with it fairly early on, and it was one of the first of his decisions I ditched. I have successfully avoided it ever since then -- until now. And I am not happy about it.
DateTime is a date and time handler. It is object-oriented; to use it, you must first make a new object of type DateTime, then initialize it to a particular date and time, like setting a clock. You can then do operations on that object, e.g. if I've made a new DateTime object called $dt, I can manipulate its contents:
$dt->add( months => 1, days => 1 ); # add one month and one day
Or retrieve some portion of it:
$dt->day_of_week # returns e.g. "Monday"
$dt->add( months => 1, days => 1 ); # add one month and one day
Or retrieve some portion of it:
$dt->day_of_week # returns e.g. "Monday"
The problem is, unless you are doing certain very specific things, the ratio of overhead to convenience in this library is very poor. The overhead you can see for yourself from the dependency list above and the fact that two of its inclusions are enormous as Perl libraries go: DateTime::Locale is over two megabytes compressed -- it may have unpacked to a lot larger than that, because it'd be all text and text compresses well, and DateTime::TimeZone is nearly a megabyte (again, compressed). That may not seem large, in these days of file bloat, but most of these packages average about 100-200 kilobytes compressed. In other words, those two packages are ten times larger than average.
Locale is probably huge because it is full of all the text strings for times and days and months for a lot of different countries. And that's one of the few places DateTime will earn its keep. If you're doing internationalization, if you're translating between time zones all over the world and you need to be able to ask it "tell me what day of the week this is, but in German," then, yes, use DateTime with a clear conscience.
But I work in one locale and one time zone. There is nobody who uses my code outside the United States, and only one person I know of who uses it from outside Eastern time. He's probably the only person who uses it outside an extremely narrow portion of Massachusetts, to be honest.
And the date manipulations are simultaneously overdesigned and insufficient. I note that DateTime has never had two convenience functions which would make the thing much more useful: there is no $dt->tomorrow or $dt->yesterday. Those are the kinds of things I do sometimes find myself doing to dates, and so I just do them myself.
It's very easy to calculate via math-with-overflow -- "Add one to the day, wait, check that against your "days in each month" table ... oops, no, there is no May 32, OK, add one to month and reset day to 1, it's June 1 now" ... and so on. There are only two places where it gets a little hairy: February and weekday names.
To do the former, you need one line of special code:
if (($tyear % 4 == 0 && $tyear % 100 != 0) || $tyear % 400 == 0)
The percent sign in Perl is the modulo (remainder) operator. What this says in English is, if the year is evenly divisible by four AND is NOT evenly divisible by 100, OR is evenly divisible by 400, it's a leap year. (And if you didn't know before this that the leap-year rule is more complex than "every four years," you do now.)
if (($tyear % 4 == 0 && $tyear % 100 != 0) || $tyear % 400 == 0)
The percent sign in Perl is the modulo (remainder) operator. What this says in English is, if the year is evenly divisible by four AND is NOT evenly divisible by 100, OR is evenly divisible by 400, it's a leap year. (And if you didn't know before this that the leap-year rule is more complex than "every four years," you do now.)
To deal with weekday names, you have to keep a separate iteration as you advance and retreat -- "OK, it was Monday, now it's Tuesday" -- and I usually don't bother.
I also have a tendency to store dates internally as YYYYMMDD, always padded with zeros where needed to exactly eight digits, because I can then do date comparisons as math: 20260526 is later than 20260525 because the former is larger! (Try this One Secret Trick They Don't Want You To Know!)
Anyway. DateTime, urgh. But the person who wrote Net:SAML2 apparently uses it, so now I'm forced to use it too. And its tons of weird dependencies, including Sub::Quote, which even my favorite Crusty Old Perl Hand admitted was "kind of dumb," and B::COW, which is really when I felt the absurdity kick in. (COW means Copy On Write, and the library has a legitimate function, but you can't tell me the author wasn't having fun with it and/or hadn't slipped into delirium a bit:
It's important to always keep an accurate reference count for your cows. If you delete the last reference, your cow disappears.
The note about Params::Validate ... if you look further up you'll see this dependency tree also uses Params::ValidationCompiler. The same guy wrote both, and in the notes for the former he says
I would recommend you consider using Params::ValidationCompiler instead. That module, despite being pure Perl, is significantly faster than this one, at the cost of having to adopt a type system such as Specio, Type::Tiny, or the one shipped with Moose.
You'll notice that one set of DateTime dependencies, the one that uses ValidationCompiler, uses Specio; but another set of DateTime dependenices later does not, and has not changed to the newer faster one, and thus I have to install both of the fuckers.
Also, the older library was the first time encountering a Perl package that is on the Build system instead of MakeMaker. My first exposure to Build has not been a positive one. More on that below. I feel there is absolutely no compelling reason for Perl library installs to use anything but MakeMaker, at least until someone makes a strong case. You know where to reach me.
And now we come to Moose. This is my first exposure to Moose as well, and again, I'm not impressed.
Moose is, in the author's words, "... an extension of the Perl 5 object system. The main goal of Moose is to make Perl 5 Object Oriented programming easier, more consistent, and less tedious." [caps hers]
The problem with this is that I can already tell there will be a huge amount of overhead for something that you basically shouldn't be doing in Perl anyway. Perl is a linear, procedural scripting language for gluing things to other things. You should not be writing large systems in Perl. If you want to write object-oriented code, then for pity's sake go write in C++ or some shit like that. (Or even do it in Perl! It's not that bad. We were just discussing an object-oriented chunk of code up there in DateTime!)
I don't want to knock the person responsible for most of Moose. It seems like a lot of people use it, so I guess the need is real. (Also, it's the rare project in the Perl community from An Woman, and I'd like to see a lot more of that. So.)
But this was the most annoying of these large chunks of dependencies. Not only did it use Build instead of MakeMaker in several places, but the package author in question always sets her tarballs with the wrong permissions, so when I unpacked them I had to chmod the resulting directory. Every time.
(I did all these installations prefixing every single command with 'sudo', you understand, so it wasn't something especially Byzantine, like "wrong group rights" or such. They were just plain wrong. You have to really break a directory for super-user not to be able to cd into it.)
Moose 2.4000 includes Moose::Role
-> Class::Load (0.09) 0.25
-> -> Data::OptList (0.110) 0.114
-> -> -> Params::Util 1.102
-> -> -> Sub::Install (0.921) 0.929
-> Class::Load::XS (0.01) 0.10
-> Devel::GlobalDestruction 0.14
-> Devel::OverloadInfo (0.005) 0.008
-> List::Util (1.56) 1.55 not by me -> in Scalar-List-Utils-1.70
-> Module::Runtime::Conflicts (0.002) 0.003
does nothing but check that Module::Runtime isn't a version that breaks Moose
-> Package::DeprecationManager (0.11) 0.18
-> Sub::Exporter (0.980) 0.991
-> Test::Needs (0.002010) 0.002006 not by me -> 0.002010
MooseX::Test::Role 0.08
MooseX::Types 0.51 includes MooseX::Types::Moose [NOT ON MAKE FUCK YOU]
AND has an uncaught dependency
-> Module::Build::Tiny (0.034) 0.053 [NOT ON MAKE]
but hardly surprising, it'd be ironic if it didn't use Build;
makes wrong perl path
-> Carp::Clan 6.08 hidden failed dependency (in test)
there's one but it's not in the right places
-> Sub::Exporter::ForMethods 0.100055 another hidden failed dependency in test
MooseX::Types::Common::String 0.001015
MooseX::Types::DateTime 0.14
MooseX::Types::URI 0.10
-> URI::FromHash 0.05 hidden failed dependency in test
-> Class::Load (0.09) 0.25
-> -> Data::OptList (0.110) 0.114
-> -> -> Params::Util 1.102
-> -> -> Sub::Install (0.921) 0.929
-> Class::Load::XS (0.01) 0.10
-> Devel::GlobalDestruction 0.14
-> Devel::OverloadInfo (0.005) 0.008
-> List::Util (1.56) 1.55 not by me -> in Scalar-List-Utils-1.70
-> Module::Runtime::Conflicts (0.002) 0.003
does nothing but check that Module::Runtime isn't a version that breaks Moose
-> Package::DeprecationManager (0.11) 0.18
-> Sub::Exporter (0.980) 0.991
-> Test::Needs (0.002010) 0.002006 not by me -> 0.002010
MooseX::Test::Role 0.08
MooseX::Types 0.51 includes MooseX::Types::Moose [NOT ON MAKE FUCK YOU]
AND has an uncaught dependency
-> Module::Build::Tiny (0.034) 0.053 [NOT ON MAKE]
but hardly surprising, it'd be ironic if it didn't use Build;
makes wrong perl path
-> Carp::Clan 6.08 hidden failed dependency (in test)
there's one but it's not in the right places
-> Sub::Exporter::ForMethods 0.100055 another hidden failed dependency in test
MooseX::Types::Common::String 0.001015
MooseX::Types::DateTime 0.14
MooseX::Types::URI 0.10
-> URI::FromHash 0.05 hidden failed dependency in test
The problem with Build, besides the fact that my fingers don't have the right reflexes for it, is that it doesn't do the same kind of seamless, painless, get-it-out-of-the-way-before-the-trouble-starts dependency checking that the MakeMaker approach does. It starts the same way ... perl Build.PL ... but then it always says "OK, good to go," and you are often not good to go, and running the Build file it just made may still not show you the problems. With Module::Build::Tiny, there were missing dependencies that didn't show up until I tried to do "Build test" (the equivalent of "make test"). This should never happen. Those dependencies should have been pointed out well before the test phase.
Also, unlike MakeMaker, Build does not understand that I am doing all this using a Perl that is not the same as the system Perl. Even though I tell it /usr/local/bin/perl Build.PL, it writes the wrong Perl path into the Build file. This caused problems in two places where it was trying to use stuff from the other Perl ... stuff I already know does not work!
The (wryly) amusing thing is that most of that pain came while I was trying to build a smaller version of the Build mechanism, which again, feels like "OK, I resent that you even handed me this pain just so you could have a somewhat simpler-to-use version of a system which really kind of isn't the way you should be doing any of this anyway."
Quirky. I tell you. They are all very quirky people.
Some odds and ends next, with one wry laugh.
Import::Into 1.002005
Sub::Override 0.12
Test::Lib 0.003
Test::Mock::One 0.011
Test::NoTabs 2.02
Test::Pod::Coverage (1.04) 1.10
-> Pod::Coverage 0.23
-> -> Devel::Symdump (2.01) 2.18
-> -> Pod::Parser (1.13) 1.67 includes Pod::Find
Types::Serialiser 1.01
An existing 1.0 installation was not detected
-> common::sense 3.74 Not THIS shit again
OK I know what happened, I didn't rebuild these; rebuilt w/o download
URI::Encode 1.1.1 which means installing a library for one line of perl
has both Build and makemaker, fascinating
URN::OASIS::SAML2 (0.007) 0.007
Sub::Override 0.12
Test::Lib 0.003
Test::Mock::One 0.011
Test::NoTabs 2.02
Test::Pod::Coverage (1.04) 1.10
-> Pod::Coverage 0.23
-> -> Devel::Symdump (2.01) 2.18
-> -> Pod::Parser (1.13) 1.67 includes Pod::Find
Types::Serialiser 1.01
An existing 1.0 installation was not detected
-> common::sense 3.74 Not THIS shit again
OK I know what happened, I didn't rebuild these; rebuilt w/o download
URI::Encode 1.1.1 which means installing a library for one line of perl
has both Build and makemaker, fascinating
URN::OASIS::SAML2 (0.007) 0.007
What happened with Types::Serializer is that a couple of years ago the Admin Gods did a thing to my system that broke my entire stack and I had to rebuild everything from scratch. In the process, a couple of dependencies became obsolete -- I no longer needed them. Apparently I left some of their build directories lying around. So I had Types::Serializer 1.0 but had never actually reinstalled it in the new regime. Unfortunately needing it again now means that I once again needed a dependency I was very pleased to have dropped.
common::sense, with its pretentious non-conforming name, is a library which I think was at least two-thirds written to troll the rest of the Perl community. The library is nothing much; it's a set of defaults the author considers sane (which levels of warnings turn on, forcing use of UTF8 by default, etc). On the face of it, it's harmless, and I agree with a lot of it. (If you don't have 'use strict vars' turned on at all times, you have a sign taped on your back saying GIVE ME TECHNICAL DEBT I LOVE IT.) It's just that the author is so fucking snotty about it. And it's on purpose, too. I can tell it's on purpose because he has actually posted some of his hatemail in the info page in the repository.
Anyway, it's small and harmless but I did not welcome it back.
URI::Encode is the library I picked on above ... 363 lines of code to do something I showed you in thirteen. (Not exactly the "one line of perl" in my notes, I admit.) I do note that it was the only package that would allow you to build it via either Build or MakeMaker methods ... which may mean its author prefers to sidestep that holy war entirely.
Finally, some XML. Specifically, XML::Enc which is for encrypting a block of XML, which is a SAML requirement so no surprise. The problem is, to encrypt some XML, you need some tools to read and write the block of XML in the first place. And I did not have those installed.
This is actually one of the places where I concede readily that you want a library. Writing your own XML parser is madness. Let someone more masochistic do that for you. Trust me, I know whence I speak: I maintained my own (crude) JSON parser for some years. When I began using Mojolicious, I noticed it had its own JSON read and write functions that did exactly what I was doing only better, and I switched to using those and never looked back.
No, the reason I've never installed any kind of XML library in Perl is that I've never needed one. I read and write XML all the time, you understand, but always in very limited ways. Like, I'll send a fixed block of about six lines of XML to some other system ... I'll just make those by hand, I'll use print statements. In another case, I get a large chunk of XML from an upstream system from which I need ONE item of data ... so I just parse for that specific tag and ignore the rest of the file.
But here we're obviously going to need an XML library, and XML::LibXML wants to use the C libxml2 library. Which, fine, but the thing is, they use Alien to do it. Alien is a tool for obtaining and using non-Perl resources (such as C libraries) in Perl builds ... including fetching them from external repositories on the fly. Here, the implication of the library names was that this build was going to try to download libxml2 from Gitlab ... which, fine, it worked, but what happens if Gitlab goes away?
This is made even more annoying by the fact that I strongly suspect I have the C libxml2 lying around on that system somewhere and I'd have been happy to go look for it and tell the Perl build where it was -- if the Perl build had given me an opportunity to do that instead of rushing straight to Alien. Alien could have potentially been a very big build; it could have resulted in a dependency tree nearly as time-consuming as the DateTime one. However, it happened that I already had most of it installed (which is why most of it doesn't show up here).
XML::Enc (0.13) 0.16
-> Crypt::PK::RSA (0.081) 0.078 -> 0.089 (in CryptX)
-> XML::LibXML 2.0213 includes XML::LibXML::XPathContext
-> -> Alien::Libxml2 0.20
Now I suspect I already have the C libxml2 installed but there may be a path issue, so.
-> -> -> Alien::Build::Plugin::Download::Gitlab 0.01
OK so this fucker's going to try to download a C library from Gitlab, eh?
-> -> XML::NamespaceSupport (1.07) 1.12
-> -> XML::SAX::Base 1.09 includes XML::SAX::Exception
-> -> XML::SAX (0.11) 1.02 includes XML::SAX::DocumentLocator
XML::Generator (1.13) 1.13
XML::Sig (0.67) 0.70
-> Class::Accessor 0.51
-> Crypt::OpenSSL::Bignum 0.09
-> Crypt::PK::RSA (0.081) 0.078 -> 0.089 (in CryptX)
-> XML::LibXML 2.0213 includes XML::LibXML::XPathContext
-> -> Alien::Libxml2 0.20
Now I suspect I already have the C libxml2 installed but there may be a path issue, so.
-> -> -> Alien::Build::Plugin::Download::Gitlab 0.01
OK so this fucker's going to try to download a C library from Gitlab, eh?
-> -> XML::NamespaceSupport (1.07) 1.12
-> -> XML::SAX::Base 1.09 includes XML::SAX::Exception
-> -> XML::SAX (0.11) 1.02 includes XML::SAX::DocumentLocator
XML::Generator (1.13) 1.13
XML::Sig (0.67) 0.70
-> Class::Accessor 0.51
-> Crypt::OpenSSL::Bignum 0.09
Anyway, so, that was a lot, I know, and I doubt anyone actually made it this far. But I'm posting it here also for my future reference, in addition to my build log in my work wiki. Other people read that on rare occasions, so the extended ranting would have been a bit inappropriate there. (The profanity they're accustomed to.)
I was able to devote the time to writing this because, by the time I finished the download and install process today (remember, that was basically my second full day of doing nothing but), I was too mentally exhausted to actually start testing and working with the library that was the end object of all this in the first place.
It's going to be hilarious when I start messing with it, tomorrow or Thursday, and it doesn't fucking work.
26 May 2026