Autodocumenting bash script using perldoc

I may know how to code in C/C++/C#/VHDL/bash/python/Perl my prefered language will always be the one that takes the less line for the job, and I will have no SHAME mixing them.

-- Doctor Frankenstein

I would rate myself as a python coder, but sometimes, I don't care which language I « prefer », there is a task and there are languages easier to use for one tasks.

Here, on my last toy project originating from a poc I chose bash because when it comes to spawning processes, sending and catching signals (python is way unreliable when it come to signal delivery (note to myself link to my post experminenting on this topic)), bash is better and above all « way more concise ». I don't want to fiddle with subprocess.popen.PIPE when I can add a & at the end or using « $() ». Also the ${VAL:-default_val} is really concise when it comes with dealing with variable passed by environment.

But, something DOES not exists in bash that exists in python : self documentation thanks to docstring and SPHINX.

And I love documentation.

So the other day, I was butchering the munin plugins to make them fit in my solution when I noticed this weired pattern :

#!/bin/sh
# -*- sh -*-

: << =cut

=head1 NAME

interrupts - list number of interrupts since boot (linux) or the interrupt rate per interrupt

=head1 CONFIGURATION

=cut

As a former Perl coder, I recognized both a perl HEREDOC and a shell HEREDOC. A way of partially doing a polyglot code. A plolyglot is valid code for two languages like this one in python+Perl

q = 0 or """ #=;$A=41;sub A { ~-$A+2};  A() && q' """
A=lambda A: -~A  #';
print( A(41) ) # python + perl = <3

So you can do : perldoc ./bash_with_here_doc_called_=cut.sh and it will yield a doc because someone found a way to transform a shell script in a valid enough Perl with heredoc. What a trick.

I am not fond of the perldoc markup syntax that is a tad too verbose for me, but I really appreciate the redactionnal advices given by the Perl community. And, when you code, the markup language is readable enough that it acts as a good comment on the API.
I may have left my occupation as a paid Perl developer, it still has a lot of community related trumps and practices I do appreciate. So the deal is done, I will live with pod verbosity in exchange for clear docs.

Soo without further ado here is how I document a script with the script itself :

#!/usr/bin/env bash
<< '=cut'
=head1 NAME

start.sh

=head2 DESCRIPTION

Launches the networked apparatus of measures. It is the reciprocal function
of stop.sh

=head2 SYNOPSIS

All arguments are passed by environment variables

    [TICK=2] [LURKER=] [BROADCAST=192.168.1.255] [RANGE=24] [SINCE=900] start.sh

=head2 OPTIONS

=over

=item TICK

TICK is the initial clock given to the system. It will however converge
to its computed value.

=item LURKER

When LURKER is set, the data collecting agent is launched and process 
all probes sent on the given broadcast address

=item BROADCAST

UDP BROADCAST address to use

=item RANGE

Range in the form [0-32] to specify the BROADCAST range. 

Ex: 24 will specify $BROADCAST/24

=item SINCE

Argument given to the html generator to know how much seconds since NOW must
be shown in the graph.

=back

=cut

LURKER=${LURKER:-}
cd $( dirname $0 )
HERE="."
SINCE=${SINCE:-900}
TICK=${TICK:-2}
BROADCAST=${BROADCAST:-"192.168.1.255"}
RANGE=24
export TICK
$HERE/stop.sh 
echo $$ > $HERE/pid/$( basename $0).pid
[ -e pid ]  || mkdir pid
[ -e log ]  || mkdir log
[ -e run ]  || mkdir run
[ -e data ] || mkdir data
if [ ! -z "$LURKER" ]; then
    SINCE=$SINCE DAEMON=1 $HERE/bin/mkhtml.sh &
    LURKER=$LURKER $HERE/bin/launch_lurker.sh &
fi
BROADCAST=$BROADCAST RANGE=$RANGE TICK=$TICK $HERE/bin/launch_writer.sh &
TICK=$TICK $HERE/bin/clock.sh &

which generates

$ perldoc -o text start.sh

NAME
    start.sh

  DESCRIPTION
    Launches the networked apparatus of measures. It is the reciprocal
    function of stop.sh

  SYNOPSIS
    All arguments are passed by environment variables

        [TICK=2] [LURKER=] [BROADCAST=192.168.1.255] [RANGE=24] [SINCE=900] start.sh

  OPTIONS
    TICK
        TICK is the initial clock given to the system. It will however
        converge to its computed value.

    LURKER
        When LURKER is set, the data collecting agent is launched and
        process all probes sent on the given broadcast address

    BROADCAST
        UDP BROADCAST address to use

    RANGE
        Range in the form [0-32] to specify the BROADCAST range.

        Ex: 24 will specify $BROADCAST/24

    SINCE
        Argument given to the html generator to know how much seconds since
        NOW must be shown in the graph.

Which also can help generates the man pages if required in the future.

So to generates AUTOMATICALLY all my docs and user my README.md (in markdown) I borrow the power of pandoc and bash :

#!/usr/bin/env bash
<< =cut

=head1 NAME

mkdoc.sh

=head2 SYNOPSIS

Generates the doc. Requires pandoc for markdown to html conversion

    ./mkdoc.sh

=cut

rm doc -rf
[ -d doc ] || mkdir -p doc/img
cp img/* doc/img/

for i in $( find . -name "*sh" ); do 
    DST="doc/$( dirname $i)"
    [ -d $DST ] || mkdir -p $DST
    pod2html --htmlroot=doc --htmldir=doc "$i" > "$DST/$(basename $i).html"
done
for i in $( find ./plugin -type f -not -path "./plugin/*enabled" ); do 
    DST="doc/$( dirname $i)"
    [ -d $DST ] || mkdir -p $DST
    pod2html "$i" > "$DST/$( basename $i).html"
done
RES=""
cd doc
pandoc ../README.md -o index.html
echo "<ul>" >> index.html

for i in $( find . -name "*html" -a -not -path ".*.git*" | sort ); do
    echo "<li><a href=$i > $(dirname $i)/$( basename $i .html )</a></li>" >> index.html ;
done
echo "</ul>" >> index.html
# and let's generate markdown and make them able to cross reference each others in both html and md
for i in $( find . -name "*html" -a -not -path ".*.git*" | sort ); do
    OUT="$(dirname $i)/$( basename $i .html ).md"
    pandoc $i -o "$OUT"
    perl -i -ane 's/\((.*).html\)/\($1.md\)/ and print $_ or print $_'  $OUT~
done

And it populates a doc dir with all the html.
At the end of the day, I have a bash framework that is self documented à la python sphinx with less overhead.

EDIT : it you can also document a python code with perldoc :

#!/usr/bin/env python3
"""<< =cut

=head1 NAME

launch_lurker.py

=head2 SYNOPSYS

[PORT=6666] ./launch_lurker

=head2 OPTIONS

see L<file:../start.sh.html> for explanation of the options
clock tick 37
=cut
"""
from socket import *
import os
...

Gives :

perldoc bin/launch_lurker.py 
clock tick 46
LAUNCH_LURKER.PY(1)   User Contributed Perl Documentation  LAUNCH_LURKER.PY(1)


NAME
       launch_lurker.py

   SYNOPSYS
       [HOST=0.0.0.0] [PORT=6666] ./launch_lurker

   OPTIONS
       see <file:../start.sh.html> for explanation of the options =back

perl v5.36.3                      2024-06-29               LAUNCH_LURKER.PY(1)
(END)