Comment la culture métier informatique en entreprise est en conflit avec l'innovation

La culture « métier »

Celui à qui il arrive des expériences tragiques qui se répètent
a quelque chose de tragique en lui.

-- Nietzsche Le gai savoir

Mon cas, comme celui du doude, et moult autodidactes a une composante tragique: faire face à la culture métier.

Levi Strauss définit la culture comme du formel qui s'ignore, la culture métier est un ensemble de règles acquises souvent d'abord dans le milieu familial puis éducatif (et dans cet ordre et non l'inverse au vu des statistiques INSEE sur la reproduction sociale).

Aux optimistes chevronnées considèrent qu'il est possible de déchiffrer des règles qu'on ignore et de réussir un jour à s'adapter à une culture inconnue, je rétorque que l'humain a un temps de vie limité, et qu'on a déjà pas mal de cultures plus importantes à déchiffrer avant la culture professionnelle.

Imaginons voir les cultures comme des lumières disposées autour de nous, certaines proches et brillantes comme celle de l'école quand on y est exposé 8 heures par jours cinq jours par semaines, celle distante du comportement des acteurs dominants du métier que l'on vise, celle de la rue à laquelle les gens modestes sont exposés et ainsi de suite. Chacune de ces cultures projettent des halos d'ombres qui symbolisent les « tabous ».

Par exemple, si à l'école il est tabou de fumer un joint on peut se retrouver dans un milieu étudiant où la socialisation passe par prendre un rail de coke après les cours, dans la même journée. Ainsi, naviguer au milieu de ces différentes cultures tient, surtout pour des gens aveuglés par d'autres choses que les « cultures » comme une constante marche au milieu de champs de mines contradictoires. Certaines connues (les règles et les lois) et d'autres grises (comme l'acceptation de la triche dans certains milieux scolaires avec parfois l'aide des professeurs).

Qu'est-ce qui peut rentrer en conflit avec la culture ?

La culture est littéralement, une émanation du conservatisme, le socle de bon sens partagé par un groupe.

La pratique autodidacte d'une activité amène rapidement à diverger de la culture. Si le débat de savoir si la divergence est positive ou négative est laissée à l'appréciation de chacun, la divergence est néanmoins la source de l'innovation qui rentre en conflit avec la culture établie. Il appartient aux organisations qui se désirent innovantes d'établir une culture de bienveillance face à la divergence, et ensuite de tri sur des critères qui lui sont propres entre les divergences positives et négatives. Un réseau de drogue et une association scout n'ont pas les mêmes critères d'innovation bénéfiques. Force est de constater que l'entreprise est malheureusement le lieu où la reproduction prime sur la divergence sans négociations.

Ainsi, dans la pratique informatique on trouvera un monde « professionnel » façonné par les cadres issus des classes de la bourgeoisie intellectuelle dont l'INSEE garantie qu'elle est un artefact de la reproduction sociale qui a comme culture d'éviter l'innovation disruptive, et de l'autre, il faut pour produire des biens et services neufs des praticiens dont l'exercice du métier diverge de la « norme culturelle » : ce sont les sherpas.

Depuis l'antiquité, le conservatisme culturel a trouvé son arme de choix : le cléricalisme. L'interprétation des textes par une minorité qui détient la Vérité de l'interprétation des textes. Ce n'est plus le texte la référence, mais une caste autorisée à commenter le texte.

De même qu'un prof de philo réussira à vous faire croire que Platon l'inspiration même du proto-facisme est le père des systèmes politiques démocratiques, un scrum master (spécialiste des méthodes agiles), enkystera une organisation dans une interprétation standardisée de « l'agile manifesto » dont le point centrale du texte est le refus de l'enkystement, de la standardisation.

De fait, les sherpas sont socialement en bas de l'échelle sociale du code, mais aussi les acteurs de l'innovation. Et c'est pas compliqué à comprendre : la pratique rend certaines choses évidentes quand on partage son code qui ne sont pas enseignées. Par exemple, ce qu'on appelle en informatique « le packaging ».

L'empaquetage en français consiste à rendre votre code aussi original soit-il utilisable par d'autres utilisateurs d'une manière standardisée, et je ne connais qu'une méthode pour arriver à bien le faire : pratiquer encore et toujours. Étant moi même mainteneur de modules python, je pratique encore et toujours et ne peut que constater le fossé qui sépare l'approche quotidienne de cette pratique avec l'approche dite « professionnelle » : les modules hors entreprises sont à de rares exceptions près (je citerais sentry en bon exemple, dont le CTO avait un historique d'excellent mainteneur avant de prendre son poste) de meilleures qualités que tout ce que j'ai pu voir fourni par les entreprises.

Pour fournir un bon module il faut : respecter des standards de faits qui sont assez souvent mobiles et multiples, tester, suivre des normes de traçabilité (versionage), de qualité (gestion de ticket), et documenter. Entre nous c'est si chiant que quand on aime développer et non maintenir ce qui est mon cas on tente de suivre « le fil du bois » pour passer le moins de temps possible sur une tâche qui n'a comparé au muscle du module qu'un intérêt intellectuel marginal. Étonnamment quand vous lisez les normes ITIL, ISO et autres, vous vous apercevez que cela semble la partie qui intéresse le plus les pros, et qui produit ironiquement les résultats les moins conformes aux attentes.

Je vais illustrer ceci avec une histoire vraie.

La fable de l'autodidacte et des ingénieurs sur le canal IRC

L'autre jour, alors que je m'ennuyais et que je devais me rendre à l'évidence que j'avais perdu mon accordeur de guitare, je décidais de rejoindre un canal IRC de développeur passionné en informatique. Donc passionné non de conditionner le logiciel mais de faire.

Ce qui m'a surpris de prime abord, c'était qu'il parlait plus de combien coûteux étaient leur hobby dont la radioastronomie que de partager leur passion avec le plus grand nombre, comme si la passion se devait d'être un club réservé à une élite qui en a les moyens. J'ai laissé pissé et j'ai lourdement parlé code.

Ils m'ont alors dirigé vers leur gestionnaire de source pour voir si je pouvais compiler leur propre projet.

J'en avais absolument pas envie ; d'une part un truc fait pour la radioastronomie en tête me hérissait le poil, d'autre part, le minimum syndical pour aider l'utilisateur -c'est à dire une doc claire à minima pour reproduire la version binaire- était absent.

Ayant cru à un bizutage, pour le fun, j'ai fait ce qu'il demandait, constatant que le code n'avait pas été fait pour être partagé mais tourner sur la machine d'une personne, j'ai du modifier le code dont je n'ai pu que constater qu'il était éparpillé comme un impact de chevrotine en maugréant.

Une fois ce truc réussit, ils s'attendaient à ce que je fasse la QA, remplisse les bugs, soumettent les patchs pour miraculeusement obtenir à moindre effort ce qui demande d'être pensé à la base. Et je me suis dit : nan et j'avais pas envie de partir dans une allégorie de la maison bâtie qui bâtie sur du sable mou ne tiendrait pas.

Une fois le bizutage terminé, je me suis dit que j'avais aussi le droit à partager mes codes, sachant qu'en général j'indique les dépendances et que je code le plus souvent en langage interprété (python) est facile à tester.

Alors, j'ai partagé mon code d'accordeur de guitare v1 (fait avec un spectrosonogramme car c'est classieux) puis v2 (garanti précis au hertz près) et je me suis fait rembarré « nous fait pas chier chatgpt (un programme dit d'IA) fait mieux que ton code prétentieux en moins de ligne de code ».

J'ai pointé le fait qu'il fallait pas confondre du code qui théoriquement marche avec un code pratique. L'application des lois connues du traitement du signal (qui font partis de l'enseignement de base en ingénierie) prouvait que chatgpt avait fourni un code incapable de discerner l'octave le plus bas sur la guitare (et surtout la basse qui est mon instrument).

Mais ils s'en foutaient, faire une détection de pic de fréquence en informatique en utilisant des transformées de Fourier rapide est un sujet trivial, partager le fait de le faire pragmatiquement pour actuellement accorder un instrument de musique et faire attention aux détails ne leur semblait pas important.

En fin de compte, ce chan annoncé comme celui de fans d'informatique, n'était qu'un canal de pro de l'informatique qui passaient juste leur temps à parler de tout leurs hobbies coûteux sauf de l'informatique. Et j'ai parti. Ça sert à rien de perdre son énergie là où l'on est pas bienvenu.

C'est représentatif de mon expérience professionnelle avec la culture « métier », où ce qui prime n'est pas de faire avec le code, mais de coder de préférence de la « bonne manière ». Genre, le langage, la façon de structurer le code, le nom des variables l'emporte sur l'objectif à résoudre.

C'est aussi représentatif à mon goût du fossé qui sépare les cadres en entreprises qui gèrent la fonction informatique de ceux qui la font en étant issus de parcours où la pratique a été le facteur de survie plus que la conformité sociale.

Si la conformité sociale est parfois souhaitable (je trouve sain une association scout qui fait perdurer la pratique de s'assurer que les adultes passionnés des enfants soient écartés de l'encadrement des enfants), quand il s'agit de domaine qui se prétendent innovant, je ne suis pas sûr que ce soit un atout.

Et ce que je viens de dire en entreprise n'est pas socialement acceptable, car en entreprise on évite de révéler les conflits : l'entreprise n'a jamais eu pour but de faire des bénéfices (c'est une condition nécessaire, mais non suffisante), mais d'apporter du statut social à ceux qui la dirige.

L'entreprise est de fait basée sur un conflit social permanent dont le moteur sont les sherpas auquel il est donné comme tabou de critiquer la discrimination négative qui les vise.

Is chatgpt good at generating code for tuning a guitar ?

I was on a french speaking IRC chan bragging a tad about how I was doing a guitar tuner and paying attention to not fall into the pit of confusing precise and exact figures as a random computer engineer.

Science he was a patented CS engineer he wanted to prove me that my new guitar tuner was useless since AI could come with a better less convoluted exact example in less lines of code than mine (mine is adapted from a blog on audio processing and fast fourier transform because it was commented and was refreshing me the basics of signal processing).

And I asked him, have you ever heard of the Nyquist frequency ? or the tradeoff an oscilloscope have to do between time locality and accuracy ?

Of course he didn't. And was proud that a wannabee coder would be proven useless thanks to the dumbest AI.



So I actually made this guitar tuner because this time I wanted to have an exact figure around the Hertz.
The problem stated by FFT/Nyquist formula is that if I want an exact number around 1Hz (1 period per second) I should sample at least half a period (hence .5 second), and should not expect a good resolution.

The quoted chatgpt code takes 1024 samples out of 44100/sec, giving it a nice reactivity of 1/44th of a second with an accuracy of 44100/1024/2 => 21Hz.

I know chatgpt does not tune a guitar, but shouldn't the chatgpt user bragging about the superiority of pro as him remember that when we tune we may tune not only at A = 440Hz but maybe A = 432Hz or other ?

A note is defined as a racine 12th of 2 compared to an arbitrary reference (remember an octave is doubling => 12 half tones = 2) ; what makes a temperate scale is not the reference but the relationship between notes and this enable bands of instrument to tune themselves according to the most unreliable but also nice instrument which is the human voice.

Giving the user 3 decimal after the comma is called being precise : it makes you look like a genius in the eye of the crowd. Giving the user 0 decimal but accurate frequency is what makes you look dumb in the eyes of the computer science engineer, but it way more useful in real life.

Here I took the liberty with pysine to generate A on 6 octaves (ref A=440) and use the recommanded chatgpt buffer size acknowledged by a pro CS engineer for tuning your guitar and my default choices.
for i in 55.0 110.0 220.0 440.0 880.0 1760.0 ; do python -m pysine $i 3; done
Here is the result with a chunk size of 1024 :
And here is the result with a chunk size corresponding to half a second of sampling :

I may not be a computer engineer, I am dumb, but checking with easy to use tools that your final result is in sync with your goal is for me more important than diplomas and professional formations.



The code is yet another basic animation in matplotlib with a nice arrow pointing the frequency best fitting item. It is not the best algorithm, but it does the job.

Showing the harmonics as well as the tonal has another benefit it answers the questions : why would I tune my string on the note of the upper string ?
Like tuning E on A ?

Well because -at least for my half broken guitar- it ensures to me that I will tune on the tonal note.

Here is a sample of tuning E on the empty string :
And this is how tuning the E string on the A note looks like :
And don't pay attention to the 56Hz residual noise triggered by my fans/appliance turning and making a constant noise :D Here is the code
import pyaudio
import matplotlib.pyplot as plt
import matplotlib.animation as animation
import numpy as np
import time
from sys import argv

A = 440.0
try:
    A=float(argv[1])
except IndexError:
    pass

form_1 = pyaudio.paInt16 # 16-bit resolution
chans = 1 # 1 channel
samp_rate = 44100 # 44.1kHz sampling rate
chunk = 44100//2# .5 seconds of sampling for 1Hz accuracy

audio = pyaudio.PyAudio() # create pyaudio instantiation

# create pyaudio stream
stream = audio.open(
    format = form_1,rate = samp_rate,channels = chans,
    input = True , frames_per_buffer=chunk
)

fig = plt.figure(figsize=(13,8))
ax = fig.add_subplot(111)
plt.grid(True)
def compute_freq(ref, half_tones):
    return [ 1.0*ref*(2**((half_tones+12*i )/12)) for i in range(-4,4)   ]

print(compute_freq(A,0))
note_2_freq = dict(
    E = compute_freq(A,-5),
    A = compute_freq(A, 0),
    D = compute_freq(A, 5),
    G = compute_freq(A,-2),
    B = compute_freq(A, 2),
    )
resolution = samp_rate/(2*chunk)

def closest_to(freq):
    res = dict()
    for note, freqs in note_2_freq.items():
        res[note]=max(freqs)
        for f in freqs:
            res[note]= min(res[note], abs(freq -f))
    note,diff_freq = sorted(res.items(), key = lambda item : item[1])[0]

    for f in note_2_freq[note]:
        if abs(freq-f) == diff_freq:
            return "%s %s %2.1f %d" % (
                note,
                abs(freq - f ) < resolution and "=" or
                    ( freq > f and "+" or "-"),
                abs(freq-f),
                freq
            )

def init_func():
    plt.rcParams['font.size']=18
    plt.xlabel('Frequency [Hz]')
    plt.ylabel('Amplitude [Arbitry Unit]')
    plt.grid(True)
    ax.set_xscale('log')
    ax.set_yscale('log')
    ax.set_xticks( note_2_freq["E"] + note_2_freq["A"]+
                  note_2_freq["D"]+ note_2_freq["G"]+
                  note_2_freq["B"] ,
                labels = (
                    [ "E" ] * len(note_2_freq["E"]) +
                    [ "A" ] * len(note_2_freq["A"]) +
                    [ "D" ] * len(note_2_freq["D"]) +
                    [ "G" ] * len(note_2_freq["G"]) +
                    [ "B" ] * len(note_2_freq["B"])
                    )
    )

    ax.set_xlim(40, 4000)
    return ax

def data_gen():
    stream.start_stream()
    data = np.frombuffer(stream.read(chunk),dtype=np.int16)
    stream.stop_stream()
    yield data
i=0
def animate(data):
    global i
    i+=1
    ax.cla()
    init_func()
    # compute FFT parameters
    f_vec = samp_rate*np.arange(chunk/2)/chunk # frequency vector based on window
                                               # size and sample rate
    mic_low_freq = 50 # low frequency response of the mic
    low_freq_loc = np.argmin(np.abs(f_vec-mic_low_freq))
    fft_data = (np.abs(np.fft.fft(data))[0:int(np.floor(chunk/2))])/chunk
    fft_data[1:] = 2*fft_data[1:]
    plt.plot(f_vec,fft_data)

    max_loc = np.argmax(fft_data[low_freq_loc:])+low_freq_loc

# max frequency resolution
    plt.annotate(r'$\Delta f_{max}$: %2.1f Hz, A = %2.1f Hz' % (
            resolution, A), xy=(0.7,0.92), xycoords='figure fraction'
    )
    ax.set_ylim([0,2*np.max(fft_data)])

    # annotate peak frequency
    annot = ax.annotate(
        'Freq: %s'%(closest_to(f_vec[max_loc])),
        xy=(f_vec[max_loc], fft_data[max_loc]),\
        xycoords='data', xytext=(0,30), textcoords='offset points',
        arrowprops=dict(arrowstyle="->"),
        ha='center',va='bottom')
    #fig.savefig('full_figure-%04d.png' % i)
    return ax,

ani = animation.FuncAnimation(
    fig, animate, data_gen, init_func, interval=.15,
    cache_frame_data=False, repeat=True, blit=False
)
plt.show()

Tune your guitar with python

Today's exercice is just about turning a very nice example of the python soundevice module into something that works for me© to help me tune my bass.

Long story short, I suck at tuning my instrument and just lost my tuner...

This will require the python module soundevice and matplotlib.

So in order to tune my guitar I indeed need a spectrosonogram that displays the frequencies captured in real time by an audio device with an output readable enough I can actually know if I am nearing a legit frequency called a Note.

The frequencies for the notes are pretty arbitrary and I chose to only show the frequency for E, A , D, G, B since I have a 5 strings bass.
I chose the frequency between 100 and 2000 knowing that anyway any frequency below will trigger harmonics and above will trigger reasonance in the right frequency frame.

Plotting a spectrogram is done by tweaking the eponym matplotlib grapher with values chosen to fit my need and show me a laser thin beam around the right frequency.
#!/usr/bin/env python3
"""Show a text-mode spectrogram using live microphone data."""
import argparse
import math
import shutil
import matplotlib.pyplot as plt
from multiprocessing import Process, Queue
import matplotlib.animation as animation

import numpy as np
import sounddevice as sd

usage_line = ' press enter to quit,'

def int_or_str(text):
    """Helper function for argument parsing."""
    try:
        return int(text)
    except ValueError:
        return text

try:
    columns, _ = shutil.get_terminal_size()
except AttributeError:
    columns = 80

parser = argparse.ArgumentParser(add_help=False)
parser.add_argument(
    '-l', '--list-devices', action='store_true',
    help='show list of audio devices and exit')
args, remaining = parser.parse_known_args()
if args.list_devices:
    print(sd.query_devices())
    parser.exit(0)
parser = argparse.ArgumentParser(
    description=__doc__ + '\n\nSupported keys:' + usage_line,
    formatter_class=argparse.RawDescriptionHelpFormatter,
    parents=[parser])
parser.add_argument(
    '-b', '--block-duration', type=float, metavar='DURATION', default=50,
    help='block size (default %(default)s milliseconds)')
parser.add_argument(
    '-d', '--device', type=int_or_str,
    help='input device (numeric ID or substring)')
parser.add_argument(
    '-g', '--gain', type=float, default=10,
    help='initial gain factor (default %(default)s)')
parser.add_argument(
    '-r', '--range', type=float, nargs=2,
    metavar=('LOW', 'HIGH'), default=[50, 4000],
    help='frequency range (default %(default)s Hz)')
args = parser.parse_args(remaining)
low, high = args.range
if high <= low:
    parser.error('HIGH must be greater than LOW')
q = Queue()
try:
    samplerate = sd.query_devices(args.device, 'input')['default_samplerate']
    def plot(q):
        global samplerate
        fig, ( ax,axs) = plt.subplots(nrows=2)
        plt.ioff()
        def animate(i,q):
            data = q.get()
            ax.clear()
            axs.clear()
            axs.plot(data)
            ax.set_yticks([
                41.20,	82.41,	164.8,	329.6,	659.3,  # E
                55.00, 	110.0, 	220.0, 	440.0, 	880.0,  # A
                73.42,	146.8,	293.7,	587.3,          # D
                49.00, 	98.00, 	196.0, 	392.0, 	784.0,  #G 
                61.74, 	123.5, 	246.9, 	493.9, 	987.8 ])#B 
            ax.specgram(data[:,-1],mode="magnitude", Fs=samplerate*2, scale="linear",NFFT=9002)
            ax.set_ylim(150,1000)
        ani = animation.FuncAnimation(fig, animate,fargs=(q,), interval=500)
        plt.show()

    plotrt = Process(target=plot, args=(q,))
    plotrt.start()

    def callback(indata, frames, time, status):
        if any(indata):
            q.put(indata)
        else:
            print('no input')

    with sd.InputStream(device=args.device, channels=1, callback=callback,
                        blocksize=int(samplerate * args.block_duration /50 ),
                        samplerate=samplerate) as sound:
        while True:
            response = input()
            if response in ('', 'q', 'Q'):
                break
            for ch in response:
                if ch == '+':
                    args.gain *= 2
                elif ch == '-':
                    args.gain /= 2
                else:
                    print('\x1b[31;40m', usage_line.center(args.columns, '#'),
                          '\x1b[0m', sep='')
                    break
except KeyboardInterrupt:
    parser.exit('Interrupted by user')
except Exception as e:
    parser.exit(type(e).__name__ + ': ' + str(e))

Hello world part II : actually recoding print

In part I we explored the pre-requisite in order to code print : having a grasp on the framebuffer.

Here, we are gonna deep inside one of the most overlooked object oriented abastraction : a file and actually print what we can of hello world in 100 lines of code.


The file handler and the file descriptor



These two abstractions are the low level and high level abstractions of the same thing : a view on something more complex which access has been encapsulated in generic methods. Actually when you code a framebuffer driver you provide function pointers that are specialized to your device and you may omit those common to the class. This is done with a double lookup on the major node, minor node number. Of those « generic » methods you have : seek, write, tell, truncate, open, read, close ...
The file handler in python also handles extra bytes (pun) of facilities : like character encoding, stats, and buffering.

Here, we work with the low level abstraction : the file which we access with fileno through its file descriptor. And thanks to this abstraction, you don't care if the underlying implementation fragments the file itself (ex: on a hard drive), you can magically always ask without caring for the gory details to read any arbitrary block or chararacters at any given position.

Two of the most used methods on files here are seek and write.

The file descriptor method write is sensitive to the positionning set by seek. Hence, we can write a line, and position ourselves one line below to write the next line in a character.

matrices as a view of a row based array



When I speak of rows and columns I evocate abstractions that are not related to the framebuffer.

The coordinates are an abstraction we build for convenience to say I want to write from this line at this column.
And since human beings bug after 2 dimensions we split the last dimnension in a vector of dimension 4 called a pixel.
get_at function illustrates our use of this trick to position the (invisible) cursor at any given position on the screen expressed for clarity in size of glyphes.
We could actually code all this exercice through a 3D view of the framebuffer. I just wouldn't be able to pack the code in less than 100 lines of code and would introduce useless abstractions.

But if you have doubt on the numerous seek I do and why I mutiply lines and columns value the way I do check the preceding link for an understanding of raw based array nth matricial dimensionnal views.

fonts, chars glyphs...



Here we are gonna take matrices defining the glyphes (what you actually see on screen) by 8x8 = 64 1D array and map them on the screen with put_char. Put char does a little bit of magic by relying on python to do the chararcter to glyph conversion through the dict lookup that expecting strings does a de factor codepoint to glyph conversion without having to pass the codepoint conversion.

The set of characters to glyphs conversion with their common property is a font.

The hidden console



The console is an abstraction that keeps track of the global states such as : what is the current line we print at. Thus, here, being lazy I use the global variables instead of a singleton named « console » or « term » to keep track of them. But first and foremost, these « abstractions » are just expectations we share in a common mental mode. Like we expect « print » to add a newline at the end of the string and begin the printing at the next line.

The to be finished example



I limited the code to 100 lines so that it's fairly readable. I let as an exercise the following points :
  • encoding the missing glyphes in the font to actually be able to write "hello world!",
  • handling the edge case of reaching the bottom of the screen,
I want to point out the true python print function is WAY MORE COMPLEX than this terse example and also handle magic conversion from memory objects to their string counterpart (like integers that are converted to their decimal representation), it handles buffering, encoding, and so much more. This is merely a toy to dive into the complexity of the mission at hand.
This example is a part of a project to write « hello world » on the framebuffer in numerous languages, bash included.

Annexe : the code




#!/usr/bin/env python3
from struct import pack
from os import SEEK_CUR, lseek as  seek, write
w,h =map(int, open("/sys/class/graphics/fb0/virtual_size").read().split(","))
so_pixel = 4
stride = w * so_pixel

encode = lambda b,g,r,a : pack("4B",b,g,r,a)

font = {
    "height":8,
    "width":8,
    'void' : [ 
        0, 0, 0, 0, 0, 0, 0, 0,
        0, 0, 1, 1, 1, 1, 0, 0,
        0, 1, 0, 0, 0, 0, 1, 0, 
        0, 0, 1, 0, 0, 1, 1, 0,
        0, 0, 0, 0, 1, 0, 0, 0,
        0, 0, 0, 0, 1, 0, 0, 0,
        0, 0, 0, 0, 0, 0, 0, 0,
        0, 0, 0, 0, 1, 0, 0, 0, 
       ],
    "l":[ 
        0, 0, 0, 0, 0, 0, 0, 0,
        0, 1, 0, 0, 0, 0, 0, 0,
        0, 1, 0, 0, 0, 0, 0, 0,
        0, 1, 0, 0, 0, 0, 0, 0,
        0, 1, 0, 0, 0, 0, 0, 0,
        0, 1, 0, 0, 0, 0, 0, 0,
        0, 1, 0, 0, 0, 0, 0, 0,
        0, 0, 1, 1, 1, 1, 0, 0,
        ],
    "o": [
        0, 0, 0, 0, 0, 0, 0, 0,
        0, 0, 1, 1, 1, 1, 0, 0,
        0, 1, 0, 0, 0, 0, 1, 0,
        0, 1, 0, 0, 0, 0, 1, 0,
        0, 1, 0, 0, 0, 0, 1, 0,
        0, 1, 0, 0, 0, 0, 1, 0,
        0, 0, 1, 0, 0, 1, 0, 0,
        0, 0, 0, 1, 1, 0, 0, 0],
    "h": [
        0, 0, 0, 0, 0, 0, 0, 0,
        0, 1, 0, 0, 0, 0, 0, 0,
        0, 1, 0, 0, 0, 0, 0, 0,
        0, 1, 0, 0, 0, 0, 0, 0,
        0, 1, 1, 1, 1, 0, 0, 0,
        0, 1, 0, 0, 1, 0, 0, 0,
        0, 1, 1, 0, 1, 0, 0, 0,
        0, 1, 0, 0, 1, 0, 0, 0],
}

def go_at(fh, x, y): 
   global stride
   seek(fh.fileno(),x*so_pixel + y *stride, 0)

def next_line(fh, reminder):
    seek(fh.fileno(), stride - reminder, SEEK_CUR)

def put_char(fh, x,y, letter):
    go_at(fh, x, y)
    black = encode(0,0,0,255)
    white = encode(255,255,255,255)
    char = font.get(letter, None) or font["void"]
    line = ""
    for col,pixel in enumerate(char):
        write(fh.fileno(), white if pixel else black)
        if (col%font["width"]==font["width"]-1):
            next_line(fh, so_pixel * font["width"])
COL=0
LIN=0

OUT = open("/dev/fb0", "bw")
FD = OUT.fileno()

def newline():
    global OUT,LIN,COL
    LIN+=1
    go_at(OUT, 0, LIN * font["height"])

def print_(line):
    global OUT, COL, LIN
    COL=0
    for c in line:
        if c == "\n":
            newline()
        put_char(OUT,COL * font["width"] , LIN * font['height'], c)
        COL+=1
    newline() 

for i in range(30):
    print_("hello lol")

Revisiting hello world : coding print from scratch part I

The « hello world » example is about standing on the shoulders of the giant and learn how to use function as tools.

Most coders will use print during their whole life without actually coding it. However, it is a fun exercise.

The framebuffer



Given you are on linux you probably have a device named /dev/fb0 if you don't, you can't do this. The framebuffer is a view of the linear graphical memory used by your video card where what you see on the screen is stored ... at the condition you are in console mode and you have the rights.

On my debian centric distribution, to give the_user permissions to write in the framebuffer I must add the_user to group_video. This can be done with sudo adduser the_user video or sudo vigr.

Then, you have to be in console mode. To switch from xorg/wayland to the console back and forth I use the Ctrl + Alt + Fn combination to switch off from X and Alt + Fn to switch back to X (it's called switching the virtual console).

Once this is done you check you have rights by doing
cat /dev/urandom > /dev/fb0
which should fill your screen with random colors and insult you stating there is no more room left on the device. SNAFU : everyhting works as intended.


The pixel



Framebuffer don't know about pixels made of Red, Green, Blue and alpha (given you have a video card that is less than 20 years old), they are just made of memory. We will have to slowly build up our understanding of what this is all about.

The in memory layout may differ according to the hardware, some are having a RGBA layout, mine, the i915 is having a BGRA layout. The following example may need to be rewritten with different hardware if the output is not consistent with your assumption.

Determining the memory layout and coordinates



We will do a test and validate code session : first we make assumption on where the colours are by writting 3 squares of Red, Blue and Green on the screen, then, we will snapshot the screen.

$ cat fb.py
#!/usr/bin/env python3
from struct import pack
w,h =map(int, open("/sys/class/graphics/fb0/virtual_size").read().split(","))
midx = w//2
midy = h//2

encode = lambda b,g,r,a : pack("4B",b,g,r,a)

with open("/dev/fb0", "wb") as f:
    for y in range(0,h):
        for x in range(0,w):
            f.write(encode(
                not x%100 and 0xA0 or x<midx and 0xFF or 0, #blue
                y<midy and 0xFF or 0,                       #green
                x>midx and y>midy and 0xFF or 0,            #red
                0,
            ))
The only « trick » is the use of pack to encode the four colour bytes in a byte array that is written to the framebuffer filehandler. If the code works correctly we should validate the following assumptions:
  • coordinates are such as 0 is top left of the screen where green and blue should superpose
  • my 1920x1080 screen should have 19 weired stripes (hence validating the geometry)
  • each colours should be in its square, red bottom right, green top right, blue bottom left.
  • RAM as a char device is accessing a low level file
And if the world is consistent we can read from the framebuffer and snapshot it in the most trivial picture encoding witch is Portable PixMap format. A portable Pixmap is made of
  • a magic number P3 followed by
  • width
  • height
  • the maximum colour value (here 255)
  • the 3 colour bytes Red, Blue, Green without Alpha value per pixel

The code for this is straigh forward :
$ cat snap.py
#!/usr/bin/env python
from struct import unpack
w,h = map( int,open("/sys/class/graphics/fb0/virtual_size").read().split(","))

# returns b g r a
decode = lambda pixel : unpack("4B", pixel)

def pr(b,g,r,a):
    print("%d %d %d" % (r,g,b))

print(f"""P3
{w} {h}
255
""")

with open("/dev/fb0", "rb") as fin:
    while pixel := fin.read(4):
        pr(*decode(pixel))
Here the only trick is we use the symetrical function of pack, unpack to decode the pixel in the four colour bytes.

wrapping up part one



Asumming you can install fim the framebuffer image wiewer and you installed imagemagick : you can now do
./fb.py && snap.py > this.ppm  && convert this.ppm this.jpg && fim this.jpg
Doing so, you should have the same picture showing twice without an error like this :
As an exercise, you can vary the fb.py to make funny output, or code a PPM viewer that print back your ppm to the screen.

The AWTFPL licence resilient against AI training

Simple is Beautiful,
Less words
makes the world
plentyful
The WTFPL what the fuck public licence is a public domain license that holds one clause « do whatever the fuck you want with this code ».
As a long time fanboy of BSD ecosystem for its terse license, this license struck me as beautiful. Except, that as a stoïcian that aim to bear more sense with fewer words I think one clause misses that actually prevents code from being used for AI training : that's why I created the do Almost What The Fuck you want Public License except claim you wrote the code. Since AI do not quote their sources they actually infringe this clause.

I actually have quite a few half baked shitty code that I want AI to learn from to see the world burn by stealing the code. So, I think most of my code will stay under vanilla WTFPL. The license that says it all with less words. I also could care less about this code but use this license to support the authors, coders, artists which creative works is being claimed as theirs by all the Large Language Models out there.

I couldn't care less
how the world evolves
but both licences solves
the choice of license's stress

3D ploter in python-tk with matplotlib.

Wishing to prove my assertion wrong on python-tk that piping python directly into tk/tcl interpreter is simple I tried to contradict myself by making a full GUI in matplotlib. Because, if you are not aware : matplotlib supports multi-target (Wx, Qt, gtk, tk, html) multi-platform widgets (Button, checkbox, text entry and so much more).

The challenge seemed pretty easy, to assemble an easy demo with simple example : Thus, the added value resided in letting people fill in the min/max/step information for the relevant dimensions.

Without the colorbar I would have just been slightly annoyed by the slowness of the reaction of matplotlib as a GUI, but the colorbar posed a new challenge because it would either stack for each drawing or make plt.clf/ax.cla erase too much (see this great resource on when to use cla/clf in matplotlib).

So ... I tried python-tk with matplotlib knowing all too well that you can embed natively matplotlib in tkinter interface.

And since it was working I kept it.

Here is a screenshot of the interface :
WARNING this code should not be let in inventive hands (such as bored teenagers) because there is an evil eval; it requires to be in care of consenting adults.

Some highlights of the code :
  • bidir python-tk requires setting the Popen PIPEs to non blocking and using select.select on the output
  • matplotlib is unusable in non blocking mode : once matplotlib has the focus you need to destroy it to plot another function
  • from np import * is evil, but it let you have access to all array oriented math function (sin, cos, exp, ...)
This said, we have a pretty compact code of 128 lines of code that is pretty more reactive than using matplolib's widget for a 3D ploter.

Strong suspicions that the game of life might not be a good base for a PRNG : Takens killed my hopes

Previously, I was having high hopes of doing open science on PRNG (Pseudo Random Number Generator) based on the game of life on an hexagonal board. However, today, after a little bit of investigations I have bad and good news on the topic.

The bad news



Takens series might have killed my hopes, but it did it before I invested much time



Takens' series are a useful tool for evaluating the quality of randomness. When choosing this method I did know what to reach as a goal, I had a compass, and this compass was filling the space phase uniformely without patterns : f(x) => f(x+1) should not show signs of preference for space. Seeing my tweaked game of life with enhanced self injected entropy converge to a Sierpinsky gasket. Maybe it's not totally bad given I don't know it's lyapounov exponent (aka fractal dimension), but I refuse to let anyone use a method that displays strongly biased entropy in the space phase.

Good randomness for scientific usage or cryptographic one should be evenly distributed. Here is a pertubed game of life, with self injection of « chaos » into itself displaying a Sierpinky Gasket as a strange attractor, followed by a test sample of consecutive deterministric pseudo random numbers which source is the random python's PRNG.



Normally, I can conclude here with : « don't let friends use biased randomness and don't roll your own crypto ».

But I won't : first and foremost I want to higlight that keeping a clear view of what you do is important and knowing fast when you fail is as much postive experience as succeeding slowly.



The good news



The importance of visualisation



Since the beginning of the experiment as a python-tk proof of concept I added a visual interface. Here is the interface :


I added the possibility not only to see in real time the Takens series, but also the non overlapping evolutions of the pertubed game of life with the original one (level of gray coding the discrepancies).

I can testify I detected A LOT OF bugs through this. Manipulating code is easy, and making mistakes even easier. By keeping a view of the initial problem I kept a strong way of checking everything was making sense.

Data visualisation is the second compass with Takens series that helped me save a lot of time to not conclude falsely I made a unique discovery.

python-tk is bearing the load



Every cycle, I communicate back and forth between tk and python, python sending up to 2500 tcl commands in less than 1 second to update de canvas. Just for this it proved useful in giving me trust in the concept.

I was often asked why not tkinter or pure tcl ?

Well, python has data structures such as sets that supports operations that maps nicely with the concepts used in bit arithmetic (xor, and, or ...) that tcl does not have.
In python I can also additionnaly call matplotlib if I want to do 3D scatters or use the very performant numpy arrays. On the other hand, I have experience in Tk/Tcl and have a better grasp of how I deal with GUI in Tk rather than in tkinter. It's a question of comfort not a question of « one best way ». And it was fun to code this way without to much abstractions in the way.

Is the experiment over ?



Yes, it is time consuming.

However, I strongly suspect and would like to check in the future, that the strange attractor in the shape of a Sierpinsky Gasket is due to the symetry in the first neighbours in an hexagonal shape. I would -if I was paid for it explore other kinds of neighbouroud and rules.

For instance we could add a second ring of neighbours and maybe use them as anti-ferromagnetic rules. Maybe I would use random neighbourhood to check if the Sierpinsky gasket disappears .... and so long.

The nice part for this experimentations would be that I have the tooling that is ready.

Annexe the code



The code is available here as a gist on github

Can game of life be used as a pseudo random generator ? Maybe so.

After having fun doing bidirectionnal python <=> tk programming I thought that an hexagonal game of life experiment would be a nice way to saturate the PIPE from python to tcl to assert how much load the concept could take.

So I made a game of life with a twist inspired from this article about using strange attractors to qualify randomness.

Conway's game of life is a pretty simple example of cellular automata. The tradition version is on a square grid, here we not only change for an hexagonal grid, we also let users change the rules and we add a feature : we plot in real time a graph with u(n-1) on the x-axis and u(n) on the y-axis every turn and I'll explain why later.

But first let's look at a classical run with the normal rules : it's boring because it converges pretty fast to oscillators.
. But, by tweaking rules, you can have all kind of boring oscillators, patterns that repeat themselves with often a period of only two.
And also configurations for which we cannot tell it they are periodical :

The tool

But actually, let's have a look at the complete interface for playing with hexagonal game of life :
As you can see, you can set the rules for both dead cells that gives birth to new one, or lively cells that spreads. As a result, it becomes interesting to explore when the game of life does not converge.

It has an assymetry due to the coding techniques of python to pure tk for which python output is priorized over tk input, hence the feedback from the pure tk interfaces that may take several full long seconds.

Takens serie usage



Game of life being a deterministic game given a configuration Cn: if we already see it we can assert with certainty what Cn+1 will be. Hence if you plot Cn, Cn1 on respectively the x and y axis you can ascertain that if new points appear in the space phase you have not yet entered a period. It is a pretty darn KISS (Keep it Simple Stupid) method for evaluating not only if we hit a repetition but also biases in the space phase.
As refered at the beginning of this post in lcamtuf post on exploring randomness : a good PRNG (pseudo random number generator) should not only have a very long period it should cover the space phase equiprobably. And you know what tool is very good at seeing non uniformity? Your eyes coupled with your brain.

Spoiler alert : if I show you this : you are gonna tell me : hum ... Why does your space phase tends to looks like sierpinsky triangles?



My greek teacher used to say the pĥilosophs are not the one who gives smart answers but those who asks clever questions. And here you are touching an interesting point. Is the sierpinsky pattern inherent to randomness ? No. Is it inherent to topology ? Maybe. Is hit inherent to the hexagonal variation on the game of life ? Surely. Is it due to the Periodic Condiditions at the Borders ? No (because patterns who don't evolve from configurations touching the borders also have it).

In one simple eye analysis we have an interesting fact : suites of numbers generated by using game of life have a signature in the space phase that reflects the topology (here hexagonal).
Even though it my not be obvious at first sight :


Qualitative analysis of the hexagonal game of life for use as a PRNG



After a little bit of trial and errors we can notice that when we set for either dead or living cells to 3, 4, 5 we have often more long lasting sequences. As you can see there for alive = 4,5,6, dead = 2,3,6, it is also very sensitive to initial conditions making it tougher to analyse by differential analysis.



Even though it seems to do a pretty good job at generating a long list of non repeating numbers, however, as is it as the following problem :

evaluating entropy seeding



Given intial entropy we may enter the following problems :
  • Initial configuration may converge fast to a null of full one configuration not evolving anymore
  • Can we quantify how much pseudo entropy we generate ?
  • How much round of non repeating numbers do we have given a known size of grid and given suitable rules ?
  • Which rules generates the maximum chaos ? Are there equivalent sets of rules we can switch too to keep the same entropy level ?
  • Real life randomness accepts to repeat preceding seen values, how much the non repetition (sign of a non converging serie) is a weakness ?


Thinking of designing a PRNG based on hexagonal game of life



Real life cryptography have good practices like regenerating an initialization vector every round to avoid leaking information since ... enigma. Here, we could actually use the entropy we generate to alter ever the configuration or the rules in a deterministic way that would actually make the Takens serie cover the full space uniformely showing no sign of priviledging a siperinsky triangle kind of shape.

Actually, Takens series are an helpful tool in determining the quality of randomness so we already have one tool available.

Also, game of life is a fairly easy to code and understand algorihtm, thus, by lowering the entry barrier to coding the chore of the engine we have more eyeballs that can spot bugs and contribute to a better solution.

As they are easy to code, they are also easy to synthethize as an ASIC or FGPA, bringing an easy gain in acceleration by going full hardware with algorithm that being intrinsicly massively parallel will be tougher to brake with thread based computing, and mathematically it's a non linear system for which theory provides fewer tools of analysis.

Talking about FGPA we also can study not only the alteration of configuration but explore the alteration of topology breaking the hexagonal symmetry that may be the root cause of the sirpinksy triangle in the space phase.

I think game of life as a base for a PRNG is a funny topic of hobby research that could empower a lot of amateurs in this field building not only better algorithms but also dispelling myth and obscurantism around this field of computer science.

PS : After some investigations I have doubts that it is possible, but it makes a fun topic

Writing an interactive tcl/tk interpreter proxy to wish in python

Maybe, you want to experiment small stuffs in wish (the tcl/tk) interpreter because of a post claiming that direct python tcl/tk is simpler in some simple cases than tkinter.

As a convinced tkinter/FreeSimpleGUI user, I see this as an extreme claim that requires solid evidences.

When all is said and done, wish interpreter is not interactive, and for testing simple stuff it can get annoying very fast. Thus, it would be nice to add readline to the interface.

So here is a less than 100 line of code exercice of soing exactly so while having fun with : readline and multiprocessing (I would have taken multithreading if threads were easy to terminate).

Readline I quote
The readline module defines a number of functions to facilitate completion and reading/writing of history files from the Python interpreter.
Basically, it adds arrow navigation in history, back search with Ctrl+R, Ctrl+K for cuting on the right, Ctrl+Y for yanking ... all the facilities of interaction you have in bash or ipython for instance.

We are gonna use multiprocessing because tcl/tl is event oriented, hence, asynchronuous hence, we may have string coming from the tcl stdout while we do nothing and we would like to print them.

We also introduce like in ipython some magic prefixed with # (comment in tcl) like #? for the help. A session should look like this :
# pack [ button .c -text that -command { puts "hello" } ]
# 
tcl output> hello # here we pressed the button "that"
tcl output> hello # here we pressed the button "that"


# set name 32
# puts $name

tcl output> 32

# #?

#l print current recorded session
#? print current help
#! calls python code like
  #!save(name="temp") which saves the current session in current dir in "temp" file
bye exit quit quit the current session

# #l
pack [ button .c -text that -command { puts "hello" } ]
set name 32
puts $name

# #!save("my_test.tcl")
# quit
The code in itself is fairly easy to read the only catch is that wish accepts multiline input. I can't because I don't know how to parse tcl. As a result I « eval in tcl » every line to know if there is an error and ask politely tcl to do the job of signaling the error with a « catch/error » (the equivalent of python try + raise an exception).
#!/usr/bin/env python3
# -*- coding: utf8 -*-

from subprocess import Popen, PIPE, STDOUT
from multiprocessing import Process
import sys, os
import atexit
import os
import readline
from select import select
from time import sleep

### interactive session with history with readline
histfile = os.path.join(os.path.expanduser("~"), ".wish_history")
try:
    readline.read_history_file(histfile)
    # default history len is -1 (infinite), which may grow unruly
    readline.set_history_length(-1)
except FileNotFoundError:
    pass

### saving history at the end of the session
atexit.register(readline.write_history_file, histfile)

### opening wish
wish = Popen(['wish'], 
        stdin=PIPE,
        stdout=PIPE,
        stderr=PIPE,
        bufsize=-1,
        )

os.set_blocking(wish.stdout.fileno(), False)
os.set_blocking(wish.stderr.fileno(), False)
os.set_blocking(wish.stdin.fileno(), False)

def puts(s):
    out = f"""set code [ catch {{ {s} }} p ]
if {{$code}} {{ error $p }}
"""
    select([], [wish.stdin], [])
    wish.stdin.write(out.encode())


def gets():
    while True:
        wish.stdout.flush()
        tin = wish.stdout.read()
        if tin:
            print("\ntcl output> " + tin.decode())
        sleep(.1)

def save(fn="temp"):
    with open(fn,"wt") as f:
        f.write(session)

session=s=""
def load(fn="temp"):
    global session
    with open(fn, "rt") as f:
        while l:= f.readline():
            session+=l + "\n"
            puts(l)


# async io in tcl requires a background process to read the input
t =Process(target=gets, arwish=())
t.start()

while True:
    s = input("# ")
    if s in { "bye", "quit", "exit" }:
        t.terminate()
        wish.stdin.write("destroy .".encode())
        break
    elif s == "#l":
        print(session)
    elif s == "#?":
        print("""
#l print current recorded session
#? print current help
#! calls python code like
  #!save(name="temp") which saves the current session in current dir in "temp" file
  #!load(name="temp") which load the session stored in current dir in "temp" file
bye exit quit quit the current session
""" )
        continue
    elif s.startswith("#!"):
        print(eval(s[2:]))
        continue
    else:
        puts(s)
        if err:=wish.stderr.readline():
            sys.stderr.write(err.decode())
        else:
            if s and not s.startswith("#"):
                session += s + "\n"

This code is available on pypi as iwish (interactive wish) and the git link is in the README.

Theming a python to tcl/tk (hence without tkinter) application

As a foreword, tkinter users should have a look at ttkbootstrap which is unique to tkinter and is an awesome flat theme for tk.

But, I will instead admit that tcl/tk default is ugly by default ...


Except for windows ...


I am not a designer, but I am not blind either. I noticed that flat (aka without relief) designs age fairly well compared to the others.

So, we want themes, and why ? Because they can help provide a consistent cross platform look and feel. Here is an exemple on windows linux, freeBSD with the breeze theme :
The most part of the code has been publish in the post about making python and tcl/tk talk bidirectionnaly without tkinter. I am gonna focus on the theme part of the code which is mainly. Python users may want to look this posts since the available themes for python that can be installed with
pip install ttkthemes
have a good documentation on how to use ttkthemes and how they look but does not include all the available themes and where to get them.

The same themes as pypi ttkthemes can be obtained on debian with
apt install tcl-ttkthemes
and they are stored in /usr/share/tcltk/ttkthemes/themes/. From there we can use tcl that has its good sides to build a combobox for choosing a theme :
set themes_dir [ glob -directory  /usr/share/tcltk/ttkthemes/themes/ -type d * ]
set themes [ list ]
foreach dir $themes_dir {
    lappend themes [ file tail $dir ]
}
ttk::combobox .cb -textvariable theme -values [ lsort $themes ] -width 12
.cb state readonly
bind .cb <<ComboboxSelected>> { 
    set theme [%W get]
    catch {
    source /usr/share/tcltk/ttkthemes/themes/$theme/$theme.tcl }} pass
    ttk::style theme use $theme
}
catch { code } result
is the equivalent of try:/except, except you capture the exception in result and catch return 1 on error. the bind part call a proc to change the theme. With this and a few more widgets and bindings we can appreciate that definitively the breeze theme that is not provided by default is the best one ^_^

The essential



A ttk theme is fairly easy to use in tcl:
  1. spot its tcl file. ex aquativo.tcl
  2. source it ONCE
     source /usr/share/tcltk/ttkthemes/themes/aquativo/aquativo.tcl 
  3. use it :
    ttk::style theme use aquativo
  4. use only the ttk:: widgets in your tcl code
And that's all.

Annexe: full code



#!/usr/bin/env python
from subprocess import Popen, PIPE
from time import sleep, time, localtime
import os

# let's talk to tk/tcl directly through p.stdin
p = Popen(['wish'], stdin=PIPE, stdout=PIPE)

os.set_blocking(p.stdout.fileno(), False)
os.set_blocking(p.stdin.fileno(), False)

def puts(s):
    for l in s.split("\n"):
        p.stdin.flush()
        p.stdin.write((l + "\n").encode())
        p.stdin.flush()

def gets():
    ret=p.stdout.read()
    p.stdout.flush()
    return ret

WIDTH=HEIGHT=500

puts(f"""

canvas .c -width {WIDTH} -height {HEIGHT} -bg white
pack .c
.c configure -background white
ttk::frame .h
pack .h -fill both -expand true -padx 0 -pady 0
ttk::frame .g 
pack .g -in .h -fill x -expand true -padx 0

ttk::button  .ba -command {{  puts ch-=1 }} -text <<
ttk::button .bb -command {{  puts cm-=1 }} -text <
ttk::button .bc -command {{  puts ch+=1 }} -text >>
ttk::button .bd -command {{  puts cm+=1 }} -text >
pack .ba .bb  -side left -anchor w -in .g
pack .bc .bd  -side right -anchor e -in .g

set themes_dir [ glob -directory  /usr/share/tcltk/ttkthemes/themes/ -type d * ]
set themes [ list ]
foreach dir $themes_dir {{
    lappend themes [ file tail $dir ]
}}
ttk::combobox .cb -textvariable theme -values [ lsort $themes ] -width 12
.cb state readonly
bind .cb <<ComboboxSelected>> {{ 
    set theme [%W get]
    catch {{
    source /usr/share/tcltk/ttkthemes/themes/$theme/$theme.tcl }} pass
    ttk::style theme use $theme
}}
pack .cb  -in .g
ttk::frame .f 
pack .f -in .h -expand 1 -anchor s -padx 0 
set h 0
set cm 0
set s 0

ttk::label .l -text "sample of label"
ttk::entry .i -text "sample of input" -textvariable theme
pack .l .i -anchor w -in .f -padx 5 -pady 5
ttk::scale .s  -from 0 -to 24 -variable h 
ttk::button .bt -text Quit -command "destroy ."
ttk::spinbox .sb -from -60 -to 60 -textvariable cm -width 5 -command {{ puts "cm=$cm" }}
ttk::progressbar .pb -maximum 60 -variable s
pack .s .sb .pb  -in .f -side left -anchor se -padx 5 -pady 5 
pack .bt -in .h -anchor s -pady 5 -padx 5
""")

# Constant are CAPitalized in python by convention
from cmath import  pi as PI, e as E
ORIG=complex(WIDTH/2, HEIGHT/2)

# correcting python notations j => I
I = complex("j")
rad_per_sec = 2.0 * PI /60.0
rad_per_min = rad_per_sec / 60
rad_per_hour= rad_per_min / 12

origin_vector_hand = WIDTH/2 *  I

size_of_sec_hand = .9
size_of_min_hand = .8
size_of_hour_hand= .65

rot_sec = lambda sec  : -E ** (I * sec  * rad_per_sec)
rot_min = lambda min  : -E ** (I * min  * rad_per_min)
rot_hour= lambda hour : -E ** (I * hour * rad_per_hour)

to_real = lambda c1,c2 : "%f %f %f %f" % (c1.real,c1.imag,c2.real, c2.imag)
for n in range(60):
    direction= origin_vector_hand * rot_sec(n)
    start=.9 if n%5 else .85
    puts(f".c create line {to_real(ORIG+start*direction,ORIG+.95*direction)}")
    sleep(.01)

diff_offset_in_sec = (time() % (24*3600)) - \
    localtime()[3]*3600 -localtime()[4] * 60.0 \
    - localtime()[5]
ch=cm=0
n=0
while True:
    n+=1
    # eventually parsing tcl output
    t = time()
    s= t%60
    m = m_in_sec = t%(60 * 60) + cm * 60
    h = h_in_sec = (t- diff_offset_in_sec)%(24*60*60) + ch * 3600 + cm * 60 

    if back := gets():
        back = back.decode()
        exec(back)

    puts(".c delete second")
    puts(".c delete minute")
    puts(".c delete hour")
    if n%10==0:
        puts(f"set s {int(s%60)}")
        puts(f"set cm {cm}")
        puts(f"set h {h/3600}")
    n%=100
    c0=ORIG+ -.1 * origin_vector_hand * rot_sec(s)
    c1=ORIG+ size_of_sec_hand * origin_vector_hand * rot_sec(s)
    puts( f".c create line {to_real(c0,c1)} -tag second -fill blue -smooth true")
    c1=ORIG+size_of_min_hand * origin_vector_hand * rot_min(m)
    puts(f".c create line {to_real(ORIG, c1)} -tag minute -fill green -smooth true")
    c1=ORIG+size_of_hour_hand * origin_vector_hand * rot_hour(h)
    puts(f".c create line {to_real(ORIG,c1)} -tag hour -fill red -smooth true")
    puts("flush stdout")
    sleep(.15)

Bidirectionnal python/tk by talking to tk interpreter back and forth

Last time I exposed an old way learned in physical labs to do C or python/tk like in the old days: by summoning a tcl/tk interpreter and piping commands to it.

But what fun is it?

It's funnier if the tcl/tk interperpreter talks back to python :D as an hommage to the 25 years awaited TK9 versions that solves a lot of unicode trouble.

Beforehand, to make sense to the code a little warning is required : this code targets only POSIX environment and loses portability because I chose to use a way that is not the « one best way » for enabling bidirectionnal talks. By using os.set_blocking(p.stdout.fileno(), False) we can have portable non blocking IO, which means this trick has been tested on linux, freeBSD and windows successfully. It's pretty much advised when using non blocking IO to use select.select to check if file descriptors are ready to use.

First and foremost, the Popen now use p.stdout=PIPE enabling the channel on which tcl will talk. As a joke puts/gets are named from tcl/tk functions and are used in python to push/get strings from tcl.

Instead of using multithreading having one thread listen to the output and putting the events in a local queue that the main thread will consume I chose the funniest technique of setting tcl/tk output non blocking which does not work on windows. This is the fnctl part of the code.

Then, I chose not to parse the output of tcl/tk but exec it, making tcl/tk actually push python commands back to python. That's the exec part of the code.

For this I needed an excuse : so I added buttons to change minutes/hours back and forth.

That's the moment we all are gonna agree that tcl/tk that tcl/tk biggest sin is its default look. Don't worry, next part is about using themes.

Compared to the first post, changes are minimal :D This is how it should look :
And here is the code, largely still below 100 sloc (by 3 lines).
#!/usr/bin/env python
from subprocess import Popen, PIPE
from time import sleep, time, localtime
import select
# import fcntl
import os

# let's talk to tk/tcl directly through p.stdin
p = Popen(['wish'], stdin=PIPE, stdout=PIPE)

# best non portable answer on stackoverflow
#fd = p.stdout.fileno()
#flag = fcntl.fcntl(fd, fcntl.F_GETFL)
#fcntl.fcntl(fd, fcntl.F_SETFL, flag | os.O_NONBLOCK)
# ^-- this 3 lines can be replaced with this one liner --v
# portable non blocking IO
os.set_blocking(p.stdout.fileno(), False)

def puts(s):
    for l in s.split("\n"):
        select.select([], [p.stdin], [])
        p.stdin.write((l + "\n").encode())
        p.stdin.flush()

def gets():
    ret=p.stdout.read()
    p.stdout.flush()
    return ret

WIDTH=HEIGHT=400

puts(f"""
canvas .c -width {WIDTH} -height {HEIGHT} -bg white
pack .c
. configure -background white

ttk::button  .ba -command {{  puts ch-=1 }} -text <<
pack .ba -side left   -anchor w
ttk::button .bb -command {{  puts cm-=1 }} -text  <
pack .bb -side left -anchor w
ttk::button .bc -command {{  puts ch+=1 }} -text >> 
pack .bc  -side right -anchor e
ttk::button .bd -command {{  puts cm+=1 }} -text > 
pack .bd  -side right -anchor e
""")

# Constant are CAPitalized in python by convention
from cmath import  pi as PI, e as E
ORIG=complex(WIDTH/2, HEIGHT/2)

# correcting python notations j => I  
I = complex("j")
rad_per_sec = 2.0 * PI /60.0
rad_per_min = rad_per_sec / 60
rad_per_hour = rad_per_min / 12

origin_vector_hand = WIDTH/2 *  I

size_of_sec_hand = .9
size_of_min_hand = .8
size_of_hour_hand = .65

rot_sec = lambda sec : -E ** (I * sec * rad_per_sec )
rot_min = lambda min : -E ** (I *  min * rad_per_min )
rot_hour = lambda hour : -E ** (I * hour * rad_per_hour )

to_real = lambda c1,c2 : "%f %f %f %f" % (c1.real,c1.imag,c2.real, c2.imag)
for n in range(60):
    direction= origin_vector_hand * rot_sec(n)
    start=.9 if n%5 else .85
    puts(f".c create line {to_real(ORIG+start*direction,ORIG+.95*direction)}")
    sleep(.01)

diff_offset_in_sec = (time() % (24*3600)) - \
    localtime()[3]*3600 -localtime()[4] * 60.0 \
    - localtime()[5] 
ch=cm=0
while True:
    # eventually parsing tcl output 
    back = gets()
    # trying is more concise than checking
    try:
        back = back.decode()
        exec(back)
    except Exception as e:
        pass

    t = time()
    s= t%60
    m = m_in_sec = t%(60 * 60) + cm * 60
    h = h_in_sec = (t- diff_offset_in_sec)%(24*60*60) + ch * 3600 + cm * 60
    puts(".c delete second")
    puts(".c delete minute")
    puts(".c delete hour")
    c0=ORIG+ -.1 * origin_vector_hand * rot_sec(s)
    c1=ORIG+ size_of_sec_hand * origin_vector_hand * rot_sec(s)
    puts( f".c create line {to_real(c0,c1)} -tag second -fill blue -smooth true")
    c1=ORIG+size_of_min_hand * origin_vector_hand * rot_min(m)
    puts(f".c create line {to_real(ORIG, c1)} -tag minute -fill green -smooth true")
    c1=ORIG+size_of_hour_hand * origin_vector_hand * rot_hour(h)
    puts(f".c create line {to_real(ORIG,c1)} -tag hour -fill red -smooth true")
    puts("flush stdout")
    sleep(.1)


Some history about this code.



I have been mentored in a physical lab where we where doing the pipe, fork, dup2 dance to tcl/tk from C to give a nice output to our simulations so we could control our instuition was right and could extract pictures for the publications. This is a trick that is almost as new as my arteries.
My mentor used to say : we are not coders, we need stuff to work fast and neither get drowned in computer complexity or endless quest for « the one best way » nor being drowned in bugs, we aim for the Keep It Simple Stupid Ways.

Hence, this is a Keep It Simple Stupid approach that I revived for the sake of seeing if it was still robust after 35 years without using it.

Well, if it's robust and it's working: it ain't stupid even if it isn't the « one best idiomatic way ». :P

Simpler than PySimpleGUI and python tkinter: talking directly to tcl/tk

Well, the PySimpleGUI rug pulling of its licence reminded me how much dependencies are not a good thing.

Even though FreeSimpleGUI is a good approach to simpler tk/tcl binding in python : we can do better, especially if your linux distro split the python package and you don't have access to tkinter. I am watching you debian, splitting ALL packages and breaking them including ... tcl from tk (what a crime).

Under debian this stunt requires you to install tk :
apt install tk8.6


How hard is it when tcl/tk is installed to do GUI programming in tk without tkinter?

Well, it's fairly easy, first and foremost coders are coders, they code in whatever language. If you do code in one language you can't do docker, simple sysadmin tasks (shell), compile C extensions (make syntax) or web applications (HTML + javascript). Hence, learning more than one language is part of doing python applications.

How hard is coding in tcl/tk natively?

Fairly easy: its difficulty is a little above lua, and way below perl thanks to the absence of references.

What value tcl have ?

It's still used in domain specific field such as VLSI (Very Large Scale Integration of electronic component).

So here is the plan : we are gonna do an application that do the math in python which is perfect for expressing complex math in more readable way than tcl and push all the GUI to the tk interpreter (albeit wish).

We are gonna make a simple wall clock ... and all tcl commands are injected to tcl through the puts function.
#!/usr/bin/env python
from subprocess import Popen, PIPE
from time import sleep, time, localtime

# let's talk to tk/tcl directly through p.stdin
p = Popen(['wish'], stdin=PIPE)

def puts(s):
    for l in s.split("\n"):
        p.stdin.write((l + "\n").encode())
        p.stdin.flush()

WIDTH=HEIGHT=400

puts(f"""
canvas .c -width {WIDTH} -height {HEIGHT} -bg white
pack .c
. configure -background "white"
""")

# Constant are CAPitalized in python by convention
from cmath import  pi as PI, e as E
ORIG=complex(WIDTH/2, HEIGHT/2)

# correcting python notations j => I  
I = complex("j")
rad_per_sec = 2.0 * PI /60.0
rad_per_min = rad_per_sec / 60
rad_per_hour = rad_per_min / 12

origin_vector_hand = WIDTH/2 *  I

size_of_sec_hand = .9
size_of_min_hand = .8
size_of_hour_hand = .65

rot_sec = lambda sec : -E ** (I * sec * rad_per_sec )
rot_min = lambda min : -E ** (I *  min * rad_per_min )
rot_hour = lambda hour : -E ** (I * hour * rad_per_hour )

to_real = lambda c1,c2 : "%f %f %f %f" % (c1.real,c1.imag,c2.real, c2.imag)
for n in range(60):
    direction= origin_vector_hand * rot_sec(n)
    start=.9 if n%5 else .85
    puts(f".c create line {to_real(ORIG+start*direction,ORIG+.95*direction)}")
    sleep(.1)

diff_offset_in_sec = (time() % (24*3600)) - \
    localtime()[3]*3600 -localtime()[4] * 60.0 \
    - localtime()[5] 

while True:
    t = time()
    s= t%60
    m = m_in_sec = t%(60 * 60)
    h = h_in_sec = (t- diff_offset_in_sec)%(24*60*60)
    puts(".c delete second")
    puts(".c delete minute")
    puts(".c delete hour")
    c0=ORIG+ -.1 * origin_vector_hand * rot_sec(s)
    c1=ORIG+ size_of_sec_hand * origin_vector_hand * rot_sec(s)
    puts( f".c create line {to_real(c0,c1)} -tag second -fill blue -smooth true")
    c1=ORIG+size_of_min_hand * origin_vector_hand * rot_min(m)
    puts(f".c create line {to_real(ORIG, c1)} -tag minute -fill green -smooth true")
    c1=ORIG+size_of_hour_hand * origin_vector_hand * rot_hour(h)
    puts(f".c create line {to_real(ORIG,c1)} -tag hour -fill red -smooth true")
    sleep(.1)

Next time as a bonus, I'm gonna do something tkinter cannot do: bidirectional communications (REP/REQ pattern).