Building a Better chrono-sociogram : a radar picture of social interactions

Humans are like a flock of birds of the same feathers that have a natural tendancy to change their feathers a lot.

-- Coco Chanel while french kissing a gestapo officer ~1943 in Paris

What is wrong with sociograms ? They don't catch the way people change their loyalty. I think an election is interesting especially when it was the most massive turn cloak event ever seen in french politic.

So to solve the lack of tooling for this there is a tool : the chrono sociogram aka : social movie.

First was the persistent map

By feature, dot will correctly guess galaxies of connection that have existed over a long period of time. Hence we begin with a persistent map that is similar to all links set to 1 over a long period of time.

Why, because the same way radar maps are useless if cluttered with too much useless informations, you will not care about WHOM talked to WHOM but much more where storms happens, when.

Unreadable ? It is here in SVG.

Well since my corei3 (sorry for being poor and not being able to afford a M1 mac and a GPU for mining bitcoin) is taking 10 hours to generate 243 dot files), imagine how much time it takes to generate the following : a movie out of keeping the persistant image and only changing the color/size of the edge by templating THIS exact picture :D

What is it useful for ?

The first time I worked on this it was in 1997 in ENS as an intern knowing linux in a complex system lab, having a grant on influencing citizen for the « greater good of Political Agricole Commune ».

UE had failed miserably at convincing peasants (red necks) that there interest was in accepting UE money and being enslaved to debt for their life being. With this grant, the network of peasanry and exchange used, sociogram was built and « highly connected nodes » identified. Nowadays we call this « influencers ».

UE just flipped the opinion of a few selected one (ignoring the background of why THEY were selected) and it cascaded in convincing farmers.

This video illustrates like storms, both exchanges in social networks and when/how people turn coats. It is ACTUALLY the very heart of social media.

Hence when I claimed some originality on this work, I point blank lied. However, try to find people sharing their tooling, methodology and you will discover that your public taxes found public research lab which work you can't read and which tools you cant' use on the topic.

Graph problems are NP, exploring graphs require a lot OF CPU, how do meta/FB handle the network analysis I just done for a fraction of the CPU ?

Well, they don't observe the network, they also model it by rewarding people who ARE ALREADY INFLUENCERS in real life thanks to « side channel ». Ex : for publishing videos at the actual quality standards in video and sound processing you need WAY MORE MONEY and better equipment than for writing a blog post that can be done with a 12yo computers. There is a bias of selection of just using the main stream tools.

How can we avoid being influenced ?

Don't : learning of foreign cultures is nice. I love manwhas, mangas and folklore and foreign languages as much as any influencers do. At the opposite of them, though, I give no interest in fame.

What traps you in the web of influence is your the reinforcment of your own bias. Trsuting hyped talkers is a bias..

Have you tried talking to friend of yours about topic on which you disagree and state that you want to explore influence ? Life if polemic. You are better prepared to it by training like a boxer : every day in a pleasant non aggressive mindset like when going to the boxing training. (Yes, I love boxing).

Ivy league education prepare to this thanks to « concours d'éloquence », it's also if you notice the core of how slang culture treat the others.

Now that I finished a project I wanted to do for long, I think my sociogram time is other, and I may do a last topic on the making (and thus introduce my bash sort of makefile and how with perl oneliner I templatized a dot file to create a usable template for python because this trick is serverly disgusting (I mean FUN).

Real life stuff

All the spoke persons even though they were not the ones spied on of the government did have a very active involvments while being paid by tax payers money. It may be legal, it does not look moral when the same one advocates for better spending of the tax money. When you are receiving public money for one reason to support your expenditures to either be a public servant or a representant of the people, you don't work for private interests on the side. But well, we neither call french state l'Assiette au beurre (the plate of butter) for nothing, nor Paris Paname without a reason : it's because of this lingering kind of corruptions scandals that don't kind of motivate you to vote.

Next episode

The poor man's makefile in bash.

Annexe

	#!/usr/bin/env python3
	import os
	import psycopg2
	from datetime import date, datetime, timedelta
	from archery import mdict
	def int_env_default(var, default):
	return int(os.getenv(var) or default)
	def float_env_default(var, default):
	return float(os.getenv(var) or default)
	MIN_MAIL = int_env_default("MIN_MAIL",6, )
	MAX_MAIL = int_env_default("MAX_MAIL",100)
	WL_MIN = int_env_default("WL_MIN", 3)
	CUT_SIZE = int_env_default("CUT_SIZE", 20)
	DATE = os.getenv("DATE") or "2016-01-01"
	END_DATE = os.getenv("END_DATE") or "2017-05-01"
	BY_DAYS = int_env_default("BY_DAYS",4) # 13x28 = 364 ~ 365.5
	EDGE_SCALE= float_env_default("EDGE_SCALE", 1)
	#from pdb import set_trace;set_trace()
	TEMPLATE=os.getenv("TEMPLATE")
	THRESHOLD_ILOT= int_env_default("THRESHOLD_ILOT",1)
	end_date = date.fromisoformat(END_DATE)
	date = date.fromisoformat(DATE)
	td = timedelta(days=BY_DAYS)
	td2 = timedelta(days=BY_DAYS/2)
	#https://stackoverflow.com/questions/47339121/how-do-i-convert-a-string-into-an-f-string
	def effify(template: str):
	non_f_str = ""
	with open(template) as f: non_f_str=f.read()
	return eval(f'f"""{non_f_str}"""')
	def is_ilot(node:str, edge_dict:tuple) -> bool:
	"""ilot == has only 1 link back and forth either in (from,) or (,to)"""
	count=0
	for edge in edge_dict.keys():
	if node == edge[1] or node == edge[0]:
	count+=1
	if count > 2:
	return False
	return True
	patt_to_col = dict({
	"e2m":"red",
	"emmanuel.macron":"red",
	"emmanuelmacron":"red",
	"alexis.kohler" : "midnightBlue",
	"gabriel.attal" : "orange",
	"sachahoulie@" : "green",
	"sejourne.stephane" : "grey15",
	"stephane.sejourne" : "grey15",
	"clement.beaune" : "aquamarine",
	"olivia.gregoire" : "darkOrange",
	"veranolivier":"green",
	"julien.denormandie" : "indigo",
	"sibeth.ndiaye" : "orange",
	"iledefrance.fr" : "cyan3",
	"barbara.frugier" : "green",
	"cedric.o" : "purple",
	"gouv.fr" : "yellow",
	"benjamin.griveaux":"blue",
	"iledefrance.fr" : "beige",
	"ismael.emelie" : "orange",
	"benjamin.griveaux":"SteelBlue",
	"laurent.bigorgne" : "darkBlue",
	"jean.pisani-ferry": "chocolate",
	"ismael.emelie" : "Olive",
	"gregoire.potton" : "grey20",
	"eric.dumas":"salmon",
	"alexandre.benalla" : "darkGreen",
	"pierre.person" : "darkBlue",
	"pierrperson" : "darkBlue",
	"quentin.lafay":"grey10",
	"jesusetgabriel.com" : "crimson",
	"fm.alaintourret" : "purple",
	"langannechristine" :"darkgoldenrod4",
	#"en-marche.fr" : "chocolate",
	#"paris.fr" : "yellow",
	})
	detected_edges_color =dict()
	wl = lambda s : any(map(str.startswith, patt_to_col.keys() ,s))
	def in_wl(mail : str):
	for l in patt_to_col:
	if mail.startswith(l) or mail.endswith(l):
	return l
	def wl(pair: tuple):
	for l in patt_to_col:
	if in_wl(pair[0]) and in_wl(pair[1]):
	return patt_to_col[in_wl(pair[0])]
	#assert wl(("jesusetgabriel.com", "jesusetgabriel.com")) == "crimson"
	is_vip = lambda t:all(map(in_wl, t))
	def set_color(mails:tuple):
	for mail in mails:
	detected_edges_color[mail]=wl(mails)
	first = True
	#from pdb import set_trace;set_trace()
	while ( not TEMPLATE and first ) or (TEMPLATE and date < end_date):
	first=False
	direct=mdict()
	final = mdict()
	conn = psycopg2.connect("dbname=ml host=192.168.1.32 port=5432 user=jul sslmode='require' ")
	with conn.cursor() as sql:
	sql.execute(f"""SELECT "to", "from" from mail where DATE BETWEEN '{date}' AND '{date+td}';""")
	while t := sql.fetchone():
	for fr in t[0]:
	fr=fr.strip()
	for to in t[1]:
	to=to.strip()
	if fr != to and fr and to and not {fr[0], to[0]} & {'"', "'"}:
	direct += mdict({ (fr,to) : 1 })
	tk= list(direct.keys())
	def has_more_than_n_neighbour(email: str, n :int, final : dict):
	count = 0
	for k in final.keys():
	if len(set([email]) & set(k)):
	count+=1
	if count >n:
	return True
	return False
	for k in tk:
	# dont modify a dict you iterate hence copy of keys
	if ( is_vip(k) or MAX_MAIL >= direct.get(k,0) + direct.get(k[::-1],0) >= MIN_MAIL ) and k not in final and k[::-1] not in final:
	final[k]=direct[k]
	final[k]+=direct.get(k[::-1],0)
	else:
	try:
	del(direct[k])
	except KeyError:
	pass
	try:
	del(direct[k[::-1]])
	except KeyError:
	pass
	tk= list(final.keys())
	for e in tk:
	# dont modify a dict you iterate hence copy of keys
	if not has_more_than_n_neighbour(e[0],THRESHOLD_ILOT,final) or not has_more_than_n_neighbour(e[1],THRESHOLD_ILOT,final):
	try:
	del(final[e])
	except KeyError:
	pass
	try:
	del(final[e[::-1]])
	except KeyError:
	pass
	else:
	set_color(e)
	color = "".join([ f"""{i[1]} pour {i[0]}{[", ",chr(0x0a),][(n%4)==3]} """ for n,i in enumerate(patt_to_col.items()) ])
	final *= EDGE_SCALE
	conn.close()
	with open(f"out/rec.{date}.dot" , "w") as f:
	quoted_mail = set({})
	for _t,_f in final.keys():
	quoted_mail |= set([_t])
	quoted_mail |= set([_f])
	edges = "\n".join([ f""" "{e}" [shape=rectangle fillcolor="{c or "lightblue"};.1:white" color={c or "green"} style=striped ];""" for e,c in detected_edges_color.items() if e in quoted_mail ])
	title = f"""label="Sociogramme de {date} à {date + td} extrait des macron leaks orienté gouv.fr, personne d'intérêts (vert), victime du hacking (rouge), et président (bleu)\n
	entre [ {MIN_MAIL}, {MAX_MAIL} ] échangés \n
	plus gros liens au dessus de {CUT_SIZE} mails échangés entre interlocuteurs \n
	couleur par priorités selon les origines \n
	{color}"
	"""
	if not TEMPLATE:
	print("""
	graph Sociogramme { """ + f"""
	fontname="Comics sans MS"
	size=120
	ratio=0.588
	{title}
	labelloc="c"
	labelloc="t";
	start=1
	{edges}
	node [ shape=rectangle style=striped fillcolor="slateblue;0.1:white" gradientangle=90 ];
	"e2m@en-marche.fr" -- "e2m@cabinets.finances.gouv.fr" -- "emmanuelmacron3@gmail.com" -- "emmanuel.macron@en-marche.fr" -- "emmanuelmacron@en-marche.fr" [label="is"]
	"alexis.kohler@en-marche.fr" -- "alexis.kohler@cabinets.finances.gouv.fr" [label=is];
	"stephane.sejourne@en-marche.fr" -- "stephane.sejourne@cabinets.finances.gouv.fr" -- "stephane.sejourne@gmail.com" [label=is];
	"julien.denormandie@en-marche.fr" -- "julien.denormandie@cabinets.finances.gouv.fr" [label=is];
	"benjamin.griveaux@en-marche.fr" -- "benjamin.griveaux@sante.gouv.fr" [label=is];
	"quentin.lafay@en-marche.fr" -- "quentin.lafay@cabinets.finances.gouv.fr" -- "quentin.lafay@sante.gouv.fr" -- "quentin.lafay@gmail.com" [label=is];
	"pierrperson@gmail.com" -- "pierre.person@en-marche.fr" [label=is];
	"ismael.emelien@en-marche.fr" -- "ismael.emelien@yahoo.fr" [label=is];
	""" ,file=f)
	for k,v in final.items():
	print(f""" "{k[0]}" -- "{k[1]}" [color={wl(k) or "lightblue"} penwidth={v} ];""", file=f)
	print("}", file=f)
	else:
	print(effify(TEMPLATE), file=f)
	date += td2

view raw reciprocal.moving.poi.py hosted with ❤ by GitHub