The Voynich Manuscript

This is a collection of various computational methods I have used in an attempt to get some understanding of the cipher used in the Voynich manuscript.

Comparisons

Here

Strong's Cipher

Supposing that Strong's conjecture about the structure of the VMs cipher is correct, we use a computational method to try to infer the alphabets and their sequence with the use of a large Latin diictionary.

Details are here.

Mapping the Voynich Manuscript to Music

Using a Java Media Framework MIDI application that processes the Voynich text, aach character is mapped to a note on the MIDI scale in a two octave range: Note = 48 + i mod 24

Each word is played as one or more chords of these notes, using the Grand Piano instrument.

If a gallows character ("h" or "k") appears in the word, then the notes preceding the gallows character are played as a chord, followed shortly by the notes after the gallows character, but shifted by either 1 octave ("h") or two octaves ("k"). If the character "9" appears as the last in the word, then the delay before playing the next word is reduced.

Using this method, here is what the common Voynich word "8am" sounds like: 8am

The word "okoe": okoe

The word "4ohc89": 4ohc89

And the complete Folio f1r (a long pause is taken after each paragraph on the folio): Folio f1r

And Folio f3r

Folio Similarities

More details here.

A Phonetic Attack

Comparing the phonetic content of various texts with VMs "words", using "Soundex" and "Double Metaphone"

For details, go here.

Phrase Analysis

This study is based on the idea that the Voynich "Words" are in fact codes for parts of plaintext words. The Voynich symbol "9" is used to delimit the words. A Genetic Algorithm is used to explore the cipher, using a large body of common Latin phrases (as opposed to single words).

For details, go here.

Vowel/Consonant Group Encryption Scheme

For details, go here.

Transitions between Glyphs

This study looks at the probabilities associated with glyph sequences in the VM. E.g. which is the most likely glyph to find at the beginning of a word in the herbal section?

For details, go here.

Anagram Analysis of Star Labels etc.

This investigation was inspired by Robert Teague's deciphering of e.g. f68r3.

For details, go here.

Analysis of the Prefixes, Stems and Suffixes in the Voynich Manuscript

Extract:

For N=3, looking at the Herbal folios f1v-f20v inclusive, 1331 different words. 

Confirmed valid prefix/stem/suffix counts 99 252 111 
Prefix/Stem/Suffix frequency, normalised 
4ok     0.1010101               o89     0.05952381              o89     0.09009009 
4oh     0.07070707              1oe     0.055555556             8am     0.09009009 
1oe     0.060606062             4ok     0.055555556             1c9     0.054054055 
1oh     0.04040404              8am     0.04761905              1oy     0.054054055 
ok1     0.04040404              4oh     0.04761905              1oe     0.045045044 
8oe     0.030303031             1oy     0.03968254              coe     0.036036037 
1oy     0.030303031             1c9     0.031746034             cc9     0.027027028 
1co     0.030303031             1co     0.023809524             e89     0.027027028 
1ok     0.030303031             8oe     0.023809524             ham     0.027027028 
4oj     0.030303031             coe     0.01984127              2c9     0.027027028 

For more results, go here.

Useful Links

This is by no means an exhaustive list, but these are some of the Voynich sites/pages I have found useful, interesting and/or amusing.

Rene Zandbergen's site

http://www.voynich.nu/

Voynich Central

http://www.voynichcentral.com/

Elmar Vogt's blog

http://voynichthoughts.wordpress.com/

H.R. Santa Coloma's site

http://www.santa-coloma.net/voynich_drebbel/voynich.html

Philip Neal's webpages

http://voynichcentral.com/users/philipneal/

Attempts to crack the cipher with Genetic Algorithms

Some Assumptions about the VM text

My assumption in the work with Genetic Algorithms (GAs) is that the Voynich is a terse, compressed, abbreviated n-Gram conversion of the plaintext language. The scribe would have consulted a table of n-Grams while converting the Latin (or whatever) to the VM text, perhaps even writing the enciphered words out on a separate work sheet before copying them into the VM.

For example, assume the following extract from a set of possible n-Gram conversion rules (or "mappings") used by the scribe:

then, to convert the English word "hypothesis" to Voynichese, we do as follows:

h => f

y => a

pot => oc

he => c

s => 8

is => 9

so that "hypothesis" => "faocc89"

The inverse mapping can be used to decode the Voynich word (perhaps not unambiguously ...).

Clearly, there are a very large number of possible n-Gram mappings!

Candidate Decipherings of f27v

Go here to look at folio 27v, and see the various Genetic Algorithm attempts at deciphering the text.

Voynich Herbs

Edith Sherwood has a web site where she details compelling possible identifications for the plants depicted in the "herbal" pages of the VM.

Dana Scott's page also has plausible identifications for the plants.

As has often been pointed out, if we look at the first Voynich "word" that appears on each page of the herbal part of the VM, we find that those words are unique, or appear elsewhere very rarely. It thus seems reasonable that the words may be the names of the plants depicted.

The GA was set up to find a set of n-Gram mappings that would convert a list of 111 Voynich first herbal words into Latin/English or Spanish. For this, dictionaries of Latin, English and Spanish herb/plant names were used.

The GA sought a mapping that would convert all the Voynich words for herbs/plants into as many valid plaintext (Spanish, English, Latin) words as possible. The best result was for a mixed English/Latin dictionary (see table): 31 of the 111 Voynich words were converted, about 30% success rate.

(One should never expect 100% success, due to missing names in the dictionary, transcription errors, missing n-Grams, incomplete n-Grams etc..)

The results are shown below in tabular form, together with Dana Scott's and Edith Sherwood's identification. The first column shows the folio in the VM, the second shows the first Voynich word on that folio. For the GA identification columns (3 and 4) the Voynich mapped word is shown, in quotation marks if not found in the associated dictionary, and in bold if found in the dictionary.

Note that, probably unsurprisingly, nowhere do the IDs from the GA in Spanish, English/Latin and Scott/Sherwood, agree! NOT YET, anyway :-)

(What amuses me about about this mapping technique is that it tends to produce words that sound plausible in the target language. E.g. for f4r the Latin/English word "paptise" sounds like a valid word.)

Character Analysis

See the section "Bibliography" at the end for the provenance of the various texts used.

• Voynich. The Herbal folios from the Voynich are processed, generating frequency tables for single characters, dual characters (digraphs), trigraphs and quadgraphs (1/2/3/4). These are also called n-Grams. Frequency tables for word lengths and common words are also generated.
• Latin. A few books from Augustinus and other places (see Bibliography)
• A German cookbook from 1553
• The Book of the Courtier, in English from 1561
• French from 1367
• a Latin Herb garden description
• Spanish from C16

Here is the ranking of the most popular words in each text (full tables are available in the Excel spreadsheet)

The 1/2/3/4 n-Grams are calculated as follows. Suppose the word "sesame" appears in the text. The following counts are made:

• "s" 2
• "e" 2
• "a" 1
• "m" 1
• "se" 1
• "es" 1
• "sa" 1
• "am" 1
• "me" 1
• "ses" 1
• "esa" 1
• "sam" 1
• "ame" 1
• "sesa" 1
• "esam" 1
• "same" 1

Then, each count is weighted by the length of the group, yielding s=2, e=2, a=1, m=1, se=2, es=2, sa=2, am=2, me=2, ses=3, etc. The counts are added to running sums for each distinct group found, and normalised at the end of processing.

The full distributions are plotted in graphical form. Here is the single character distribution. The leftmost points correspond to the top row of the middle table above.

Here is the word length distribution. (Note the striking similarity between mediaeval German and Voynich.)

Here is the frequency of the appearance of words. Note that the Latin Herbs (a specialized text) has a similar shape to the Voynich Herbal.

Here is the 1/2/3/4 letter group frequency plot.

See also: Comparison of "Voyn_101" and "PagesH" Transcriptions

Genetic Algorithm

Armed with the character analysis above, the following cipher presents itself as a possibility: in the plaintext, convert each group of 1, 2, 3 or 4 characters into a Voynich group of 1,2,3 or 4 characters. We call this a "mapping". For example, referring to the tables above, when creating Voynich from Latin, an obvious cipher just based on the frequencies of the groups would be:

e => o, i => 9, ...

er => 4o, is => ok, ti => 8a, ...

ent => 9k, ant => A, ...

This can be encoded into an algorithm thus which maps strings in "repl" to strings in "seek"

	String seek[] = {"4ok1",
			 "4oh","8am","1oe","4ok","ok1","o89","1oy","oh1","o8a","oha","ohc","c89","1co","k1o","1c9",
			 "c79","h1o","1o8","oko","oho","coe","8ae","co8","k19","h19","8ay","ham","hcc","koe","oka",
			 "hco",
			 "1o", "oe", "oh", "4o", "ok", "8a", "89", "am", "1c", "oy", "o8", "co", "ay",
			 "k1", "h1", "19", "hc", "c9", "ha", "ae", "79", "2o", "cc", "ko", "ho", "c8", "9h",
			 "9k", "c7", "2c", "ka", "kc", "1a", "an", "h9", "o,", "e8", "k9", "ap", "8o", "e,",
			 ",1", "7a", "81", 
			 "o",  "9",  "1",  "a",  "8",  "c",  "h",  "e",  "k",  "y",  "4",  "m",  ",",
			 "2",  "7",  "s",  "K",  "C",  "p",  "g",  "n",  "H",  "j",  "A"};

	
	//Latin
	String repl[] = {"un",
			 "ri", "on", "f",  "es", "g",  "em", "de", "se", "co", "ne", "ur", "si", "ic", "ui", "me",
			 "ere","eb", "la", "ma", "le", "id", "bu", "nti","no", "cu", "eba","qui","ie", "al", "ul",
			 "ns",
			 "c",  "d",  "l",  "er", "is", "ti", "nt", "en", "re", "in", "um", "am", "us",
			 "te", "it", "v",  "tu", "ta", "ra", "di", "an", "ni", "li", "et", "ba", "ae", "mi",
			 "ent","st", "h",  "nd", "ci", "pe", "im", "ua", "io", "tur","il", "ve", "iu", "as",
			 "vi", "ita","ca",
			 "e",  "i",  "a",  "t",  "u",  "s",  "r",  "n",  "m",  "o",  "p",  "b", "q",
			 "qu", "at", "or", "ia", "ar", "ce", "ib", "ec", "ab", "ru", "ant"};

Such an algorithm is used inside a Chromosome of the Genetic Algorithm. The Chromosome decodes Voynich into Latin by  matching character groups in the Voynich word against each of the strings in the "seek" list in turn. If a match occurs, then the  Voynich group is translated into the Latin group in the "repl" list at the same position. Thus "4ok1" in Voynich is translated into "un" in Latin.

Once the Voynich word has been translated into Latin, the Latin word is looked up in a Latin dictionary. If the word is found, then the "cost" of the Chromosome is increased ... if the word is not found, then the cost is decreased. After all words in the Voynich text have been converted to Latin, and the aggregate cost of the Chromosome evaluated, it can be judged whether the mapping "seek" to "repl" is a good one or not.

Generating the Chromosome Population

We generate a large number of Chromosomes, each of which has a different "seek" to "repl" mapping. We do this by simply shuffling the order of the "repl" strings in each Chromosome.

Thus, one Chromosome may map "4ok1" to "s" and another may map it to "qui".

This population of Chromosomes is then evaluated: each Chromosome converts the Voynich words to Latin, and each then gets a cost. The higher the cost, the better. The highest possible cost would be a Chromosome that had a seek-repl mapping that produced a valid Latin word for each Voynich word.

Training the Chromosomes

The Chromosomes are ordered in decreasing cost, and then the best of them (i.e. at the top of the list) are "mated" together to produce offspring Chromosomes. The mating process essentially involves taking sequences of the "repl" strings from both parents and combining them to form a new "repl" string.

Some of the offspring Chromosomes are then "mutated". This involves replacing one of the "repl" strings with some randomly selected letters from the Latin character set.

The process repeats (ordering the Chromosomes, mating the best ones, mutating the offspring) until a predefined cost value is reached, or the population of Chromosomes refuses to improve itself.

In the end, the best, trained Chromosome will contain the optimal arrangement of "seek" to "repl" mappings for conversion of Voynich to Latin.

The same procedure can be used for a Voynich to English, to German, French or any other language, provided that a dictionary and substantial texts are available to process.

First Results - Voynich to Latin

This is a limited attack on the first five "sentences" of f1r, using 200 chromosomes and a Latin dictionary of around 15,000 words. The best chromosome scores 9.4 after 500 training epochs (cf a score of 20 for a one-to-one translation of Latin into Latin).

Here are the deciphered sentences:

1) Voynich: fa19s 9hae ay Akam 2oe !oy9 �scs 9 hoy 2oe89 soy9 Hay oy9 hacy 1kam 2ay Ais Kay Kay 8aN s9aIy 2ch9 oy 9ham +o8 Koay9 Kcs 8ayam s9 8om okcc9 okcoy yoeok9 ?Aay 8am oham oy ohaN saz9 1cay Kam Jay Fam 98ayai29

Latin: ?ereieas vias is asasita meas ?ereis ?astuas is quinti mensis asereis vis ereis sttunti viasita viis as?as alis alis qui? asisere? nti quere ere viis ?ita alamisis altuas ereis asis quiita quantis querenti ntiviquis ?asis qui amita ere am? asere?is viis alis ?is ?is isereere?viis

2) Voynich: � o8ay ? !oe Go9 o98ay !?s Foam #o8ay9 9+c9 2o89 oh1o9 ok1oe 1oK9 os19 8an 1oy hos 8am 2os Foe 2o89 8an oy kco8(

Latin: ? quinti ? ?vi ?amis amisere ??as ?amis ?quintiis is?ere meam famis quas qualis amasie quid ere quias qui meas ?vi meam quid ere a ntita?

3) Voynich: � 98an Gcsam oes Gc9 9kan 2o29 Jo8ae cs oh2o h2o9 okazn okcoe ohaN 2o8an sv19 8am 2o9 Hc9 ho8am G9 Jo8aIes L9 2o oe7an 7 8an om 1 oe o8am 1o8an 189 ohan 7aN K9 ho8 8am 2Hc9 Hoy 1ay 3c9 hoe1oe 1oe hoy 1ae 3o 1oe 2o8aN h+9 h1e 8oy 1o8am 2o hccap 91o k1c9 1chan +cog2oe 89898 K9 8ai�9 (ko 3oe 3c ho82c9 Jcae9 8ay an 8an H98s 81ay +Kam ohaZ1c9 �19 �koe Koes 8aoEko 2oh 1oy 1c9 8an Hc9 okoe 8aM

Latin: ? isquid ?tuasis vias ?ere quis meviis ?quias tuas eme asmeis no?d quere am? mequid as?ie qui meis vere quiqui ?is ?qui?asas ?isme vitad ere quid amita ere quiita alis viam amd ta? alis quis qui vivere vere quinti ?ere quiasere ere quinti quias ?am ere mequi? as?is cas quinti alis me quere isqu vere vistd ?ereasmeas amams alis qui??is ?at ?vi ?tu quisquis ?tuasis ere is quid vissas quis ?alis am?qui ?ie ?as alvias quiam?at meas ere qui quid vere amas qui?

4) Voynich: Go 2am !oh1C9 1oe k2o8ccs9 2c9 g98C9 19 yo 8ay (7an 1oe 8an Kae 8ay +cay ham 8ay 5c9 Kay 1o?o ham 2oam ohoe 8am ?ay Koe 8am Koe8ay 91C9 ohC9 oh9 8am oh1c9 hoham o?1oe hA719 8ae 81co 2o89 ho?19 K9 oh1c9 hcc9 hcc9 8ae 1koy 2o? 1oe 1H1ok9 1okc9 81am

Latin: ?am viis ?fnsis ere atmesantasis quis asissnsis ie ntiam ere ?tad ere quid alas ere ?tuis qui ere ?ere alis qu?am qui meis quas qui ?is alvi qui alruis isvinsis ensis eis qui fere quiqui am?ere asasereie qui quere meam qui?ie alis fere quis quis qui viatnti me? ere vivquat quantis quis

5) Voynich: h1s9 1o8am oe oek1c9 1ay Fax ap 9kcc9 1ay oy o19 81o eho89 oho8ay 1o89 8o H9 HoH9 59 8h3C9 K9 hok1o89 8ae 8oe 1ohco 8az 8ap so1c9 1o ho89

Latin: casis alis vi vivere quinti ?ere? ere quantis quinti ere amie quam asquiam quere alis qui vis vamvis ?is sas?nsis alis quiquam qui quias quere qui? quin asamqui qu quiam


Here is a "translation" from Latin to English (chromsome cost ~5)

Latin: Magnus es, domine, et laudabilis valde: magna virtus tua, et sapientiae tuae non est numerus. et laudare te vult homo, aliqua portio creaturae tuae, et homo circumferens mortalitem suam, circumferens testimonium peccati sui et testimonium, quia superbis resi stis: et tamen laudare te vult homo, aliqua portio creaturae tuae.tu excitas, ut laudare te delectet, quia fecisti nos ad te et in quietum est cor nostrum, donec requiescat in te. da mihi, domine, scire et intellegere, utrum sit prius invocare te an laudare te,
 et scire te prius sit an invocare te. sed quis te invocat nesciens te? aliud enim pro alio potest invocare nesciens. an potius in
vocaris, ut sciaris? quomodo autem invocabunt, in quem non crediderunt? aut quomodo credent sine praedicante? et laudabunt dominum
 qui requirunt eum. quaerentes enim inveniunt eum et invenientes laudabunt eum. quaeram te, domine, invocans te, et invocem te cre
dens in te: praedicatus enim es nobis. invocat te, domine, fides mea, quam dedisti mihi, quam inspirasti mihi per humanitatem fili
i tui, per ministerium praedicatoris tui.

English: ?seouething be? seneenbe? her ngehaseatkese srabe? riouethse hathethese these? her sesestoudeeth theeth teeth bend ethingsing? her ngehaseras and sound rnesne? rasesese stssrane ingyengendeth theeth? her rnesne eltheelsoutide ssstheethands seets? eltheelsoutide andtheethiis haingyera seeth her andtheethiis? these sehahatheise yeyethese? her thetheeth ngehaseras and sound rnesne? rasesese stssrane ingyengendeth theeth?the s?elthese? hand ngehaseras and bevingandnd? these oungesera tese sese and her herthehering bend hathe tethedes? seinge yethebeyend her and? sese enrse? seneenbe? seelye her herandsevouti? handdes yend staing hersneyeye and the ngehaseras and? her seelye and staing yend the hersneyeye and? these these and hersneyend beseeltese and? raise teeth sttheneraou stneandthe hersneyeye beseeltese? the stneraing hersneyease? hand seelrase? senesnesene sehaands hersneyeouv? her seeth teeth ingyesebedev? sehand senesnesene ingyebev yebe stndsseyevs? her ngehaseatsend seneenething the yethedev sing? seethyevbe teeth herndmasend sing her herndmavbe ngehaseatsend sing? seethnds and? seneenbe? hersneyede and? her hersnenges and ingyebede her and? stndsseyethese teeth be teise? hersneyend and? seneenbe? ouvebe these? seeth besethese enrse? seeth hersestsendthese enrse hathe ringthengeands oukesese these? hathe enmathesis stndsseyendssse these?

I'm still digesting these, but some thoughts:

1) The size of the Dictionary used for training is important
2) Opinion: the translation Latin->English looks less plausible than the Voynich->Latin
3) After training, the solution found is just one solution - not necessarily the optimum solution (there may be many local minima).
4) Running the GA several times with the same input parameters yields different translations
5) The cost function only uses a dictionary lookup, but should perhaps penalise chromosomes that have multiple Voynich n-grams that map to the same Latin n-gram. E.g. if all Voynich n-grams mapped to "qui" then the chromosome would score very highly, but produce repetitious nonsense.

Results with a Refined GA and with Landini's "Challenge" Text

Here are some thought-provoking results from looking at Landini's challenge text, as suggested by Knox, the VM text, and comparisons with English, Latin, German, French and Spanish. These use a new form of the Genetic Algorithm, described below.

Summary

It looks to me like that Landini either generated his text from a transcription of the VM itself, or his algorithm for generating that text is a good emulation of the encoding process used in the VM. In other words the Landini "language" is a good candidate as a plaintext language for the VM, as opposed to the European languages tested.

Results

Here is a table which shows the GA's efficiency at converting/translating between Voynich, Landini, and the other languages.



(In the table, the best possible score is 1.0 - see below for an explanation)

Asking the GA to translate English to English, or Latin to Latin, etc. results in a high efficiency score, as expected. Note that the Landini to Landini  efficiency is 0.97 - almost perfect.

The GA performs moderately at converting between the languages and the Landini text. But what is most striking (to me) is the good efficiency for converting Voynich to Landini (0.74) and Landini to Voynich (0.89)

Some Notes on the table

To look at this I revised my GA code so that it was more flexible, and I jettisoned the use of separate dictionaries. Here is how the GA now functions. It can convert/translate between any language text samples.

1) Two text files are read in: the "source" text, and the "target" text. This could be, for example, a source file containing Landini's text, and a target file containing Spanish text, if we want to convert from Landini to Spanish.

2) The text in each file is processed separately, producing two word lists, and two sets of n-Gram frequency tables.

3) The chromosomes are generated with random mappings between the source n-Grams and the target n-Grams

4) The GA evolves the chromosomes by trying to maximise their cost. The difference now is that when a target word is generated from the source text using the mappings, it is looked up in the target word list created in 2) above, rather than in a separate dictionary.

5) After training, the best chromosome can have a maximum cost value of 1.0, which would correspond to a perfect conversion between the source text and the target
text (i.e. every word produced from the source text is found in the target text dictionary)

6) So we can feed the GA with two identical texts, and after training the score of the best chromosome should be 1.0, and indeed it approaches that (it doesn't quite
get there because only the top 100 n-Grams are translated, and so some characters in the source text cannot be translated).

7) The word and n-Gram frequency lists are made from the entirety of each text, but (for this exploratory study) the training takes place on only the first 50 "words" in the source text, and uses only the first 100 n-Grams for mapping.  Thus if the 50 words of Voynich chosen contain several rare characters, then for those the mapping will fail because those rare characters do not appear in the n-Gram list, and this will result in a lower score.

8) In all cases the "X->X" score in the table (i.e. the diagonal)  represents the best score possible for that language, and is a normalisation for the other numbers in the table. I should really revise the table and divide out the off-diagonal scores by the diagonal normalisations.

9) An improvement would be to configure the n-Gram list to be, say, 200 long, and use more source (Voynich) words for the training. The downside of this is mainly execution speed.

10) These runs were with n-Grams up to 3: it would be better to go to 4 at least.

11) I think Landini gets good scores because the character set he uses is very small. Knox comments " A factor must be that the Landini Challenge has built-in frequency matches to any transcription of the VMs. Also, there is no meaningful correspondence in the letter sequence of one word to another in Landini. The difficulty fits what I said the VMs may be."

Letter and nGram Frequencies

Here are the letter frequencies for the whole of the VM, in the Voyn_101 encoding

Here are the word length frequency counts

The most frequent words

Here is the 1/2/3/4 nGram frequency for the whole VM, showing only the top 100

Bibliography