Yes, hello, hello, everybody. Welcome to Nikolai Stoke. Yes, unfortunately, he was
able to make it. So, yeah, my imposter syndrome was never as bad as today because, yeah,
that's not my talk. Okay, I will try hard to, you know, to deliver what he wanted to deliver.
Nikolai's friend of mine, he's an amateur musician and a music lover. So, he put an epigraph to
his talk, saying from a French list. And he asked me to tell that this is like free open source
conference and at least was probably one of the free spirit like people and history. And
mine this epigraph will get to it later. Well, why just don't print out like we saw examples of,
you know, the document producing like technologies. So, probably we have somewhere this print
menu item or expert to PDF or expert to like auditi. Why just it doesn't work. Like, you know,
from practice, it doesn't. So, what to do with, for example, you have code listing and it has
like very wide line and what do you do? Like, you may scale, just make it smaller, but scale,
if you are doing like semantic markup, scaling is not semantic, you know, part. Or you may
change just to use landscape orientation for this page only. And again, it's not not a part of
semantic or should it be like should we insert some hints there? It's not known. Or probably we
would like to do both. Or probably we would like to fire an exception like, you know, now your
code building failed and author please rewrite your code example so that your snippet will contain like
short lines. Or probably you would like to rub them and again there are options. You can use
line feed and spaces, but if you care about validity of your code, somebody copies it from PDF
and inserts it like it may break. Or you may use indents, which will work with text processors,
but will not work with PDF. So, choices, choices, choices. And while on the web, in HTML,
we can just add horizontal scroll bar and everything. And this like button copy to clipboard.
And that's just one example. So, there is a core mismatch. semantic markup meets the rigid
world of print. And yeah, what's interesting like this talk is like doing the opposite, like going
in the opposite direction from the doc link that we like heard before because now we are going back
from like doc link back from printed media to semantics and we are going from semantics to printed
media. And this is also like very hard task. So, the main focus like what Nikolai wanted to present
in this is this unidoc publisher repository, GitHub repository. This is actually a result of evolution
because like over the years trying to solve this problem. He did three approaches.
Uh, first one was the access cell of four approach from aski doctor. The second one is the open
document converter. So, the idea was simple. You have aski doc document just produce nicely looking
open document like the library office writer format for this. And finally, unidoc publisher,
which claims to be like any markup to any printing, painting rendering engine, quite a series, quite a
bold claim. Let's see like what it does and how. First of all, what formats do we have for printing?
Like first that comes to mind, of course, PDF, PDF of course, if we have PDF we can print it.
Second is text processing format, right? So, you if you are using word or library office,
like you can print it because there are page, you know, break HTML. Who thinks that HTML is a
format for printing? Wow, some people, some people do. Well, actually we have CSS page media,
which is a CSS extension, which defines styling specific for printing. But the harsh truth of life
is that also we have been with CSS page media for 15 years. Now, still browser support is limited.
And what in practice, like how it is using practices, why are some open source tools,
which convert HTML, plus CSS, page CSS into PDF. So, it's kind of intermediary format,
rather than, you know, well, used format. But still, technically, yeah, this is possible.
Okay, how do we render to this format? Well, if you want to go low level, really low level,
good luck. You can use native PDF generating. There is access LFO. There is tech, of course,
if we are like speaking about PDF and printing, we cannot miss it. And as mentioned before,
HTML plus page media CSS. Also, if we have, like text processor, you know, files,
we also can render to, you know, print it media either by direct printing or converting it to PDF.
The truth is that these technologies are not aligned in a great number of various ways.
So, yeah, we can compare every way, as its own limitations and problems. Like a patch,
fob has long-standing problems with dots in a table of contents. Libre office writer doesn't support
typography, like keep with next, with in tables. Microsoft doesn't recommend using, like,
automation, automating work, actually, to produce PDF. So, there are lots of problems, and it looks
like, you know, old joke about those paleologists who meet in a narrow cave, and one says,
I'm coming from the dead end, and the other says, like, I'm coming from the dead end, too.
So, like, dead ends everywhere, and the world of printing is the world of constraints.
And those constraints, they differ for each technology. So, often, you end up supporting several
chains. For example, your main output is exclusively looking text tech book, and in order to,
you know, navigate reviews or approvals, you are sending Libre office files just for this,
you know, workflow. So, there are no universal solutions. And Nikolari Kamens is still
unidoc publisher, if at least one of these holds. So, if you don't prepare the
documentation, especially for printing purposes. So, if you are doing something like, you know,
on Torah, the documentation like huge tutorials website, and you just need occasionally,
you just need to present a PDF to somebody. If you are automating the documentation generation,
I'm going to speak about it one like our later. And, like, much of your documentation is not
manually produced, but automatically. And you hope it will look good, no matter what will be generated.
And, if you're output format is one of the text processing format, this is also impossible.
So, where do we start? Like, where did Nikolai start? He first he tried to create and ask
you doctor open document converter. So, the idea was the following. So, ask you doctor,
parses it's own mark up into an abstract syntax tree. You may extend it, you may transform this
IST with the aski doctor tree processor and, you know, run each template. So, actually writing
a custom aski doctor processor. And, the writer can be already done either with pure ruby,
or with special templates. So, this is how it looked in practice. So, in the left side of this slide,
this is a simplified AST of an aski doctor document after parsing to the right. There is a slim template.
And, you see, we have him mixed ruby code, everything which starts with the dash is ruby.
Everything which doesn't start with dash is a part of the template. It's just XML output.
So, yeah, you can use aski doctor parser, this extension point that it provides and built
whatever you like, including open document format. It was great, but, yeah, you know, this template
cannot be universal. Everybody wanted their own, you know, particular features. And, it wasn't
it was hard to graduate just part of the template. People had to copy this template paste it
and do actual their own work. That's not how it can be maintained. And, then you should invent
something for styling. By styling, we mean the styling for open documents for the word processor format.
You know, then when you select text and you apply style from the list of styles. And, you say,
okay, you have semantic styles and the word processors. They do support styling.
Well, actually, this is kind of different things because, like here, like both bold and green
are applied to some part of the text, but in open, like in libre office or in Microsoft,
what you can only apply one style. So, you actually need three styles, one for bold, one for green,
and one for bold green, and all the combinations. So, what did he have to do is,
like invent slightly extended intermediary open document format, which contains not only standard
attributes, but extended semantic attributes, and then post-processing it in order to add missing styles,
like missing combination of styles. Well, it kind of worked. And, one of the interesting,
like finding was that users of this approach, they started to transform this extended open document
format. This wasn't intended, intended, you know, feature or intended capability of this product.
Yet, Nikolay noticed that people wanted to transform a ST before converting it into something
printable. Yet, this idea of post-processing and adding, like separately styles, proved to be useful.
And, also, Nikolay liked very much the idea of using Gradle, Gradle is a built-to-from-JVM world,
quite powerful with statically typed DSL. So, it has statically typed checks before you run
this Gradle script. Magnificent in gluing all the parts together. So, before we go to the second step,
the Unidoc publisher, like some final thoughts, like if creating, universal converter is impossible,
which will create meta-converter, a platform for building converters. So, that everybody who
face the problem of transforming their existing, like, body of the documentation into PDF,
they can just simply do it by a custom scripting, but without too much effort.
And, what are the requirements? Native converter is a reader. We will explain this at the next slide,
what's your name? So, we're learning AST. So, yeah, this should be more or less easy to do, right?
And, styling as a separate focus, well, this is just, you know, learning or finding from the previous
previous version. And, yeah, ideally, it should have good integration with CICD with focus on
homogeneity. What is meant here is that everything must be from the same ecosystem, like either JavaScript,
or Java, or Python, but not mix, because mix is often problematic. What do we mean by native
converter as a reader? The idea is that each converter outputs HTML and HTML is quite semantic.
So, if we take as an input, an HTML output, we can actually build the universal thing.
So, markdown can produce HTML, ask your doctor can produce HTML, Microsoft Word, whatever, every,
you know, we can produce HTML. So, you can do it, just read it and there are good, you know, HTML,
reading HTML parsing libraries in open source, if it's in Java, it's JSON, which is quite, quite good.
So, the experiment, let's convert this presentation, the presentation that you see
into printable form. So, this presentation, by the way, is written in asking doctor. So, this is the source
code of Nikolai's presentation. What we want to get is document like this. So, it's LibreOffice
Reiter, and as you can see, it's converted into this printable form. The fact that it is converted
into something else, you can see before you're all nice. It's this presentation, it's asking doctor
review JS pipeline, which just converts it into this clickable, you know, HTML. So, how do we
do this? Using Nikolai's tool. So, yeah, he's using Kotlin internally because, well, it's JVM approach,
is using Gradle with the Gradle script. And the following code listings, they actually as snippets
taking from from the scripts. So, every code snippet that you can see on this presentation,
is actually included from the presentation source code. We will share a GitHub repository. So,
you will see that this is actually not something copy and paste that it's included. So, this is
workable code, which actually works. I will speak like a bit more about this in my next presentation.
So, the boilerplate, this is like a snippet of Gradle script, build script. And what does it do?
Like, first we need the HTML. So, we are using a skiddoctor.js is its library, which allows
to build HTML from a skiddoctor, it's Java library, to build this HTML. Then we are
adding a template. The template here is the starting point. It's just to know that empty
word or library office file, which we are populating with content. Then some, some other technical
details, right, we need, but if we, I'm need this part, which is commented out, and just
run it without any further customizations, we will get something like this, let me show. So,
this is the default transformation. So, already like not bad, probably. So, as you can see,
there is some, you know, output. You can see page headers, but on the first page, you have
page header as well. The first slide looks ugly because like, you know, sizes of images.
The epigraph is lost. So, we need to fix it somehow, right? So, the idea is that, by default,
you are getting like pretty good result, but if you want an ideal result, you should, like,
get your hands dirty and do some abstract syntax tree transformations. That's it. So, how do you do this?
Like, after parsing, you will get access to abstract syntax tree. For example, you need to
extract the first section to build this beautiful header. So, you just filter this out by
source tag name, like take the first section, and make it title out of this. So, how does this first
section look like in askiDoctor? So, this is the source code, and actually, yeah, this is quite
interesting because it's technically, this is an askiDoctor page, which includes the part of itself
within the askiDoctor page. So, yeah, this is like a listing like a,
a part of the listing of this askiDoctor. So, we are going to extract symmetrical variables from
there. Like, you can see roles here, full name, title, photo, biolog and stuff. So, this is how we're
doing this. Like, we have access to AST, so we just, you know, filter out these roles and assign
them to a bunch of Kotlin variables. That's it. And then, having this Kotlin variables,
we can rebuild just three orange, rebuild reconstruct the header, the beginning of this document.
So, what we have here, a pinched out table, table row group, is the builder for open document format.
It can be some other format, but in this case, we are building it. And we are using this, you know,
photo by a contact variables that we extracted in our abstract syntax three. So, what do we have
in the end? Like, let me, let me show it once again. So, we have beautifully looking, you know,
document, which contains all the slide contents. It contains all the speaker notes. And also,
we have an epigraph. We didn't lose it. Also, it used some, you know, unsupported
troll of whatever feature of, you know, of ASCII doctor, which prevented it from, you know, default
expert. Now, it's being shown and it's being shown in, in, in its correct place, in, as the result.
Sorry?
Could you enable the unprintable character? Unprintable characters?
Yes, if it's possible via a builder like LibreOffice Builder. So, it's, like,
I see, you mean this one, okay?
Yeah, yeah, yeah, yeah, you wanted this, yeah, sure, sure, of course. Yeah, yeah, yeah. So, it's,
you wanted to see if it's not all, like, spaces, right? Okay, okay, okay. No, no, it's, it's building, like,
it's building, it's structurally. Yes, it's building it structurally. I'm going to skip this one.
It's, like, highly technical and the Nikolai knows better. By the way, Nikolai is available in
element chat for this group. So, he will be, uh, happy to answer everything. Uh, a bit about testing.
So, it's, uh, kind of, a lot of code already, this transformational code. And we need to make sure
that we, if we are maintaining it, if we are keeping it for some, you know, documentation project,
we need, uh, to, to put some regression testing on it. So, the idea is quite simple. It's, uh,
snapshot testing. So, uh, what we do is we prepare small, small snippets, like a small table, or,
some, you know, combination of, uh, styles. And we ask this pipeline to output the picture,
like printed. So, uh, we have an approved picture just in, in Git. And, uh, if, uh, due to some
version changes or something, uh, the picture becomes to something becomes to, um, something is sliding, right?
Uh, we see it immediately. Our, uh, CI build fails. And, uh, if we are okay with this sliding,
we may just re-approved, which is, say, okay. Now, this version is, uh, kind of fine. So,
this is how it's gonna be. But if something breaks, like, something like this, uh, table disappears
completely, right? In this case, it will be obvious that we need to fix, uh, to fix the code.
Uh, okay. So, what did we get, uh, the result? Uh, so, uh, we got everything CI friendly. What can't
be more CI friendly than, you know, a build script, grade-all build script, because grade-all is a build tool.
And everything is built, uh, with a DSL within grade-all. Uh, like, there are no declarations.
What is meant, meant here is that, uh, it's, uh, this product doesn't, uh, offer some default, uh,
behavior that you can override with some, you know, knobs and switches. And, uh, it's definitely,
like, it was, um, in principally, did there other way around. So, if you want to change something,
you need to get your hands dirty and do some AST transformations. And, uh, it has type, uh, AST,
and clean and testable code. Well, that's, get to conclusion. So, three printing is engineering.
Designed, coded, tested, automated. Printing is a loss of transformation. Not only, you know,
from printed media to semantic markup, also vice versa. It is a loss of transformation. Some
semantics cannot survive it. Uh, keep rendering logic programmable under your control.
And have a look, check out this, uh, you need dog publisher and this repository, which contains
this exact slides and the way how, like, printed documents can be built from them. Thank you very much.
Questions?
Yeah.