modifications to allow compilation of PDF manual #45512

New Issue

Teo Guo Ci · 2015-07-21T13:05:19+02:00

Teo Guo Ci commented

2015-07-21 13:05:19 +02:00

Currently, the PDF manual cannot be created without modifying the sources. This modification changes the sources to allow latex to work on it. The html manual remains unchanged. Below is a list of changes.

patch some files in exts directory to tell sphinx to ignore vimeo and youtube videos.
In image file names, dots that do not represent file extension are replaced with "dot". rst sources are modified and some image files are renamed for this.
some jpg images do not have dimensions. I have added dimensions to them. If this is not done some images will appear oversized in the PDF.

There are 12 gifs in use and all are animated. Those gifs are included in the .tex file, but if you ignore the errors due to those files, the PDF can be compiled.

I am using the following commands to compile the PDF.
sphinx-build -b latex ./manual ./latex
cd latex
pdflatex -interaction nonstopmode blender_manual.tex # 3 times

The tarball of the modified repository and the compiled PDF is available here:
https://www.dropbox.com/sh/rvt5nld2f7fj1p3/AACf14A4q8ESltKvl79vywgWa/blender_manual.pdf?dl=0

blender_docs.tar.gz
svn_status.txt
PDF.diff

Currently, the PDF manual cannot be created without modifying the sources. This modification changes the sources to allow latex to work on it. The html manual remains unchanged. Below is a list of changes. 1. patch some files in exts directory to tell sphinx to ignore vimeo and youtube videos. 2. In image file names, dots that do not represent file extension are replaced with "_dot_". rst sources are modified and some image files are renamed for this. 3. some jpg images do not have dimensions. I have added dimensions to them. If this is not done some images will appear oversized in the PDF. There are 12 gifs in use and all are animated. Those gifs are included in the .tex file, but if you ignore the errors due to those files, the PDF can be compiled. I am using the following commands to compile the PDF. sphinx-build -b latex ./manual ./latex cd latex pdflatex -interaction nonstopmode blender_manual.tex # 3 times The tarball of the modified repository and the compiled PDF is available here: https://www.dropbox.com/sh/rvt5nld2f7fj1p3/AACf14A4q8ESltKvl79vywgWa/blender_manual.pdf?dl=0 [blender_docs.tar.gz](https://archive.blender.org/developer/F211214/blender_docs.tar.gz) [svn_status.txt](https://archive.blender.org/developer/F211197/svn_status.txt) [PDF.diff](https://archive.blender.org/developer/F211198/PDF.diff)

Teo Guo Ci commented

2015-07-21 13:05:19 +02:00

Changed status to: 'Open'

Teo Guo Ci commented

2015-07-21 13:05:19 +02:00

Added subscriber: @guoci

Aaron Carlisle self-assigned this 2015-07-21 17:57:20 +02:00

Aaron Carlisle removed their assignment 2015-07-21 17:57:58 +02:00

Aaron Carlisle commented

2015-07-21 17:57:58 +02:00

Added subscriber: @Blendify

Campbell Barton commented

2015-07-22 02:04:38 +02:00

Added subscriber: @ideasman42

Campbell Barton commented

2015-07-22 02:04:38 +02:00

Thanks for looking into this.

Are you interested to have commit access to the manual to help maintain this?

Details below:

I've committed changes to svn based on your edits:

Mostly the _dot_ in the names were redundant so replaced with -.
Removed strange characters from images.
Set all jpeg's to 72dpi (using exiftools so no resaving).
Added back PDF target to makefile, using -interaction nonstopmode as you suggested.

There are some remaining issues though, so I wouldnt consider the PDF quite finished/complete yet.

Unicode characters aren't showing (we use for cycles support for eg https://www.blender.org/manual/render/cycles/features.html)
There are some parts of the PDF that look odd/wrong still.
The sphinx makefile finishes with an error, Im not sure why:

P241: "make pdf" output

Package longtable Warning: Table widths have changed. Rerun LaTeX.

[1537] (./blender_manual.aux)

Package rerunfilecheck Warning: File `blender_manual.out' has changed.
(rerunfilecheck)                Rerun to get outlines right
(rerunfilecheck)                or use package `bookmark'.


LaTeX Warning: There were undefined references.


LaTeX Warning: Label(s) may have changed. Rerun to get cross-references right.

 )
(see the transcript file for additional information){/usr/share/texmf-dist/font
s/enc/dvips/base/8r.enc}</usr/share/texmf-dist/fonts/type1/public/amsfonts/cm/c
msy10.pfb></usr/share/texmf-dist/fonts/type1/urw/courier/ucrb8a.pfb></usr/share
/texmf-dist/fonts/type1/urw/courier/ucrr8a.pfb></usr/share/texmf-dist/fonts/typ
e1/urw/courier/ucrro8a.pfb></usr/share/texmf-dist/fonts/type1/urw/helvetic/uhvb
8a.pfb></usr/share/texmf-dist/fonts/type1/urw/helvetic/uhvbo8a.pfb></usr/share/
texmf-dist/fonts/type1/urw/helvetic/uhvr8a.pfb></usr/share/texmf-dist/fonts/typ
e1/urw/helvetic/uhvro8a.pfb></usr/share/texmf-dist/fonts/type1/urw/times/utmb8a
.pfb></usr/share/texmf-dist/fonts/type1/urw/times/utmbi8a.pfb></usr/share/texmf
-dist/fonts/type1/urw/times/utmr8a.pfb></usr/share/texmf-dist/fonts/type1/urw/t
imes/utmri8a.pfb>
Output written on blender_manual.pdf (1539 pages, 38938838 bytes).
Transcript written on blender_manual.log.
Makefile:58: recipe for target 'blender_manual.pdf' failed
make- [x]: *** [blender_manual.pdf] Error 1
make- [x]: Leaving directory '/src/manual/build/latex'
Makefile:60: recipe for target 'pdf' failed
make: *** [pdf] Error 2

Thanks for looking into this. Are you interested to have commit access to the manual to help maintain this? Details below: ---- I've committed changes to svn based on your edits: - Mostly the `_dot_` in the names were redundant so replaced with `-`. - Removed strange characters from images. - Set all jpeg's to 72dpi (using exiftools so no resaving). - Added back PDF target to makefile, using `-interaction nonstopmode` as you suggested. There are some remaining issues though, so I wouldnt consider the PDF quite finished/complete yet. - Unicode characters aren't showing (we use for cycles support for eg https://www.blender.org/manual/render/cycles/features.html) - There are some parts of the PDF that look odd/wrong still. - The sphinx makefile finishes with an error, Im not sure why: [P241: "make pdf" output](https://archive.blender.org/developer/P241.txt) ``` Package longtable Warning: Table widths have changed. Rerun LaTeX. [1537] (./blender_manual.aux) Package rerunfilecheck Warning: File `blender_manual.out' has changed. (rerunfilecheck) Rerun to get outlines right (rerunfilecheck) or use package `bookmark'. LaTeX Warning: There were undefined references. LaTeX Warning: Label(s) may have changed. Rerun to get cross-references right. ) (see the transcript file for additional information){/usr/share/texmf-dist/font s/enc/dvips/base/8r.enc}</usr/share/texmf-dist/fonts/type1/public/amsfonts/cm/c msy10.pfb></usr/share/texmf-dist/fonts/type1/urw/courier/ucrb8a.pfb></usr/share /texmf-dist/fonts/type1/urw/courier/ucrr8a.pfb></usr/share/texmf-dist/fonts/typ e1/urw/courier/ucrro8a.pfb></usr/share/texmf-dist/fonts/type1/urw/helvetic/uhvb 8a.pfb></usr/share/texmf-dist/fonts/type1/urw/helvetic/uhvbo8a.pfb></usr/share/ texmf-dist/fonts/type1/urw/helvetic/uhvr8a.pfb></usr/share/texmf-dist/fonts/typ e1/urw/helvetic/uhvro8a.pfb></usr/share/texmf-dist/fonts/type1/urw/times/utmb8a .pfb></usr/share/texmf-dist/fonts/type1/urw/times/utmbi8a.pfb></usr/share/texmf -dist/fonts/type1/urw/times/utmr8a.pfb></usr/share/texmf-dist/fonts/type1/urw/t imes/utmri8a.pfb> Output written on blender_manual.pdf (1539 pages, 38938838 bytes). Transcript written on blender_manual.log. Makefile:58: recipe for target 'blender_manual.pdf' failed make- [x]: *** [blender_manual.pdf] Error 1 make- [x]: Leaving directory '/src/manual/build/latex' Makefile:60: recipe for target 'pdf' failed make: *** [pdf] Error 2 ```

Teo Guo Ci commented

2015-07-22 02:15:34 +02:00

I will like to have commit access and will look into the issues you have mentioned.

Horus R. IceRaven commented

2015-07-22 03:11:12 +02:00

Added subscriber: @HorusIceRaven

Horus R. IceRaven commented

2015-07-22 03:11:13 +02:00

in the PDF if they link to videos on YouTube would be an asset to the visualization.

Marco Ardito commented

2015-07-22 09:19:51 +02:00

Added subscriber: @MarcoArdito

Marco Ardito commented

2015-07-22 09:19:51 +02:00

yes, in my old wiki-to-pdf manual, I did something like that. It was a horrible hackish php script though... the result was not so great... but I hosted also translated versions, here: https://archive.org/details/BlenderWikiPDFManual

I just grabbed the video url using a regexp and then inserted a link instead, pointing to the url.

Thanks so much for bringing pdf manual to blender users...!

yes, in my old wiki-to-pdf manual, I did something like that. It was a horrible hackish php script though... the result was not so great... but I hosted also translated versions, here: https://archive.org/details/BlenderWikiPDFManual I just grabbed the video url using a regexp and then inserted a link instead, pointing to the url. Thanks so much for bringing pdf manual to blender users...!

Teo Guo Ci commented

2015-07-22 16:28:08 +02:00

blender_manual.pdf
I updated the PDF after resolving the issues. Modified the generated tex file only, so nothing to commit. Link to pdf:
https://www.dropbox.com/sh/rvt5nld2f7fj1p3/AADMgOuj57J7N4XfCjsgV50ba

[blender_manual.pdf](https://archive.blender.org/developer/F211613/blender_manual.pdf) I updated the PDF after resolving the issues. Modified the generated tex file only, so nothing to commit. Link to pdf: https://www.dropbox.com/sh/rvt5nld2f7fj1p3/AADMgOuj57J7N4XfCjsgV50ba

Campbell Barton commented

2015-07-27 00:27:41 +02:00

@guoci, manually updating the PDF isn't really an option long term.

Can you note what you changed in the TEX file, so there is a possibility to automate it?

@guoci, manually updating the PDF isn't really an option long term. Can you note what you changed in the TEX file, so there is a possibility to automate it?

Teo Guo Ci commented

2015-07-27 09:32:34 +02:00

@ideasman42
Please see the attached script.
sphinx_tex_pdf.py

@ideasman42 Please see the attached script. [sphinx_tex_pdf.py](https://archive.blender.org/developer/F212866/sphinx_tex_pdf.py)

Sergey Sharybin commented

2015-07-28 11:42:20 +02:00

Added subscriber: @Sergey

Campbell Barton was assigned by Sergey Sharybin

2015-07-28 11:42:32 +02:00

Gyro Gearloose commented

2015-08-02 18:20:43 +02:00

Added subscribers: @teo-3, @pink.vertex

Gyro Gearloose commented

2015-08-02 18:20:43 +02:00

I tested rst2pdf. It did not succeed for the full documentation.

The output of the first chapter looks like this.
It includes bookmarks.

@teo-3 Guo Ci
Maybe parts of it can be done via sphinx extension ?
pdf_modifications.py

I tested [rst2pdf](http://rst2pdf.ralsina.me/handbook.html#sphinx). It did not succeed for the full documentation. The output of the first chapter looks like [this](https://drive.google.com/file/d/0B9MT2HzbSofpcXpkY1ZyelhBcEk/view). It includes bookmarks. ``` ``` @teo-3 Guo Ci Maybe parts of it can be done via sphinx extension ? [pdf_modifications.py](https://archive.blender.org/developer/F217273/pdf_modifications.py)

Teo Guo Ci commented

2015-08-03 04:16:20 +02:00

@pink.vertex
I used sphinx with pdflatex to generate the pdf, not rst2pdf. You can see the generated pdf in the link below.
https://www.dropbox.com/sh/rvt5nld2f7fj1p3/AADMgOuj57J7N4XfCjsgV50ba?dl=0

@pink.vertex I used sphinx with pdflatex to generate the pdf, not rst2pdf. You can see the generated pdf in the link below. https://www.dropbox.com/sh/rvt5nld2f7fj1p3/AADMgOuj57J7N4XfCjsgV50ba?dl=0

Gyro Gearloose commented

2015-08-03 15:41:55 +02:00

@guoci

I know. It was just a test to see wether rst2pdf would work out of the box and wether the result would look better.

Modified the .tex file generation via sphinx extension and pdflatex exits without errors now (someone else might test this). I did not deal with the unicode characters.

Only intended to build without errors. There might be some things missing now in the final result because they don't work with latex, like figures in figures (i.e. modifiers/simulate/particle_instance.rst)

pdf_modifications_only_translator.py

@guoci I know. It was just a test to see wether rst2pdf would work out of the box and wether the result would look better. Modified the .tex file generation via sphinx extension and pdflatex exits without errors now (someone else might test this). I did not deal with the unicode characters. Only intended to build without errors. There might be some things missing now in the final result because they don't work with latex, like figures in figures (i.e. `modifiers/simulate/particle_instance.rst`) [pdf_modifications_only_translator.py](https://archive.blender.org/developer/F218151/pdf_modifications_only_translator.py)

Gyro Gearloose commented

2015-08-04 16:16:51 +02:00

It turns out replacing the unicode characters in the extension is rather easy.

from sphinx.util.texescape import tex_escape_map, tex_replace_map
#...

def setup_latex(app):
    if app.builder.name == "latex":
    #...

        substitutes = "\\ding{51}", "\\ding{51}", "\\ding{55}"
        for char, substitute in zip("✓✔✗", substitutes):
            tex_escape_map[ord(char)] = substitute
            tex_replace_map[ord(char)] = '_'

Add the package in conf.py

# Additional stuff for the LaTeX preamble.
  'preamble': '\\usepackage[left=0.75in, right=0.75in, top=1.0in, bottom=1.0in]{geometry}\n'
              '\\usepackage{pifont}',

Tables with images get screwed. Some who knows latex better than me might fix this.

It turns out replacing the unicode characters in the extension is rather easy. ``` from sphinx.util.texescape import tex_escape_map, tex_replace_map #... def setup_latex(app): if app.builder.name == "latex": #... substitutes = "\\ding{51}", "\\ding{51}", "\\ding{55}" for char, substitute in zip("✓✔✗", substitutes): tex_escape_map[ord(char)] = substitute tex_replace_map[ord(char)] = '_' ``` Add the package in `conf.py` ``` # Additional stuff for the LaTeX preamble. 'preamble': '\\usepackage[left=0.75in, right=0.75in, top=1.0in, bottom=1.0in]{geometry}\n' '\\usepackage{pifont}', ``` Tables with images get screwed. Some who knows latex better than me might fix this.

Aaron Carlisle commented

2015-08-04 16:36:10 +02:00

Removed subscriber: @Blendify

Campbell Barton commented

2015-08-07 03:26:57 +02:00

Did anyone look into having images show in tables?

As far as I can see this is a show-stopper, since we have this reasonably often.
We could always remove, but having images side-by-side is quite reasonable thing to do.

If theres no solution for that, we can consider PDF generation incomplete ~ (report to sphinx developers, and only look into it again if they resolve the problem).

Also, if theres fixes to existing PDF generation, please provide a diff instead of code-snippets. eg: svn diff > pdf.patch, they can be attached or linked to.

Did anyone look into having images show in tables? As far as I can see this is a show-stopper, since we have this reasonably often. We could always remove, but having images side-by-side is quite reasonable thing to do. If theres no solution for that, we can consider PDF generation incomplete ~ *(report to sphinx developers, and only look into it again if they resolve the problem).* Also, if theres fixes to existing PDF generation, please provide a diff instead of code-snippets. eg: `svn diff > pdf.patch`, they can be attached or linked to.

Teo Guo Ci commented

2015-08-07 03:54:49 +02:00

@ideasman42
I used the script attached to modify the sphinx generated .tex file before compiling with pdflatex. The images in tables do show up in the pdf.

sphinx_tex_pdf.py

@ideasman42 I used the script attached to modify the sphinx generated .tex file before compiling with pdflatex. The images in tables do show up in the pdf. [sphinx_tex_pdf.py](https://archive.blender.org/developer/F212866/sphinx_tex_pdf.py)

Gyro Gearloose commented

2015-08-07 07:52:34 +02:00

manual_pdf.diff

make pdf still exits with an error . However

cd ./build/latex
pdflatex -interaction=nonstopmode blender_manual.tex

does not.

Also in the log file blender_manual.log there are no more errors. Just warnings about floats too large and undefined references.
So the makefile might need some fix?

I removed the entries from the index.rst file because these are already defined in the subfolder below which caused duplication in the .tex file.

This extension currently splits rows in image tables with figures into two rows, where the first row contains the images, second one contains the captions.

Does not look that good and might be improved by using a latex package like subfig or subcaptions. Did not look into that, yet.

[manual_pdf.diff](https://archive.blender.org/developer/F221165/manual_pdf.diff) `make pdf` still exits with an error . However ``` cd ./build/latex pdflatex -interaction=nonstopmode blender_manual.tex ``` does not. Also in the log file `blender_manual.log` there are no more errors. Just warnings about floats too large and undefined references. So the makefile might need some fix? I removed the entries from the `index.rst` file because these are already defined in the subfolder below which caused duplication in the .tex file. This extension currently splits rows in image tables with figures into two rows, where the first row contains the images, second one contains the captions. Does not look that good and might be improved by using a latex package like subfig or subcaptions. Did not look into that, yet.

Aaron Carlisle commented

2016-05-01 01:13:29 +02:00

Added subscriber: @Blendify