Page MenuHome

Manual Versioning
Closed, ResolvedPublic

Description

Since we have moved to sphinx we no longer are versioning the manual

Things That Need to be DONE

  1. Setup symlinks for current and trunk. Current should lead to the latest release and I think that for trunk it should just be then same as things are now.
  2. Update the context-sensitive manual access python file. This means that this should happen right before a release. (This could happen for a bug release e.g. 2.78a)

Questions

  • Should we create a new manual version for every release?
  • Who would be responsible for uploading versions?

Event Timeline

+1 for releasing a separate version of the manual for each blender release.

From a users POV, I found it rather disheartening when, after updating many old wiki links in the wild, the shiny new manual links also started breaking too due to re-organizations on the manual side.

I vote for a policy of maintaining "permalinks", where once a page is put at a URL, it doesn't leave it.

One way we could go about this is by keeping the organization of pages fixed once a version of the manual is released (which would happen at the same time as the blender release?).
The manual docs for the next version of blender (as yet unreleased) would be the ones we work on, and we wouldn't touch the docs for the current version, except for crucial in-page edits (no link-breaking organization changes, or documenting features not in that version).

AFAIK the python API docs handle this in a similar fashion.

Bastien Montagne (mont29) changed Type from Bug to Design.Sep 6 2016, 12:00 PM

One thing I just realized is that I said that https://www.blender.org/manual/ should lead to two different things:

  1. Idea is that is should be treated as Wikipedia home page.
  2. It is the update to date manual with the latest changes from trunk

Which leads me the think of a third option:

  1. The two ideas above get merged so that links to other versions of the manual are found on the home page (https://www.blender.org/manual/).

See also pythons solution:

https://docs.python.org

Unless we get search to work massively better, I'm in favor of having the content tree on the landing page (python style).
Perhaps language and version drop-downs could be perpetually in the top corner somewhere,
and in addition, perhaps we may want to make them more prominent on the landing page (wikipedia style).

To maintain permalinks, I suggest that the "version" drop-down be set to the current release of blender by default,
with an option in the drop-down to switch to the currently in-development version (which would also be the version of the docs being actively developed).

Docs for past releases should have a notice at the top every page stating that this is the old documentation,
and link to the equivalent page in the new documentation (if there is one, else maybe link to a search with the old page's title).
If we wanted to get fancy and complicated, perhaps we could auto-redirect people from old doc pages if the modern one hasn't changed..
But that'll probably create more problems than it solves.


A couple questions..

  • Why put language codes as subdomains on blender.org? Having those at subdomain level implies that blender.org also has translations, which afaik it doesn't have.

    What about something like blender.org/manual/en/2.78/modeling/modifiers/generate/remesh.html, where the "en" and "2.78" components are optional and will get redirected to defaults if omitted?
  • What's the context-sensitive manual access python file?

I rather want to keep the rate of new versions the same as the wiki before:
Each major release (second digit). Next would be 2.8.
When starting a new version the old should be in a more/less complete state (added features).

Setting up the versioning is not trivial -
It's going to take 1+1/2 week full time.

An alternative to py's breadcrumb version picker could be the readthedoc one:

http://blender-manual.readthedocs.io/en/testing/contents.html
Sidebar bottom left corner.

There is no need for a language picker portal since there aren't enough different languages to justify it. Neither a landing page.
Priority should be on current and English.

Please, don't forget about translations! People talks not only English, and adding localization feature «sometime later» will be more difficult. Besides, the localization of documentation is already have (not full, but still) and actively translating :).

Quick change of language would be very useful and it should be possible to do from any docs page.

By the way, why not use Read the Docs for docs hosting? It has out of box versioning and localization support.

So it looks like the idea is to not have a landing page and blender.org/manual would default to the current release?

I have a question about "rna_manual_reference.py".

Why the specific URL's is stored in the program code?
Why not store something like "blender.org/manual/?id=xxx" and redirect on server side.

https://www.blender.org/manual/?rna=bpy.types.menu

So the docs structure will be independent of the code.

@Yevgeny Makarov (jenkm) That is possible and I have already been looking into it.
This will be possible by 2.8.

I investigated this a bit further:
there are two options:

  • rtd: server side, automatic
  • pydoc: client side, manual

rtd:
detect tagged versions > inject this list into template context > build with template
The server has to rebuild all when a version is added (?)

pydoc:
build with template that adds a placeholder > run js-script that replaces the placeholder
The script's list ( var all-versions = { '3.6': 'dev (3.6)', ...} ) needs to be manually updated for each new version.

The versioning template can either:
Override rtd theme's version.html (by ~simply copying & removing the if) or
extend layout.html to add a custom design e.g. under the logo.

résumé:
The js version seams to be the most practical: it's simple, not so error-prone.
It adds a tiny bit manual work, but it has to be tagged and a config. update is needed anyway.


Links:

theme:
versions.html https://github.com/snide/sphinx_rtd_theme/blob/master/sphinx_rtd_theme/versions.html

rtd:
context: https://github.com/rtfd/readthedocs.org/blob/master/readthedocs/doc_builder/templates/doc_builder/conf.py.tmpl

pydoc:
js: docs > tools > static > version_switch.js [68 lines]
placeholder #8: https://hg.python.org/cpython/file/tip/Doc/tools/templates/layout.html

I agree that we should do something like pydoc. I think we make version picker badge like rtd
which include two separate dropdown/up menus that also have search functionality (at least for language)
Here is a quick mockup:

ToDo:

  1. Tagging

Currently the manual is tagged: 2.78, dev
https://svn.blender.org/svnroot/bf-manual/tags/

In my opinion it should be: 2.77, release, dev
That's minus one (can be done by: it's like renaming a folder + config. file)
Because the work on the released version is finished when the new release comes out.

Dev branch:
I think a dev tag could not only be used for testing and demos (like the wiki sandbox) but also for developers to allow cooperation on user docs e.g. OpenHMD. That would have the major benefit that content is more likely fit in better. That's especially important for docs where the matter is so complex that only the dev could provide it or hardware is needed ;).

  1. Building the tags

For testing

  1. Version switch

40% is done (10% for the language problem, 50% for design and polish)

Hi,

I recommend a simpler approach:

  • 2.77, 2.78
  • dev

Let's not use a 'release' tag. If that is used in links for SO answers (for example) it will point to something that changes overtime, which will make reading support answers harder.

Regarding how to do this in practice:

  • a script needs to be run on a new release to point old docs to the latest version (again, a hardcoded number)
  • we need to agree on a design for the language/version selector (this is for a separate thread)

I would like to hear agreement on the versioning schema first.

It's possible to have both: a numbered build and a release build (100% copy) on a new B. release the release tag gets a new clone.
Isn't that how it's handled in the api (current).

The mapping script:
It's complicated/time consuming to write such script. IMO it has no priority.
Especially if there's no script available that just has to be customized.
I would declare it optional.

There is also: sphinxcontrib-versioning , but it doesn't support internationalization (languages). It would do the first method "auto build" as described above. I can't tell if it works with SVN, then it maybe can be used for the API.

@Tobias Heinke (TobiasH) are you available to help with the java scripting again? We are ready to work on this. I tried to finish it but its not working.

@Aaron Carlisle (Blendify) , https://www.blender.org/manual/ currently links to the 2.80 manual. Please keep 2.79 as the default until 2.80 is released?

@Brecht Van Lommel (brecht) I poked @Dan McGrath (dmcgrath) over email but will poke him again here to address this.


There are three redirects that I would like to have changed:

https://docs.blender.org/manual/ --> https://docs.blender.org/manual/en/latest/ current: https://docs.blender.org/manual/en/dev/
https://blender.org/manual/ --> https://docs.blender.org/manual/en/latest/ current: https://docs.blender.org/manual/en/dev/
https://docs.blender.org/manual/<lang>/ --> https://docs.blender.org/manual/<lang>/latest current: https://docs.blender.org/manual/en/dev/

I think we are ready to upload final versions of all 2.79, for the API we rsync the files to the server. Do we want to do that for the manual? Or was the FTP transfer good enough?

@Brecht Van Lommel (brecht) Actually, it always linked to the head of git master, which used to say 2.79. It's only because it naturally changed that it now says 2.80. Anyway, there might be some issue here since we do a regex that matches urls to redirect, but only en and fr languages were given to me, which would be a bit of hassle to try rip apart the regex's and config entries that match the languages to deal with the fact that Aaron only uploaded partial. So TL;DR: he only uploaded 2.79 tar balls, so there is nothing to "keep" at 2.79 since it was a rolling target in the first place and we need to do a final one time build on the manual.

Ok, so as I understand it dev/ started including 2.80 changes too early, before a complete version of 2.79 with all languages was uploaded. So next time we should make sure the latest release docs are in order first.

And to fix things now @Aaron Carlisle (Blendify) will upload a complete version of 2.79 that @Dan McGrath (dmcgrath) will build and then he will change the blender.org redirects to point to that.

For the new versions to appear in the version selector of old versions, will they all need to be rebuilt whenever a new version is added?

Can someone document what the intended directory layout will be? There was discussion in this task but no conclusion.

From what I gather the current state is:

  • What we link to from blender.org: dev/ (wrong, should become latest/)
  • Canonical version for search engines: latest/
  • Under development for next release: dev/
  • What we link to from inside Blender: dev/ (wrong, should become 2.xx/ for official releases and dev/ only for daily builds)
  • Where you end up when using the version selector: 2.xx/
  • What users link to or bookmark: any of dev/, latest/ and 2.xx/ depending on how the user got to the page. (not ideal)

I think that dev/ is actually correct, isn't it? It tracks the master of git, which if anything should have it's git commit revision in the actual sphinx files changed to be the git revision or something perhaps? I can definitely foresee a problem is search engines see multiple copies of 2.xx in the page content.

If I understand correctly, latest/ is meant to point to the latest official release?

As for the redirects, I realise now that I perhaps didn't fully explain the issue (in case there is some confusion). Currently we have an Apache configuration that (approximately) uses this logic:

  • If they visit /manual/, redirect them to /manual/en/ (default to english)
  • If they visit /manual/<lang>/ (ie: just hit step above), redirect them to /manual/<lang>/dev/ (latest commit for that language)

It's not terribly complicated, logic wise, but the redirect itself uses the languages in an all or nothing regex that can't say "do this for just en and fr for now", and there are also some redirects in there for zh.(cn|tw) -> zh-han(s|t) to further made the config a headache.

As for the url's, I wonder if instead we should be redirecting people that land on latest/ to an actual number, like 2.xx/ so they end up book marking the a specific version, rather than a concept (such as the latest version always, which they would get if they bookmark latest/ and we don't send them elsewhere).

I mention this because a bit of an issue might that we now have to constantly update that redirect over time. Not a huge deal, but it's not automatic, and a human would have to remember to include this step as part of the release cycle. Also, naming conventions would need to be decided on for rc releases and such.

Anyway, I hope this sheds some light on the issue. Perhaps it makes sense to also paste the current (relevant) config for redirects here so that others may offer a patch to add? Open to suggestions on the config. It can surely be refactored at least something, but gets tricky to debug live.

# Redirect everything to the canonical name
RewriteEngine On
RewriteCond "%{HTTP_HOST}" "!^docs.blender.org$"
RewriteRule "^/(.*)" "https://docs.blender.org/$1" [R=301,L]

# Redirect /manual/ to /manual/en/
RewriteRule "^/manual/$" "/manual/en/" [R=301,L]

# Redirect languages to the latest dev/
RewriteRule "^/manual/(en|de|es|fi|fr|it|ja|ko|nb|pt|ru|sr|uk|zh-hans|zh-hant)(/)?$" "/manual/$1/dev/" [R=301,L]

# Redirect everything else to /manual/en/dev/
RewriteCond "%{REQUEST_URI}" "!^/manual/(en|de|es|fi|fr|it|ja|ko|nb|pt|ru|sr|uk|zh-hans|zh-hant)"
RewriteRule "^/manual/(.*)" "/manual/en/dev/$1" [R=301,L]

# Redirect Chinese urls for rename
RewriteRule "^/manual/zh\.cn/(.*)" "/manual/zh-hans/$1" [R=301,L]
RewriteRule "^/manual/zh\.tw/(.*)" "/manual/zh-hant/$1" [R=301,L]

Redirecting to specific 2.xx versions is fine with me. Though we should make it clear which version is still under development, otherwise users might just go to the latest version without realising it has not been released yet and their Blender does not contain those features. So it the version selector and title it could be called e.g. "2.80 (dev)".

I think uploading the version for each release is a manual process anyway, so as long as we document all the steps that are needed it may be ok to update the redirects as well?

I don't think we need separate docs for "rc" or "alpha" releases, just one 2.xx/ version should be enough.

The only thing I am not sure about is how this will work with search engines. We have the problem now for the Python API that googling gives you random versions, which is pretty bad. In the manual each page contains <link rel="canonical" "https://docs.blender.org/manual/en/latest/...html"> which ensure only a single version shows up. I hope that still works if latest/ is merely a redirect, I guess it does.

Sorry guys for the late reply I had some unrelated issues come up that I had to deal with.

@Brecht Van Lommel (brecht) Actually, it always linked to the head of git master, which used to say 2.79. It's only because it naturally changed that it now says 2.80. Anyway, there might be some issue here since we do a regex that matches urls to redirect, but only en and fr languages were given to me, which would be a bit of hassle to try rip apart the regex's and config entries that match the languages to deal with the fact that Aaron only uploaded partial. So TL;DR: he only uploaded 2.79 tar balls, so there is nothing to "keep" at 2.79 since it was a rolling target in the first place and we need to do a final one time build on the manual.

I will compile all official "2.79" docs shortly (today/tomorrow if every goes ok)

Ok, so as I understand it dev/ started including 2.80 changes too early, before a complete version of 2.79 with all languages was uploaded. So next time we should make sure the latest release docs are in order first.
And to fix things now @Aaron Carlisle (Blendify) will upload a complete version of 2.79 that @Dan McGrath (dmcgrath) will build and then he will change the blender.org redirects to point to that.
For the new versions to appear in the version selector of old versions, will they all need to be rebuilt whenever a new version is added?

Yes dev was changed to "2.8" but only the version number changed to test how the version switcher works. No 2.8 features have been documented yet.

Yes see above.

For new versions old version will not need to be recompiled as the version menu to dynamic and grabs the information from a .json file found here:
https://docs.blender.org/manual/en/dev/versions.json

Can someone document what the intended directory layout will be? There was discussion in this task but no conclusion.
From what I gather the current state is:

  • What we link to from blender.org: dev/ (wrong, should become latest/)
  • Canonical version for search engines: latest/
  • Under development for next release: dev/
  • What we link to from inside Blender: dev/ (wrong, should become 2.xx/ for official releases and dev/ only for daily builds)
  • Where you end up when using the version selector: 2.xx/
  • What users link to or bookmark: any of dev/, latest/ and 2.xx/ depending on how the user got to the page. (not ideal)

Everything you say above is correct. Including the things that you say are wrong, I am working on getting these fixed.

Re the last point yes this is not ideal but I cannot think of a better solution. We can add a warning message like on readthedocs: https://solidity.readthedocs.io/en/v0.4.0/index.html (this is dynamic).

Thanks for the info.

Re the last point yes this is not ideal but I cannot think of a better solution. We can add a warning message like on readthedocs: https://solidity.readthedocs.io/en/v0.4.0/index.html (this is dynamic).

Do you have any objects to redirects from latest/ to 2.79/ and dev/ to 2.80/? That's the solution we are proposing to reduce the number of URLs that can be linked to, and to ensure links stay valid even if content changes.

A prominent link to the latest version would be useful as well, but it solves a different problem?

@Brecht Van Lommel (brecht) no originally I thought we would just sym link the versions, this is what we do for the API. However, having less urls out there would be a benefit. The only downside like Dan said was that this needs to be changed after every release. Also I am not sure how the chronical url would behave for search engines if it is a redirect. An other thing is that with it being a symlink that makes it more of a url that people will use and share. It would be better SEO has the latest URL would be linked to more. Then again a downside would be that link to the latest URL does not garantee that those docs do exist if the symlink changes. There are pros and cons of each but we need to choose one that we think would work better for us.

A prominent link to the latest version would be useful as well, but it solves a different problem?

It solves the issue of people being linked to an older version, them being able to realize this, and go to the updated version.

Also I am not sure how the chronical url would behave for search engines if it is a redirect. An other thing is that with it being a symlink that makes it more of a url that people will use and share. It would be better SEO has the latest URL would be linked to more.

I expect that links to different pages with the same canonical URL will count towards the search ranking of the one canonical URL, but I'm not sure.

It solves the issue of people being linked to an older version, them being able to realize this, and go to the updated version.

Right, and that's a problem worth solving too.

I have been busy, sorry. I will try to work on this this weekend.

I saw there were changes made, but it still seems a work in progress. What is the status? If you don't have time to get things ready soon, what do we need to do still?

I've now done a number of changes:

  • Set redirects on the server now to go to latest/ by default, instead of dev/.
  • Copied the dev/ folders on the server to 2.79/ for every language, rather than just english.
  • Set up latest/ and 2.80/ symlinks.
  • Bumped the version to 2.80 in SVN.
  • Fixed broken links in version menu.

I left a freeze_version.sh on the server that does this copying and symlinking.

I think this is the main stuff needed to start writing 2.80 docs in SVN, without it interfering with the 2.79 manual.

@Brecht Van Lommel (brecht) Thats about it, the last thing is trying to fix the added warning message.

I started this in rBM4457 but it does not work yet. I am not very proficient in JS and cant figure out what I did wrong.

Once this is done, we should tag releases for both the manual and the translations svn repositories.

Also the current language switch is broken due to a missing comma in the language tuple.

Aaron Carlisle (Blendify) closed this task as Resolved.Jul 7 2019, 12:00 AM
Aaron Carlisle (Blendify) claimed this task.