Add support for 'changelog' generation in API documentation automated process on the buildbot. #97663

New Issue

James Tomkinson · 2022-04-27T18:24:45+02:00

James Tomkinson commented

2022-04-27 18:24:45 +02:00

The 'changelog' page of the Blender API documetation used to be generated manually as a separate, independent step, prior to the actual API doc generation.

Now that the API doc is generated automatically on the buildbots as part of the build process, the change log generation needs to be integrated there as well.

After talking with @Arnd, here is the proposal:

Store the API dump of a release together with its API doc on the server.
- Switch from a dict-based py file storage to a JSON format, to avoid having potentially executable files on the server.
- Have all available API dumps listed in another index JSON file at the root of the API doc directory.
As part of the API doc generation tool (sphinx_doc_gen.py):
- Download the index file
- Find the 'most recent' previous version available
- Download found dump JSON file
- Generate JSON dump file for current blender version, and update the index accordingly
Generate the changelog page based on the diff between the two API dump files

ORIGINAL REPORT:

blender_python_reference/change_log shows changes from 2.83 to 2.90 in 3.1 / 3.2 release

System Information
Operating system: Windows-10-10.0.19043-SP0 64 Bits
Graphics card: NVIDIA GeForce GTX 1050/PCIe/SSE2 NVIDIA Corporation 4.5.0 NVIDIA 511.65

Blender Version
Broken: version: 3.1.2, branch: master, commit date: 2022-03-31 17:40, hash: blender/blender@cc66d1020c
Worked: (newest version of Blender that worked as expected)

Short description of error
It appears that the python change-log information is not being updated.

Exact steps for others to reproduce the error
Go to Help >> Python API Reference
a web page is displayed, click on Change Log link
taken to https://docs.blender.org/api/3.1/change_log.html
In the title it says:

Blender API Change Log
2.83 to 2.90

The 'changelog' page of the Blender API documetation used to be generated manually as a separate, independent step, prior to the actual API doc generation. Now that the API doc is generated automatically on the buildbots as part of the build process, the change log generation needs to be integrated there as well. After talking with @Arnd, here is the proposal: - [x] Store the API dump of a release together with its API doc on the server. - [x] Switch from a dict-based `py` file storage to a JSON format, to avoid having potentially executable files on the server. - [x] Have all available API dumps listed in another index JSON file at the root of the API doc directory. - [x] As part of the API doc generation tool (`sphinx_doc_gen.py`): - Download the index file - Find the 'most recent' previous version available - Download found dump JSON file - Generate JSON dump file for current blender version, and update the index accordingly # Generate the changelog page based on the diff between the two API dump files ------------ **ORIGINAL REPORT:** # blender_python_reference/change_log shows changes from 2.83 to 2.90 in 3.1 / 3.2 release **System Information** Operating system: Windows-10-10.0.19043-SP0 64 Bits Graphics card: NVIDIA GeForce GTX 1050/PCIe/SSE2 NVIDIA Corporation 4.5.0 NVIDIA 511.65 **Blender Version** Broken: version: 3.1.2, branch: master, commit date: 2022-03-31 17:40, hash: `blender/blender@cc66d1020c` Worked: (newest version of Blender that worked as expected) **Short description of error** It appears that the python change-log information is not being updated. **Exact steps for others to reproduce the error** Go to Help >> Python API Reference a web page is displayed, click on Change Log link taken to https://docs.blender.org/api/3.1/change_log.html In the title it says: > Blender API Change Log > 2.83 to 2.90

James Tomkinson commented

2022-04-27 18:24:45 +02:00

Added subscriber: @jamestomk

blender-admin commented

2022-04-27 18:24:45 +02:00

blender/documentation#98757 was marked as duplicate of this issue

Philipp Oeser commented

2022-04-28 09:34:00 +02:00

Added subscriber: @lichtwerk

Philipp Oeser commented

2022-04-28 09:34:00 +02:00

Can confirm.

Philipp Oeser commented

2022-04-28 09:34:10 +02:00

Changed status from 'Needs Triage' to: 'Confirmed'

Philipp Oeser commented

2022-05-05 16:18:31 +02:00

Added subscribers: @mont29, @ideasman42

Philipp Oeser commented

2022-05-05 16:18:31 +02:00

@ideasman42 , @mont29 : do you know if this was done by hand?

Campbell Barton commented

2022-05-06 01:58:31 +02:00

The changelog generator needs to be run manually. The main reason this wasn't automated is you need two different Blender versions of generate output, although if we kept the previous releases output in a fixed location we could compare against that which will work as long as the file format doesn't change.

Philipp Oeser changed title from ~~blender_python_reference/change_log shows changes from 2.83 to 2.90 in 3.1.0 release~~ to blender_python_reference/change_log shows changes from 2.83 to 2.90 in 3.1 / 3.2 release

2022-06-10 08:23:46 +02:00

Philipp Oeser commented

2022-06-10 08:24:41 +02:00

Added subscriber: @EAW

Philipp Oeser commented

2022-06-10 08:25:50 +02:00

Will raise prio here since this is not a nice thing to have.

Danny McGrath commented

2022-06-10 19:04:37 +02:00

Added subscribers: @Arnd, @dmcgrath

Danny McGrath commented

2022-06-10 19:04:37 +02:00

I believe this was intentional, at least temporarily.

@Arnd and I were speaking in chat on matters relating to a recent buildbot problem, when he mentioned:

short-term: i've changed the 'stable doc' to point to 3.2
but the /api/current is still pointing to 3.1.2 for now; so there's something there

I expect that at some point he will move it back to the proper versions, after he has got to the bottom of the build issue he was looking into.

I believe this was intentional, at least temporarily. @Arnd and I were speaking in chat on matters relating to a recent buildbot problem, when he mentioned: > short-term: i've changed the 'stable doc' to point to 3.2 > but the /api/current is still pointing to 3.1.2 for now; so there's something there I expect that at some point he will move it back to the proper versions, after he has got to the bottom of the build issue he was looking into.

James Tomkinson commented

2022-06-12 18:55:10 +02:00

Guess what: the python manual suffers from the same deficiency.

Arnd Marijnissen commented

2022-06-13 14:47:48 +02:00

This , at first glance, does not seem to be a buildbot related thing. The files that are produced in the build-steps are being delivered properly.
The remark above was related to the failure of the 3.2-upload process to adapt the 'current' and 'latest' links in docs. That's been fixed.

The Python changelog not being updated is something that, to me, looks to be part of the python-api-build-process itself. I'll look to see if I can see where exactly that is happening.
My intuition tells me it's not a buildbot issue.

This , at first glance, does not seem to be a buildbot related thing. The files that are produced in the build-steps are being delivered properly. The remark above was related to the failure of the 3.2-upload process to adapt the 'current' and 'latest' links in docs. That's been fixed. The Python changelog not being updated is something that, to me, looks to be part of the python-api-build-process itself. I'll look to see if I can see where exactly that is happening. My intuition tells me it's not a buildbot issue.

Bastien Montagne commented

2022-06-13 16:09:48 +02:00

Think this was indeed never integrated in the automated API doc generation process.... Will look into how to do that locally first, then we can work with @Arnd into integrating it to the buildbot system.

Bastien Montagne changed title from ~~blender_python_reference/change_log shows changes from 2.83 to 2.90 in 3.1 / 3.2 release~~ to Add support for 'changelog' generation in API documentation automated process on the buildbot.

2022-06-13 17:45:28 +02:00

Aaron Carlisle commented

2022-06-14 00:42:46 +02:00

Added subscriber: @Blendify

Aaron Carlisle self-assigned this 2022-06-14 00:42:46 +02:00

Aaron Carlisle commented

2022-06-14 00:42:46 +02:00

I will be looking into this and discuss a solution with Cambell and Arnd.

Aaron Carlisle was unassigned by Bastien Montagne

2022-06-14 09:50:33 +02:00

Bastien Montagne self-assigned this 2022-06-14 09:50:33 +02:00

Bastien Montagne commented