hi, were you still developing this? I think it's great concept but seems hard to set up? As 2.8 beta has been released, we could look at something like this again at some stage. It may be better to archive this task and start a new task when it's updated to Blender 2.8 series. One note, Easy of use and installation set up should be all behind the scenes or via check boxes in the addons preferences. I think matpi? did a similar one also?

@Tuomo, note box removed.

Glad that you like the result! Cheers for ye olde wiki! :-)

In T48096#611847, @Brecht Van Lommel (brecht) wrote:

Let's consider this resolved then.

I have left a copy of the tarball in /root/en.blender.org.tar.bz2. Feel free to grab a copy (~5.5G). This contains the vhost docroot, the old txt only dumps, and the database dump.

I wouldn't know what to do with that file, I just imagined you had someplace to archive these kinds of things.

Let's consider this resolved then.

I forgot to mention that the old vhost docroot files have be entirely deleted, as well as the original database. The old docroot is a mere empty skeleton, needed to satisfy the Apache docroot path, as well as provide a means for Certbot (Lets Encrypt) to acquire its certificates. Once the snapshots rotate enough, the space from the old files will be reclaimed.

Good to hear.

Redirects are working well.

In T48096#611638, @Francesco Siddi (fsiddi) wrote:

Great! I guess that the 2 remaining points (Utilities and Delete the old read-only PHP wiki) can be wrapped up as well?
After that, I propose to close this task and move any other todo to separate topics, tagged as Documentation.

Great! I guess that the 2 remaining points (Utilities and Delete the old read-only PHP wiki) can be wrapped up as well?
After that, I propose to close this task and move any other todo to separate topics, tagged as Documentation.

In T48096#611447, @Francesco Siddi (fsiddi) wrote:

Thanks @Tuomo Keskitalo (tkeskita)! And thanks @Brecht Van Lommel (brecht).
From a quick look the archived wiki seems to be running well in all its legacy glory.
Looking forward to the last steps that Dan will perform.

Thanks @Tuomo Keskitalo (tkeskita)! And thanks @Brecht Van Lommel (brecht).
From a quick look the archived wiki seems to be running well in all its legacy glory.
Looking forward to the last steps that Dan will perform.

In T48096#610410, @Brecht Van Lommel (brecht) wrote:

Now it's just a matter of setting up a redirect from en.blender.org to archive.blender.org/wiki and taking the old wiki offline (I guess we keep a backup).

@Dan McGrath (dmcgrath), can you do that?

It looks great, I found no problems. Thanks a lot for doing this.

OK, I'm glad to inform you that the static old wiki baking is finished. The current version is uploaded temporarily to http://tkeskita.kapsi.fi/temp/en.blender.org and download package is available in http://tkeskita.kapsi.fi/temp

I would recommend having a clear message stating that the wiki contains legacy information and to visit wiki.blender.org for the active wiki documentation. Something along the lines of what @nBurn (nBurn) suggested.

Third retrieve is proceeding (slowly), and in ~4 days it should be finished. At that time (before running html processing for all pages) I need feedback about suggestion in T48096#594983 by @nBurn (nBurn) to change warning text (and add link to an explanatory page). @Inês Almeida (brita_), @Francesco Siddi (fsiddi), @Aaron Carlisle (Blendify), anyone: do you agree? What URL to add for explanatory page? Some page in new wiki? Or better to leave warning as it is now?

It looks like old Mediawiki doesn't want to work nicely with php7. Also there's no guarantee that the result will look anything like the current one, so I guess its gonna be a new 10 d crawl..

Hi, I'm now contemplating to do another full retrieve, so this would be a perfect time to make decisions about suggestions from @nBurn (nBurn) in T48096#594983. Anyone else also want those changes?

Hi Henrique, we do not handle Blender Network issues here on developer.blender.org.
I forwarded the issue to support at blendernetwork.org and we will be in touch via email.

Yes, the only special pages currently included are the edit and history pages. Technically, all those specials (non-existing parent page, what links here, recent user contributions for User:X pages) can be added, if someone else also seconds this motion? "What links here" and "recent user contributions" could be retrieved to e.g. links.html and contributions.html located under page folders. I don't think they would add too much to archive size, it would just take me time to do it.

There seems to be some weirdness going on if wiki user's created subpages without a base page.

I would keep all the deleted pages and their redirects. This would avoid broken links.

I would keep all the deleted pages and their redirects. This would avoid broken links.
I don't think that the asccii character issue is is a good compromise as this will result in broken links? Or maybe I misunderstood something.

Ok, new version of archived wiki is available temporarily for review at http://tkeskita.kapsi.fi/temp/en.blender.org/

• Page list available in text and html formats. Pages now include also deleted pages and redirections to deleted pages. Page count is 53173 53209 out of 53221, and all page directories listed in the XML dump have index.html, so I think it is now pretty much complete.
• Includes index.html, edit.html and history.html pages for each wiki page. History page lists date and username of 50 latest edits.
• Page names may contain underscores instead of spaces, and they should both be found. This was done by creating directory symlinks like "Release_Notes" that point to "Release Notes". Is this OK?
• All links to en.blender.org in pages have been converted into relative paths, so this can be freely placed in any directory and links should work. Since the whole script size is now 350 lines, the current bash script is attached here (updated version 2019-01-04):

Agree, I wouldn't worry too much about fixing every broken link. It's nice if it doesn't take too much time, but for an archived website it's ok if only the basics are functional.

I think proposals and plans can be on subpages of the relevant pages of code documentation. For example for Cycles we have roadmap and todo subpages.

I wouldn't worry about saving the full edit history. I'm not sure how you could preserve that without a significant increase to the total size and file count of the wiki archive. It might be helpful to see who the last 50 or 100 people that edited the page are (and maybe the first 50 edits), but if someone needed more info than that they could download the wiki data dump.

It would be good to decide on the best place for work-in-progress documents and proposals.

I've been wondering about history pages. After pages are static it will not be possible to do comparisons between versions, so you will only see who edited the page last and when. Is it still interesting to save history page? @Inês Almeida (brita_)?

In T48096#586329, @Tuomo Keskitalo (tkeskita) wrote:

In T48096#585785, @LazyDodo (LazyDodo) wrote:

I was browsing the scraped copy last week, and bumped into a searchbox somewhere, we'd probably have to get rid of those otherwise users will use them and make tickets that it is not working.

I guess you mean the quick search on lower left corner? Good idea! I'll also remove the "Page" menu from top right and "Wiki" menu from bottom right.

In T48096#585634, @Tuomo Keskitalo (tkeskita) wrote:

1. The old wiki contains deleted pages, such as this, which redirects to here which is deleted. wget does not want to download such pages. Is it OK to leave these pages out, or should I try to find some way to get also them in? My script is collecting a list of these failed pages, let's see how long it gets.

In T48096#585785, @LazyDodo (LazyDodo) wrote:

I was browsing the scraped copy last week, and bumped into a searchbox somewhere, we'd probably have to get rid of those otherwise users will use them and make tickets that it is not working.

I was browsing the scraped copy last week, and bumped into a searchbox somewhere, we'd probably have to get rid of those otherwise users will use them and make tickets that it is not working.

In T48096#585634, @Tuomo Keskitalo (tkeskita) wrote:

4. @Dan McGrath (dmcgrath) : My script has stopped running a few times, wget says "Unable to establish SSL connection." When I try to continue, the page is loaded without errors. Any ideas what is causing this? I'm not choking the server, am I?

1. The old wiki contains deleted pages, such as this, which redirects to here which is deleted. wget does not want to download such pages. Is it OK to leave these pages out, or should I try to find some way to get also them in? My script is collecting a list of these failed pages, let's see how long it gets.
2. The old wiki contains pages with non-ascii characters in their page names, which at least my ext4 filesystem does not like to use correctly as directory name, such as this. I'm gonna convert all special characters in page names into %XX hexadecimal representation, using wget option --restrict-file-names=ascii. That way the pages will at least be downloaded, but I think they will not be found if you just try to access the static wiki like this, instead they would be accessible like this which makes it impossible to read. This affects only the page name, the page contents is still intact. Is this compromise acceptable? This affects only non-english pages.
3. Just realized that the static wiki URL for "Page" would be "https://archive.blender.org/wiki/index.php/Page, so the base path is /wiki/index.php and not just /wiki. Is that OK?
4. @Dan McGrath (dmcgrath) : My script has stopped running a few times, wget says "Unable to establish SSL connection." When I try to continue, the page is loaded without errors. Any ideas what is causing this? I'm not choking the server, am I?

@Brecht Van Lommel (brecht), ah, much blind. Ok then :)

People could use a throwaway email address.

@Brecht Van Lommel (brecht), ity shouldn't be possible to have any activity in the dev.b.o for until you verify your email. So that part i didn't quite understand.

I don't think changes to the form make that much of a difference either way. We have been through many iterations on the old bug tracker and on Phabricator. We had fields in the old tracker, and some people would put wrong information just to fill things in. They're just a significant amount of users who don't follow instructions no matter what you do.

In T48096#577069, @Benjamin Humpherys (brhumphe) wrote:

Does the wiki log whenever someone tries to visit a missing page? That would be a good way to identify broken links.

@Benjamin Humpherys (brhumphe) I don't know about mediawiki, but at least www server logs errors. Good idea to use those to make the final check, when I get there!