Mantaflow not caching if path contains characters with accent #98467

New Issue

Didier TRESSE · 2022-05-28T15:54:10+02:00

Didier TRESSE commented

2022-05-28 15:54:10 +02:00

System Information
Operating system: Windows-10-10.0.22000-SP0 64 Bits
Graphics card: NVIDIA GeForce RTX 3060 Laptop GPU/PCIe/SSE2 NVIDIA Corporation 4.5.0 NVIDIA 512.96

Blender Version
Broken: version: 3.1.0, branch: master, commit date: 2022-03-08 18:16, hash: c77597cd0e

Short description of error
If the cache folder path for Manta FLow contains "french" accent character, no sim data is computed.

Exact steps for others to reproduce the error

Create a simple blen file in a folder named "d:\Données\Blender\2 - Projets Blender\MyFluidSim\fluidSim.blend"
Create a simple sphere
Choose Object/Quick effect/Quick fluid to create a simple domain
Set the cache directory to a subfolder of the .blend file

Move forward to the fluid sim... No simulation data create

To fix it :

Just rename "d:*Données*\Blender\2 - Projets Blender\MyFluidSim\fluidSim.blend" to "d:*Workspace*\Blender\2 - Projets Blender\MyFluidSim\fluidSim.blend"
Re-open the same file

It works

If cache folder is relative to .blend file or absolute, it doesn't create data if the path contains any character with accent.

Thank you for the great job you are doing
Have a nice day

Didier

**System Information** Operating system: Windows-10-10.0.22000-SP0 64 Bits Graphics card: NVIDIA GeForce RTX 3060 Laptop GPU/PCIe/SSE2 NVIDIA Corporation 4.5.0 NVIDIA 512.96 **Blender Version** Broken: version: 3.1.0, branch: master, commit date: 2022-03-08 18:16, hash: `c77597cd0e` **Short description of error** If the cache folder path for Manta FLow contains "french" accent character, no sim data is computed. **Exact steps for others to reproduce the error** - Create a simple blen file in a folder named "d:\Données\Blender\2 - Projets Blender\MyFluidSim\fluidSim.blend" - Create a simple sphere - Choose Object/Quick effect/Quick fluid to create a simple domain - Set the cache directory to a subfolder of the .blend file # Move forward to the fluid sim... No simulation data create To fix it : - Just rename "d:\**Données**\Blender\2 - Projets Blender\MyFluidSim\fluidSim.blend" to "d:\**Workspace**\Blender\2 - Projets Blender\MyFluidSim\fluidSim.blend" - Re-open the same file # It works If cache folder is relative to .blend file or absolute, it doesn't create data if the path contains any character with accent. Thank you for the great job you are doing Have a nice day Didier

Didier TRESSE commented

2022-05-28 15:54:10 +02:00

Added subscriber: @MagicDid

Richard Antalik commented

2022-05-30 05:05:13 +02:00

Added subscriber: @iss

Richard Antalik commented

2022-05-30 05:05:13 +02:00

Changed status from 'Needs Triage' to: 'Confirmed'

Richard Antalik commented

2022-05-30 05:05:13 +02:00

Can confirm, but since there is no active developer for mantaflow, will set as known issue

Sonny Campbell self-assigned this 2022-06-27 12:23:36 +02:00

Sonny Campbell commented

2022-06-28 13:54:19 +02:00

@MagicDid Can you check if this bug is still happening for you in the 3.3 daily build? I think it might have already been resolved there...

Didier TRESSE commented

2022-06-28 15:48:41 +02:00

Hi,

I've downloaded the 3.3 daily build and tried to bake a Mantaflow simulation and it doesn't work if the path of the .blend file contains an accent. A single default cube and apply quick effet / smoke doesn't generate simulation data if the file is stored in ..../Données/MyProject/anyfile.blend

I just renamed it to .../Data/MyProject/anyfile.blend, reopened it and this time simulation data are generated.

Tell me if you need any log/file/etc...

Hi, I've downloaded the 3.3 daily build and tried to bake a Mantaflow simulation and it doesn't work if the path of the .blend file contains an accent. A single default cube and apply quick effet / smoke doesn't generate simulation data if the file is stored in `..../Données/MyProject/anyfile.blend` I just renamed it to `.../Data/MyProject/anyfile.blend`, reopened it and this time simulation data are generated. Tell me if you need any log/file/etc...

Sonny Campbell commented

2022-06-28 16:29:04 +02:00

@MagicDid |I have created a C:\Données\test.blend . I added a Object > Quick Effects > Quick Smoke to the cube. The cache directory is relative to the blend file as indicated, and when I hit Bake All I can play the animation and see the simulation in Blender 3.3 alpha.

When I try that on Blender 3.2.1 I don't see any simulation. Are you certain it was the 3.3 build you used to test it? Or am I missing somethiing in my test?

@MagicDid |I have created a C:\Données\test.blend . I added a Object > Quick Effects > Quick Smoke to the cube. The cache directory is relative to the blend file as indicated, and when I hit Bake All I can play the animation and see the simulation in Blender 3.3 alpha. When I try that on Blender 3.2.1 I don't see any simulation. Are you certain it was the 3.3 build you used to test it? Or am I missing somethiing in my test? ![image.png](https://archive.blender.org/developer/F13234689/image.png)

Didier TRESSE commented

2022-06-28 17:06:08 +02:00

Here is the test environment (see screenshot). The 3.3.0 alpha is from June 28th (this morning).

It seems you're doing the right test. Cannot understand what is the difference between the 2 environment. The only thing I have to change to make it work is the Données changed to Data.

Here is the test environment (see screenshot). The 3.3.0 alpha is from June 28th (this morning). It seems you're doing the right test. Cannot understand what is the difference between the 2 environment. The only thing I have to change to make it work is the `Données` changed to `Data`. ![Capture.PNG](https://archive.blender.org/developer/F13234775/Capture.PNG)

Didier TRESSE commented

2022-06-28 17:09:22 +02:00

Have you tried both accent and space in the path ?

Didier TRESSE commented

2022-06-30 14:37:02 +02:00

Hello,

I've downloaded the last Blender release from Git, rebuilt it and when I check the MANTA_main.cpp file, it seems that the modification detailed in https://developer.blender.org/D15301 are not included in this release.

Didier

Hello, I've downloaded the last Blender release from Git, rebuilt it and when I check the `MANTA_main.cpp` file, it seems that the modification detailed in https://developer.blender.org/D15301 are not included in this release. Didier

Didier TRESSE commented

2022-06-30 14:50:49 +02:00

Hi again,

I've added some Log to understand what happens. Here are the Python command strings "tried" to be executed :

PythonString : bake_fluid_data_3('D:\\Mes Documents\\2 - Personnel\\Blender\\Blender_Projects\\Donn├®es\\TrashTest\\cache_fluid_4b85f272\\data', 241, '.vdb')
PythonString : smoke_save_data_3('D:\\Mes Documents\\2 - Personnel\\Blender\\Blender_Projects\\Donn├®es\\TrashTest\\cache_fluid_4b85f272\\data', 241, '.vdb', False)

Indeed the "é" character is interpreted into something else between the GUI and the internal Manta wrapping implementation.

Hi again, I've added some Log to understand what happens. Here are the Python command strings "tried" to be executed : ``` PythonString : bake_fluid_data_3('D:\\Mes Documents\\2 - Personnel\\Blender\\Blender_Projects\\Donn├®es\\TrashTest\\cache_fluid_4b85f272\\data', 241, '.vdb') PythonString : smoke_save_data_3('D:\\Mes Documents\\2 - Personnel\\Blender\\Blender_Projects\\Donn├®es\\TrashTest\\cache_fluid_4b85f272\\data', 241, '.vdb', False) ``` Indeed the "é" character is interpreted into something else between the GUI and the internal Manta wrapping implementation.

Sonny Campbell commented

2022-06-30 15:11:56 +02:00

Hey @MagicDid , yeah the problem is that code doesn't actually fix the issue. I thought that was fixing the issue, but the problem is that it's already working for me in the latest release. So when I tested my changes I thought it was fixing the bug, but it wasn't actually doing anything different.

I tried to recreate this issue with a space in the filepath C:\Données\Folder With Space and it also works as expected for me on 3.3, but not on 3.2.1. I'm not really sure why we're seeing different results, maybe it's an issue with environments, but I'm not able to dig any further on it without being able to recreate it I'm afraid...

Hey @MagicDid , yeah the problem is that code doesn't actually fix the issue. I *thought* that was fixing the issue, but the problem is that it's already working for me in the latest release. So when I tested my changes I *thought* it was fixing the bug, but it wasn't actually doing anything different. I tried to recreate this issue with a space in the filepath `C:\Données\Folder With Space` and it also works as expected for me on 3.3, but not on 3.2.1. I'm not really sure why we're seeing different results, maybe it's an issue with environments, but I'm not able to dig any further on it without being able to recreate it I'm afraid... ![image.png](https://archive.blender.org/developer/F13242898/image.png)

Sonny Campbell removed their assignment 2022-06-30 15:12:42 +02:00

Sonny Campbell commented

2022-06-30 15:12:42 +02:00

Added subscriber: @SonnyCampbell_Unity

Sonny Campbell commented

2022-06-30 15:24:31 +02:00

@MagicDid where did you add that logging and I can add it in the same place to see if I can spot any difference?

Didier TRESSE commented

2022-06-30 16:33:32 +02:00

In file C:\Projects\Blender\blender\intern\mantaflow\intern

In runPythonStringmethod :

bool MANTA::runPythonString(vector<string> commands)
{
  bool success = true;
  PyGILState_STATE gilstate = PyGILState_Ensure();

  /* Temporarily set `sys.modules["__main__"]` as some Python modules expect this. */
  PyObject *main_mod_backup;
  manta_python_main_module_backup(&main_mod_backup);

  /* If we never want to run this when the module isn't initialize,
   * assign with `manta_python_main_module_ensure()`. */
  BLI_assert(manta_main_module != nullptr);
  manta_python_main_module_activate(manta_main_module);

  for (vector<string>::iterator it = commands.begin(); it != commands.end(); ++it) {
    string command = *it;

    PyObject *globals_dict = PyModule_GetDict(manta_main_module);
    PyObject *return_value = PyRun_String(
        command.c_str(), Py_file_input, globals_dict, globals_dict);
	string tmp = "PythonString : " ;
	tmp = tmp + command ;
	cout << tmp.c_str() << endl ;

I'm working on using the BLI_get_short_name function to solve this issue (at least on Windows platforms)

In file `C:\Projects\Blender\blender\intern\mantaflow\intern` In `runPythonString`method : ``` bool MANTA::runPythonString(vector<string> commands) { bool success = true; PyGILState_STATE gilstate = PyGILState_Ensure(); /* Temporarily set `sys.modules["__main__"]` as some Python modules expect this. */ PyObject *main_mod_backup; manta_python_main_module_backup(&main_mod_backup); /* If we never want to run this when the module isn't initialize, * assign with `manta_python_main_module_ensure()`. */ BLI_assert(manta_main_module != nullptr); manta_python_main_module_activate(manta_main_module); for (vector<string>::iterator it = commands.begin(); it != commands.end(); ++it) { string command = *it; PyObject *globals_dict = PyModule_GetDict(manta_main_module); PyObject *return_value = PyRun_String( command.c_str(), Py_file_input, globals_dict, globals_dict); string tmp = "PythonString : " ; tmp = tmp + command ; cout << tmp.c_str() << endl ; ``` I'm working on using the `BLI_get_short_name` function to solve this issue (at least on Windows platforms)

Sonny Campbell commented

2022-06-30 16:45:30 +02:00

Yeah I see the same in mine:

PythonString : bake_fluid_data_3('C:\\Donn├®es\\Folder With Space\\cache_fluid_4b85f272\\data', 50, '.vdb')
PythonString : liquid_save_data_3('C:\\Donn├®es\\Folder With Space\\cache_fluid_4b85f272\\data', 50, '.vdb', False)

but my simulation runs when I play the animation

and my fluid cache has the data

I'm on Windows 10 too.

Yeah I see the same in mine: ``` PythonString : bake_fluid_data_3('C:\\Donn├®es\\Folder With Space\\cache_fluid_4b85f272\\data', 50, '.vdb') PythonString : liquid_save_data_3('C:\\Donn├®es\\Folder With Space\\cache_fluid_4b85f272\\data', 50, '.vdb', False) ``` but my simulation runs when I play the animation ![image.png](https://archive.blender.org/developer/F13243216/image.png) and my fluid cache has the data ![image.png](https://archive.blender.org/developer/F13243220/image.png) I'm on Windows 10 too.

Didier TRESSE commented

2022-07-05 16:07:35 +02:00

Hi,

I finally succeeded in understanding what happens in my case. The issue is due to the OpenVDB implementation at the end of the C++ => Python => C++ architecture of Mantaflow in Blender. Here is a sequence diagram of the successive calls leading to the erroneous interpretation of the path at the end.

During all the sequence the path is correctly interpreted as UTF-8 and finally in the OpenVDB implementation the path is interpreted as a basic string (ASCII) by use of std::ofstream with std::string instead of std::swstring. As this is in OpenVDB distribution, there is no chance this to be fixed.

To prove this, I changed the export format to "UNI cache" instead (which is not implemented the same way) and the simulation bake works and I can play it...

Now I know it I'll change my folder's name.

Thanks for your time

Question : how can I help by fixing some issues in Blender ? How to pick up issues ?

Hi, I finally succeeded in understanding what happens in my case. The issue is due to the OpenVDB implementation at the end of the C++ => Python => C++ architecture of Mantaflow in Blender. Here is a sequence diagram of the successive calls leading to the erroneous interpretation of the path at the end. ![TrashSequence.png](https://archive.blender.org/developer/F13261034/TrashSequence.png) During all the sequence the path is correctly interpreted as UTF-8 and finally in the OpenVDB implementation the path is interpreted as a basic string (ASCII) by use of std::ofstream with std::string instead of std::swstring. As this is in OpenVDB distribution, there is no chance this to be fixed. To prove this, I changed the export format to "UNI cache" instead (which is not implemented the same way) and the simulation bake works and I can play it... Now I know it I'll change my folder's name. Thanks for your time **Question :** how can I help by fixing some issues in Blender ? How to pick up issues ?

Richard Antalik commented

2022-07-07 15:48:26 +02:00

@MagicDid Thanks for detailed analysis.

In #98467#1385779, @MagicDid wrote:
Question : how can I help by fixing some issues in Blender ? How to pick up issues ?

In general, pick area you would like to work on and contact developers that maintain this area. They can probably provide you with easier tasks here on tracker so you can get familiar with development process. See https://wiki.blender.org/wiki/Modules

It's good idea to read through wiki and good starting point is https://wiki.blender.org/wiki/Developer_Intro/Overview

@MagicDid Thanks for detailed analysis. > In #98467#1385779, @MagicDid wrote: > **Question :** how can I help by fixing some issues in Blender ? How to pick up issues ? In general, pick area you would like to work on and contact developers that maintain this area. They can probably provide you with easier tasks here on tracker so you can get familiar with development process. See https://wiki.blender.org/wiki/Modules It's good idea to read through wiki and good starting point is https://wiki.blender.org/wiki/Developer_Intro/Overview

Philipp Oeser removed the

Download

What's New

Blender Studio

Manual

Developers Blog

Documentation

Benchmark

Blender Conference

Development Fund

One-time Donations

Mantaflow not caching if path contains characters with accent #98467

Move forward to the fluid sim... No simulation data create

It works