Great!

Bit to give feedback we will need to have a build :D

I have some production ABC's, I can share one of them, the other I can test it here, but I can't share it.

Is there still a need to compile alembic separatedly apart from adding the patch and compiling blender as is with the patch?

I'm eager to test this! :D

Kevin you are going to become my hero! Alembic, OpenVDB... wow!

Cheers.

Alembic needs to be compiled separately (and linked like any system wide library). I don't think the Alembic source will ever be shipped with Blender, as it might be implemented in Cycles as well in the future.

Yes, a build would help to test this.
So animation im-/export is not supported right now, but will be soon? Do I get that right?

For export you can choose a frame range, and various other settings which I have yet to document. Importing only loads the first frame, then you can put a 'Mesh Cache' modifier on the imported objects to 'load' the animated data (which could be automated through an addon or so).

For builds, I can only (try to) provide a portable 64-bit Linux build, as I don't have access to a Windows or Mac machine, maybe someone else can share a build for those platforms.

I would take a linux build for testing, I just don't have time to build it myself.
Also I can probably provide some alembic files later.

Here is my humble attempt to create a portable Linux build (several features like Cycles, or Smoke simulation are turned off to simplify the build process): (edited: see task description).

Not working for me, sorry (Fedora22). Maybe I will find some time later to build it myself.

Alright, have it running now. On import both Forward and Up- axis are set do +X by default. Changing up to Z does something, but results in some crooked angle.
More tests later.

Ok, then if someone can guide me to build alembic and link it I can prepare a VM and make a build, I'm not sure how to do those two things, for the time being I know how to build blender and how to apply a patch, but I don't know what do I have to build regarding alembic and how the linking works.

Cheers.

I manage to build it using cmake-gui under Ubuntu.
I referred to alembic libs that were build by install_deps script as is.

Some cmake targets are not obvious.

Cmake target / lib name under linux
ALEMBIC_ABCCORE_ABC_LIBRARY / libAlembicAbcCoreAbstract.a
ALEMBIC_ABCOGAWA_LIBRARY / libAlembicAbcCoreOgawa.a
ALEMBIC_ABCUTIL_LIBRARY / libAlembicUtil.a
ALEMBIC_ABC_LIBRARY / libAlembicAbc.a
ALEMBIC_OGAWA_LIBRARY / libAlembicOgawa.a

Here is an Alembic archive to play around with: Alembic_Data.7z
Animated Camera with animated FoV, deforming meshes with multiple materials and a bit of hierarchy. Also there is a jpg-sequence for reference.
The original patch is handling it quite well (except for the FoV animation).

Hi!

I am a newbie and I would like to share the steps I have followed trying to build Alembic support in Blender (because I don't know if I did them well).

Please could you give a bit of feedback in order to know if I did it well? thanks :-)

My system is a Linux x64 (Linux Arch)

1. I downloaded the Alembic latest source code, cloning its github repository, lets say that the work directory where I'm going to work is "/home/jorge/Blender"
   
   cd /home/jorge/Blender git clone https://github.com/alembic/alembic.git

2. I have used "cmake-gui" in order to generate a build directory where Alembic will be built
   
   cd alembic cmake-gui
   
   I put: "Where is the Source Code" --> /home/jorge/Blender/alembic "Where to build the binaries" --> /home/jorge/Blender/build_alembic
   
   The rest of options are left "as is". I press "Configure" and after "Generate".

3. Build alembic:
   
   cd ../build_alembic make -j8 sudo make install sudo ldconfig

4. Download blender source code and build it from "master" branch (just for checking that everything is ok):
   
   cd .. I just followed the instructions at: https://wiki.blender.org/index.php/Dev:Doc/Building_Blender/Linux/Arch/CMake And I built Blender and tested it in order to ensure that it worked.

5. Go to "alembic_basic_io" branch:
   
   cd /home/jorge/Blender/blender git checkout alembic_basic_io

6. Configure blender in order to have alembic enabled:
   
   cd /home/jorge/Blender/blender cmake-gui I put: "Where is the Source Code" --> /home/jorge/Blender/blender "Where to build the binaries" --> /home/jorge/Blender/build_blender
   
   Press "Configure"
   
   Enable "WITH_ALEMBIC"
   
   Press "Configure"
   
   Put "ALEMBIC_INCLUDE_DIR" --> /usr/local/include Put "ALEMBIC_ABC_LIBRARY" --> /usr/local/lib/libAlembic.so Put "ALEMBIC_ABCCOREFACTORY" --> /usr/local/lib/libAlembic.so Put "ALEMBIC_ABCCORE_ABC_LIBRARY" --> /usr/local/lib/libAlembic.so Put "ALEMBIC_ABCCORE_HDF5_LIBRARY" --> /usr/local/lib/libAlembic.so Put "ALEMBIC_ABCGEOM_LIBRARY" --> /usr/local/lib/libAlembic.so Put "ALEMBIC_ABCMATERIAL" --> /usr/local/lib/libAlembic.so Put "ALEMBIC_ABCOGAWA_LIBRARY" --> /usr/local/lib/libAlembic.so Put "ALEMBIC_ABCUTIL_LIBRARY" --> /usr/local/lib/libAlembic.so Put "ALEMBIC_OGAWA_LIBRARY" --> /usr/local/lib/libAlembic.so
   
   The rest of options are left "as is".
   
   I press "Configure" and after "Generate"

QUESTION: Could it be possible to left only three entries for Alembic configuration, "WITH_ALEMBIC", "ALEMBIC_INCLUDE_DIR" and "ALEMBIC_ABC_LIBRARY"?
As I'd like to become a newbie Blender developer, how could I do that? (I know a bit about cmake, but I would like to be able to do this and do a commit)

7. Build blender with alembic support:
   
   cd /home/jorge/Blender/build_blender make -j8 make install

8. Open Blender and do some tests:
   
   I have downloaded an .abc example (an octopus) from:
   
   https://storage.googleapis.com/google-code-archive-downloads/v2/code.google.com/alembic/Alembic_Octopus_Example.tgz
   
   (Extract this file)
   
   In Blender, open file using the menu:
   
   File --> Import --> Alembic (Default) (.abc)
   
   and select the file "alembic_octopus.abc" and the octopus will appear in Blender Nice (See the screenshot that I have put in: http://jgascon.net/?p=179 )

9. Testing the Alembic exportation:
   
   Once the octopus is loaded in Blender, I would like to export it and re-import it later, so, I press the menu
   
   File --> Export --> Alembic (Default) (.abc)
   
   And choose a name for the Alembic file (in my case "octopus_exported.abc") and export it using the default options.
   
   And the file is saved but it only weights ~3 MB (the original octopus weights ~35 MB).
   
   If I create an empty scene in Blender and press the menu:
   
   File --> Import --> Alembic (Default) (.abc)
   
   And I select the "octopus_exported.abc", nothing happens :-( Only an empty mesh called "octopus low" is created.

Is it something that I'm not doing right?

Thank you in advanced for your feedback.

In T48075#368785, @Thomas Volkmann (knekke) wrote:

Alright, have it running now. On import both Forward and Up- axis are set do +X by default.

Those should be set the forward and up axis of the object contained in the .abc archive. I've set them to +Y/-Z in the alembic_basic_io branch as a default value, but I'm not too sure what other software are generally using.

In T48075#369591, @Thomas Volkmann (knekke) wrote:

Here is an Alembic archive to play around with: Alembic_Data.7z
Animated Camera with animated FoV, deforming meshes with multiple materials and a bit of hierarchy. Also there is a jpg-sequence for reference.
The original patch is handling it quite well (except for the FoV animation).

Thanks for the file. I can see that I forgot to port some code to create object hierarchies from the original Python script. Though, I don't really know how we'd handle camera animation, since cameras can't have modifiers. Maybe it is acceptable to import the data as keyframes.

In T48075#369993, @Jorge Gascon Perez (jgascon) wrote:

NOTE: I had to copy the file "Config.h" from: /home/jorge/Blender/build_alembic/lib/Alembic/Util to /home/jorge/Blender/alembic/lib/Alembic/Util Is there a way to avoid to copy this?

Did you set CMAKE_PREFIX_PATH (to set the path for make install)?

QUESTION: Could it be possible to left only three entries for Alembic configuration, "WITH_ALEMBIC", "ALEMBIC_INCLUDE_DIR" and "ALEMBIC_ABC_LIBRARY"?
As I'd like to become a newbie Blender developer, how could I do that? (I know a bit about cmake, but I would like to be able to do this and do a commit)

Afaik, the various variables are required since Alembic is split in multiple libs. At least it is the case when compiling as static libs, you are compiling it as a dynamic library and apparently only one lib is output (?). Or maybe Alembic as compile flags to tell whether or not to output miultiple libs.

NOTE: I had to copy the file "libAlembic.so" from: /home/jorge/Blender/build_alembic/lib/Alembic/libAlembic.so to /home/jorge/Blender/build_blender/bin Is there a way to prevent this copy?

You need to add the path to the alembic shared object in your LD_LIBRARY_PATH environment variable.

9. Testing the Alembic exportation:

...

And the file is saved but it only weights ~3 MB (the original octopus weights ~35 MB).

If I create an empty scene in Blender and press the menu:

         File --> Import --> Alembic (Default) (.abc)

 And I select the "octopus_exported.abc", nothing happens :-( 
 Only an empty mesh called "octopus low" is created.

Is it something that I'm not doing right?

I would say there is a bug. I've been testing imports of exports from Blender and it seemed fine.

Dear Kévin,

Thank you very much for your quick answer :-)

I'm figuring that most of the bugs might come since I am using the Alembic github "master" branch and maybe it is not as stable as the last stable version of alembic (Alembic_1.5.3_2013121700.tgz), next time I am gonna try to build Blender with this version instead the github repository.

Regarding your questions:

Afaik I don't see the CMAKE_PREFIX_PATH entry in the Alembic cmakefile that I'm using, the most similar entry I find is CMAKE_INSTALL_PREFIX.

And regarding the LD_LIBRARY_PATH system var, I just have seen that in my Arch Linux this var is not initialized :-O
So I'm going to define it and try to build Blender with the version 1.5.3 of Alembic.

Thank you very much Kévin, I hope with these changes everything will go fine :-)

Regarding building Alembic:
Make sure to checkout a release version (git checkout 1.5.8) this will create a nice folder /usr/local/alembic-1.5.8 when you build it.
Instead of assigning all the alembic libs in cmake, you can "export ALEMBIC_ROOT_DIR=/usr/local/alembic-1.5.8" before running cmake. This way all the alembic stuff is found automatically.
All this from the top of may head, so all this might not exactly be as I stated....

As for keyframed Cameras: while this would definitiley work, it feels more like a hacky workaround. there might be a workflow where you just want to overwrite an alembic file to update your scene (something I would only do in rare situations, but still).

Orientation of imported abc files worked perfectly ootb with the original patch, no need to change it.

cheers

In T48075#370005, @Thomas Volkmann (knekke) wrote:

As for keyframed Cameras: while this would definitiley work, it feels more like a hacky workaround. there might be a workflow where you just want to overwrite an alembic file to update your scene (something I would only do in rare situations, but still).

This here patch is for a basic implementation of Alembic in Blender as using an import/export scheme. What you describe would require a tighter implementation, at the scene level, of Alembic, which was attempted during the Gooseberry project, and failed because the caching system in Blender wasn't up to the task. Such an implementation is kinda planned for Blender 2.8x though (along with a rewrite of the caching system). So for the scope of this patch, even if it is hacky, it could work as a temporary solution to get camera streaming.

Orientation of imported abc files worked perfectly ootb with the original patch, no need to change it.

Yes, but they were hardcoded, which is bad since you can't assume that every Alembic archive out there is Y-up. Also, all the importers in Blender work this way, so let's be consistent.

So for the scope of this patch, even if it is hacky, it could work as a temporary solution to get camera streaming.

Fair enough.

Yes, but they were hardcoded, which is bad since you can't assume that every Alembic archive out there is Y-up. Also, all the importers in Blender work this way, so let's be consistent.

Maybe take their hard-coded orientation as a default then?

What I don't get is: if all that stuff is going to change in 2.8 anyway, why go through the hassle of porting a fine working patch to a less functional version just to meet some requirements that will no longer be valid next year?
Why can't you just say: This is the end of 2.7...we finally give you some Alembic but please don't look at the code, as it will be rewritten anyway.
Everyone that actually uses Blender would be quite happy about it I guess. And the devs couldn't care less, because they are off to a rewrite anyway.

just my 2cent

Hi Kévin, thanks for looking into this.

I hope this will use Alembic from git master (instead of release 1.5.8) since the updated CMakeLists.txt is much more useful, without having to hack/patch it to provide only libs. There's also only one library now - libAlembic.so - instead of it being split up.

I have succesfully installed and tested Alembic (from git) in this Blender branch, using my supplied "FindALEMBIC.cmake". I configured Alembic using these commands from the cloned directory:

mkdir build; cd build
cmake \
-D CMAKE_BUILD_TYPE=Release \
-D CMAKE_INSTALL_PREFIX=/opt/lib/alembic-1.5.8 \
-D USE_STATIC_BOOST=OFF \
-D USE_TESTS=OFF \
-D ALEMBIC_LIB_USES_BOOST=ON ../

This is on Arch Linux with system OpenEXR, Boost and HDF5, and Blender build with OpenVDB and OpenSubdiv.

My initial testing was simply using the Alembic octopus. Without changing any import settings the model was upside-down, which is ok since I figured it would be corrected when using the Mesh Cache modifier, and it did - almost. Thing is, I needed to reset rotation (Alt+R) after setting up the modifier, before the model was correctly rotated.

Regarding the modifier, would it be possible to have a search field for Object Name, like this:

instead of opening a file browser? Or is the plan to use the file browser on .abc files like when browsing a .blend file?

Also could the import/export settings have a scale/unit option when dealing with files from apps using centimeters by default?

Thanks again, I hope you find this helpful. I can work on updating the install_deps.sh with these updated Alembic build options if you like?

In T48075#370035, @Thomas Volkmann (knekke) wrote:

What I don't get is: if all that stuff is going to change in 2.8 anyway, why go through the hassle of porting a fine working patch to a less functional version just to meet some requirements that will no longer be valid next year?

Only the operators (import/export) will be rewritten (if required). The code to read and write Alembic data will stay the same, unless there are fundamental changes in the Blender and/or Alembic in those areas. So, the majority of the code will probably stay the same.

In T48075#371792, @Ejner Fergo (ejnersan) wrote:

Hi Kévin, thanks for looking into this.

I hope this will use Alembic from git master (instead of release 1.5.8) since the updated CMakeLists.txt is much more useful, without having to hack/patch it to provide only libs. There's also only one library now - libAlembic.so - instead of it being split up.

Whilst it does seem to simplify a lot the build process, I prefer to stick with the latest official release here. (Then people are free to install whatever version of the library as they want).

My initial testing was simply using the Alembic octopus. Without changing any import settings the model was upside-down, which is ok since I figured it would be corrected when using the Mesh Cache modifier, and it did - almost. Thing is, I needed to reset rotation (Alt+R) after setting up the modifier, before the model was correctly rotated.

Would need to check on that. Generally the whole object importer would need to be revisited to get the transform hierarchy right.

Regarding the modifier, would it be possible to have a search field for Object Name, like this:

instead of opening a file browser? Or is the plan to use the file browser on .abc files like when browsing a .blend file?

The file browser actually comes from a copy/paste :) I removed it in the branch. I don't think a drop down menu would work here, since I believe it deals with Blender ID objects, and not arbitrary data.

Also could the import/export settings have a scale/unit option when dealing with files from apps using centimeters by default?

Just added that in the branch.

Thanks again, I hope you find this helpful. I can work on updating the install_deps.sh with these updated Alembic build options if you like?

I'm not against any sort of help to get things done, but it has to make use of the latest Alembic official release (as "latest development version" is too vague to be referenced for users, official docs and so on).

Hi Kevin,

Does your implementation include import/export of particle/varying point number meshes? I can send fluid meshes back and forth between Maya and Modo using Alembic. Would this be possible with Blender?

Hi Kévin,

Sorry for the long delay, been following Alembic development for a hopefully imminent 1.6.0 release :) I get your point about using only official releases, but as in the case of OSD it is pulled from git in the install_deps script (could be a leftover, will check). The ease of building, not needing to patch across files and simpler/cleaner blender cmake is definately a plus of Alembic-git (there's a 1.6 branch now). In any case, this is not super relevant.

I've been testing some more, and found some issues.

Exporting an .abc from Blender and importing it, the mesh has no vertices. Blender also crashes randomly after I entered and exited edit-mode on this "empty" object. I can't reproduce this consistently. Simple test is to export the default Blender scene and import it. The camera rotation is also wrong. Unfortunately I can't test the exported file in a 3rd party program presently, but the output from 'abcecho' looks fine.

Speaking of export, Ogawa archive format should be default. This is cosmetic, but could you make the export archive setting look like this?:

Also could End Frame be set to whatever the scene end frame is, instead of 1?

Using the test file from @Thomas Volkmann (knekke) I'm seeing glitches when more than one object has animation. With one object the animation is fine, but with 2 objects (Trex & torus), when scrubbing the timeline, on random frames the models "flip" to the rotation they had when importing, only on one frame. Testing again just now, Blender crashed when setting up the second modifier. Trying again, Blender didn't crash, showed the glitching briefly, and then hang completely with the console spamming HDF5 errors, even though the file archive is Ogawa.

I don't know how far you are with this implementaion, but cameras are not imported correctly (wrong rotation and no animation). Also empties (null objects) are not imported. The torus is parented to an empty, so that animation is lost.

I know you said a drop-down isn't possible for alembic data, but I really hope some solution will be found. The hierachy is not always /name/nameShape. For example the Trex animation is on /Trex/Trex/TrexShape. It would be optimal if this imformation could be found directly in Blender, without having to use the Alembic CLI tools. I found that 'abcechobounds' provides the clearest view of the objects (but not lights/cameras/empties), so if this could be replicated in Blender somehow.

For reference, here's the output of abc_scene.abc without the digits:

/null/null1/torus/torusShape
/right/rightShape
/Trex/Trex/TrexShape
/front/frontShape
/left/leftShape
/back/backShape
/

I also found another example where the naming convention isn't followed:

/Teapot001Xfo/Teapot001
/

Sorry to dump all this on you...

@Galen Beals (galenb)
Can you provide a test file?

In T48075#373222, @Ejner Fergo (ejnersan) wrote:

Hi Kévin,

Sorry for the long delay, been following Alembic development for a hopefully imminent 1.6.0 release :) I get your point about using only official releases, but as in the case of OSD it is pulled from git in the install_deps script (could be a leftover, will check). The ease of building, not needing to patch across files and simpler/cleaner blender cmake is definately a plus of Alembic-git (there's a 1.6 branch now). In any case, this is not super relevant.

(OpenSubdiv had to be patched to work in Blender) It appears there is a better chance Alembic 1.6 will be released before this patch lands in Blender, so maybe we can make the switch.

Speaking of export, Ogawa archive format should be default. This is cosmetic, but could you make the export archive setting look like this?:

Done (Ogawa + UI) :)

Also could End Frame be set to whatever the scene end frame is, instead of 1?

Unfortunately we don't have access to the scene data in the place where the UI for the operator is generated. Now I think maybe it''d be better to replace the end frame setting with a "number of frames" setting?

Exporting an .abc from Blender and importing it, the mesh has no vertices.

This bug was brought to my attention in D1783, and should be fixed now.

Using the test file from @Thomas Volkmann (knekke) I'm seeing glitches when more than one object has animation. With one object the animation is fine, but with 2 objects (Trex & torus), when scrubbing the timeline, on random frames the models "flip" to the rotation they had when importing, only on one frame. Testing again just now, Blender crashed when setting up the second modifier. Trying again, Blender didn't crash, showed the glitching briefly, and then hang completely with the console spamming HDF5 errors, even though the file archive is Ogawa.

I don't know how far you are with this implementaion, but cameras are not imported correctly (wrong rotation and no animation). Also empties (null objects) are not imported. The torus is parented to an empty, so that animation is lost.

As far as the implementation goes, importing needs a bit of rewrite. Specifically we should build a transformation hierarchy and use that to create objects. For now we are only testing if a given node in the archive is a Mesh, Camera or NURBS (we also need to support more object types). Alembic does not appear to have the concept of an empty object, in the case of @Thomas Volkmann (knekke)'s file, empties are just transform matrices with a "locator" metadata attached to them (apparently this is how Maya exports empties). Anyway, import needs to work on/create a transform hierarchy, that alone should solve a few import problems.

I know you said a drop-down isn't possible for alembic data, but I really hope some solution will be found. The hierachy is not always /name/nameShape...

I think it could be okay to add the modifier on the objects during imports, and set the path at that stage (since we have all the information at that point), otherwise we would have to collect maybe hundreds of paths and expose them to the user somehow.

@Galen Beals (galenb)

Not really, the first frame can be loaded, but for streaming through the modifier, the number of points needs not change.

In T48075#373230, @Kévin Dietrich (kevindietrich) wrote:

Done (Ogawa + UI) :)

Wow that was fast :) Thanks!

Unfortunately we don't have access to the scene data in the place where the UI for the operator is generated. Now I think maybe it''d be better to replace the end frame setting with a "number of frames" setting?

Hmm.. maybe better to keep end frame then, so us lazy people don't have to calculate number of frames if start frame is eg. 37 ;) Unless I'm misunderstanding that setting in alembic context..

This bug was brought to my attention in D1783, and should be fixed now.

Ah yes cool, just build and tested.
So, I tried to export a super simple RBD cube falling (with baked keyframes), but when importing and using the modifier, there was no animation. I exported with start frame = 1 and end frame = 60. Am I missing something?

As far as the implementation goes, importing needs a bit of rewrite. Specifically we should build a transformation hierarchy and use that to create objects. For now we are only testing if a given node in the archive is a Mesh, Camera or NURBS (we also need to support more object types). Alembic does not appear to have the concept of an empty object, in the case of @Thomas Volkmann (knekke)'s file, empties are just transform matrices with a "locator" metadata attached to them (apparently this is how Maya exports empties). Anyway, import needs to work on/create a transform hierarchy, that alone should solve a few import problems.

I see. Do you know what could be the issue with HDF5 error when having 2 objects with the modifier (Ogawa archive)? The error is:

HDF5-DIAG: Error detected in HDF5 (1.10.0) thread 0:
  #000: H5F.c line 410 in H5Fis_hdf5(): unable open file
    major: File accessibilty
    minor: Not an HDF5 file
  #001: H5Fint.c line 544 in H5F_is_hdf5(): unable to close file
    major: Low-level I/O
    minor: Unable to close file
  #002: H5FD.c line 927 in H5FD_close(): close failed
    major: Virtual File Layer
    minor: Unable to close file
  #003: H5FDsec2.c line 441 in H5FD_sec2_close(): unable to close file, errno = 9, error message = 'Bad file descriptor'
    major: Low-level I/O
    minor: Unable to close file
...
repeated many times

I think it could be okay to add the modifier on the objects during imports, and set the path at that stage (since we have all the information at that point), otherwise we would have to collect maybe hundreds of paths and expose them to the user somehow.

Yes, that would be great. Even if the paths were exposed, just the saved modifier setup time alone would help :)

I'm sorry, the HDF5 error was on my end. Turns out HDF Group had made threadsafe and high-level-library options mutually exclusive* in version 1.10.0 and 1.8.16, and Arch Linux had updated to 1.10.0 which was build without threadsafe (Alembic needs this when using HDF5). I found some other issues with Alembic using HDF5-1.10.0 and how to fix them, and made the Alembic developer aware of them. I also communicated the problem with not having the new HDF5 build with threadsafe option to the Arch package maintainer.

• There is an option to allow build HDF5 with both options (--enable-unsupported), so I'm using that on my locally installed v1.10.0.

I still get the glitching when 2 or more objects have the modifier enabled. If I disable the "visibility" of all modifiers but one, the animation is fine.

Some updates from me:

• The rotation issues should mostly be fixed now, there are still some minor ones though, but at least objects should be in the right position (provided that up and forward axises are set properly in the import panel).
• The crashes and random glitches when having multiple objects with animation should be fixed now.
• Null objects (locators in Maya) are imported now.
• (usability) Mesh cache modifiers are added by default.

I updated the link in this task's description with a build containing those changes.

There are no solutions for animations of cameras, empties and nurbs just yet, though I have some ideas here.

@Ejner Fergo (ejnersan) have you tried compiling Alembic 1.6 with C++11? With the current official version (1.5.8) I'm getting a lot of linking errors. Being able to compile using C++11 would be neat since some people might be using newer compiler (such as GCC 6) which default to that standard, or simply they might be compiling in C++11 mode.

@Galen Beals (galenb), just in case for testing, do you have a simple example alembic file contianing fluid meshes?

In T48075#375733, @Kévin Dietrich (kevindietrich)

@Galen Beals (galenb), just in case for testing, do you have a simple example contianing fluid meshes?

I'll see if I can rustle something up.

Hi,

I get a segmentation fault when I load my example .abc. It loads, but when I start scrubbing the timeline it crashes after a second or so (clicking the timeline seems to work):

HDF5-DIAG: Error detected in HDF5 (1.8.14) thread 0:
  #000: ../../src/H5F.c line 439 in H5Fis_hdf5(): unable open file
    major: File accessibilty
    minor: Not an HDF5 file
  #001: ../../src/H5Fint.c line 565 in H5F_is_hdf5(): unable to close file
    major: Low-level I/O
    minor: Unable to close file
  #002: ../../src/H5FD.c line 1103 in H5FD_close(): close failed
    major: Virtual File Layer
    minor: Unable to close file
  #003: ../../src/H5FDsec2.c line 442 in H5FD_sec2_close(): unable to close file, errno = 9, error message = 'Bad file descriptor'
    major: Low-level I/O
    minor: Unable to close file
HDF5-DIAG: Error detected in HDF5 (1.8.14) thread 0:
  #000: ../../src/H5F.c line 439 in H5Fis_hdf5(): unable open file
    major: File accessibilty
    minor: Not an HDF5 file
  #001: ../../src/H5Fint.c line 565 in H5F_is_hdf5(): unable to close file
    major: Low-level I/O
    minor: Unable to close file
  #002: ../../src/H5FD.c line 1103 in H5FD_close(): close failed
    major: Virtual File Layer
    minor: Unable to close file
  #003: ../../src/H5FDsec2.c line 442 in H5FD_sec2_close(): unable to close file, errno = 9, error message = 'Bad file descriptor'
    major: Low-level I/O
    minor: Unable to close file
Writing: /tmp/blender.crash.txt
[1]    1564 segmentation fault (core dumped)  bin/blender

and blender.crash.txt:

less /tmp/blender.crash.txt
# Blender 2.77 (sub 1), Commit date: 2016-05-25 15:16, Hash 3a875e0
bpy.context.space_data.recent_folders_active = 0  # Property

# backtrace
bin/blender(BLI_system_backtrace+0x1d) [0x13d9f4d]
bin/blender() [0xab6839]
/lib64/libc.so.6() [0x304f034a50]
/lib64/libhdf5.so.9(H5FD_set_eoa+0x40) [0x3058cb4840]
/lib64/libhdf5.so.9(H5FD_locate_signature+0x109) [0x3058cb4c19]
/lib64/libhdf5.so.9(H5F_is_hdf5+0x4a) [0x3058ca048a]
/lib64/libhdf5.so.9(H5Fis_hdf5+0x47) [0x3058c9c4a7]
bin/blender(_ZN7Alembic11AbcCoreHDF52v76ArImplC2ERKSsNSt3tr110shared_ptrINS_15AbcCoreAbstract2v720ReadArraySampleCacheEEEb+0x155) [0x21baa71]
bin/blender(_ZNK7Alembic11AbcCoreHDF52v711ReadArchiveclERKSsNSt3tr110shared_ptrINS_15AbcCoreAbstract2v720ReadArraySampleCacheEEE+0x5d) [0x21b9b61]
bin/blender(_ZN7Alembic3Abc2v78IArchiveC2INS_11AbcCoreHDF52v711ReadArchiveEEET_RKSsNS1_12ErrorHandler6PolicyENSt3tr110shared_ptrINS_15AbcCoreAbstract2v720ReadArraySampleCacheEEE+0xdb) [0x1437e9b]
bin/blender() [0x14316c0]
bin/blender(ABC_get_vertex_cache+0x53) [0x1434be3]
bin/blender(MOD_meshcache_read_abc_index+0x17) [0xfb47b7]
bin/blender() [0xfb4624]
bin/blender() [0x10a6f55]
bin/blender() [0x10a8100]
bin/blender(makeDerivedMesh+0x6d) [0x10a81fd]
bin/blender(BKE_object_handle_data_update+0x480) [0x1191e40]
bin/blender(BKE_object_handle_update_ex+0x1e0) [0x118c720]
bin/blender() [0x11c65a6]
bin/blender(BLI_task_pool_work_and_wait+0xdb) [0x13da8fb]
bin/blender() [0x11c74d7]
bin/blender(BKE_scene_update_for_newframe_ex+0x27a) [0x11c990a]
bin/blender(ED_update_for_newframe+0xaf) [0xd7f37f]
bin/blender(wm_event_do_notifiers+0x338) [0xabd168]
bin/blender(WM_main+0x20) [0xab73d0]
bin/blender(main+0x395) [0xa9f3a5]
/lib64/libc.so.6(__libc_start_main+0xf0) [0x304f020700]
bin/blender(_start+0x29) [0xab38e9]

Maybe something is wrong with my libs, but I wonder why hdf5 is making trouble, because the .abc should be ogawa (or not? I don't remember)
But from what I could see, the orientation and hierarchy looked fine now.

Hi @Kévin Dietrich (kevindietrich)

Great progress! So nice having the modifier set up automatically, and no glitching anymore! I'm certain you are on top of what needs to be done, but I'll ask just to be sure: Will it be posible to support object transformations (outside of mesh cache)?

Regarding building Alembic 1.6-git I just did one without Boost (so using C++11) and had no errors, and Blender build and works as expected - This is on Arch Linux w. GCC 6.1. I did not enable C++11 support in Blender cmake though, is that what you meant?

Edit: Sorry, just looked at 'ccmake ../blender' and WITH_CXX11 is ON (has that changed recently?)

@Thomas Volkmann (knekke)
I had the same issue as you can see in my previous posts. What distro are you using? If Alembic is build with HDF5 support (which it needs for legacy reasons), the HDF5 lib has to be build with threadsafe support. You are right the .abc file is using Ogawa, but it will still crash just by having HDF5 support (don't know why). Build Alembic without HDF5 support, and the file works, but then you can't open files saved with HDF5 from other software/studios... So, make sure your HDF5 is build with the option --enable-threadsafe

BTW: Alembic 1.6 does not need hdf5_hl library anymore, and it works with HDF5-1.10.0 now (files are forward-compatible).
However, Blender complains about the missing 'libhdf5_hl.so.100' file, even after I changed my FindALEMBIC file. This is with my own build HDF5-1.10.0 (--disable-hl --enable-threadsafe), Alembic 1.6-git using this HDF5 and Blender using both these libs. I have of course run ldconfig before compiling, and ran 'grep -nir hdf5_hl' in blender dir only finding references in FindALEMBIC.cmake.
What am I missing?

Edit2: My bad, again! It's an Arch issue... Hopefully the whole debacle of what needs hdf5_hl and/or hdf5_threadsafe will be solved soon.

Some updates from me:

• I decided to remove the ability to choose the transform axis during export and import, now Y-up <-> Z-up transforms are hardcoded to what the original patch was doing. The reason being that I couldn't get it exactly right, so I'd rather sacrifice a bit of flexibility here for the time being. Using the scale property during import and export does not work anymore, I forgot about it during the revert :|
• It's possible to import hair now (hair export was already supported). Hair are imported as curves for the time being (say, as a proof of concept), because that's how Alembic stores them. I'm not sure if it's possible to detect the cases where curves in Alembic are actual hair to be able to add a Blender hair system instead of using curves object.

So now it can be interesting to get some test files from other software which contain hair or fur.

@Ejner Fergo (ejnersan):

• In the current paradigm the only way to modify an object after an Alembic import would be to add a modifier to it after the mesh cache modifier.
• C++11 (WITH_CXX11) should be enabled in Blender if you use GCC 6, that was indeed recently changed (https://developer.blender.org/rB6115267a845a49bdf9a5d4c701fcf5b995fc499a). I might bump to Alembic 1.6 then, since 1.58 seems to fail with C++11.

In T48075#375872, @Kévin Dietrich (kevindietrich) wrote:

• In the current paradigm the only way to modify an object after an Alembic import would be to add a modifier to it after the mesh cache modifier.

So baked Rigid Body sims won't work then? :-/

• C++11 (WITH_CXX11) should be enabled in Blender if you use GCC 6, that was indeed recently changed (https://developer.blender.org/rB6115267a845a49bdf9a5d4c701fcf5b995fc499a). I might bump to Alembic 1.6 then, since 1.58 seems to fail with C++11.

Alrighty then!

In T48075#375892, @Ejner Fergo (ejnersan) wrote:

So baked Rigid Body sims won't work then? :-/

This implementation won't cover any kind of caching like that. Rigid bodies are supposed to be supported for export actually, though I think it can be implemented if it's not the case. Then when you import the file back, it will create new objects, and add a mesh cache modifier to them for streaming the data (as long as the vertex count/topology matches).

You are right! I tried a stupid simple test again - a RBD cube falling on a plane (without baking to keyframes) and a cube with 2 LocRot keys - export to alembic, import in Houdini Apprentice and it works! Importing in Blender there is no motion on either cubes though...

An idea for importing animated alembics into blender:

Hi there!

These days I was thinking of how to load animated alembics and I have an idea, would you like to listen it?

Note 1: I have not a depth understanding of the Alembic code inside Blender yet, cause I am still reading/understanding it.
Note 2: This is a quick-and-dirty approach that is easy to implement (I guess), it has a lot of limitations but I think that they can

be fixed quickly.

Note 3: Maybe someone have thought this idea before, in that case thanks for reading my comment.

Explaining this idea with an example:

Let's figure that I have an alembic file with an animated character inside whose animation goes from frame 1 to 100, ok?

Now, imagine that in Blender I have an empty scene and it is currently in frame 4.

Now, I load this alembic file in Blender, and automatically, only the frame 4 of the alembic object is loaded, nothing more. As a result, I will have the animated character in its frame 5 in the scene, right?

If I change the frame to, let's say, to frame 5, Blender, being aware that this object is feeded from an alembic file, will automatically unload the mesh of the character and it will load the frame 5 of the alembic file, as a result, we will have the frame 5 of the animated character in Blender.

If we go to frame 101 or more, the last frame of the character (100) will be loaded.

As you can figure, we have some advantages and disadvantages:

Advantages:

• It is a quick & easy trick to implement, jut we have to override the callback of the framenumber change event of blender.
• You can load objects with varying number of vertices, for example simulated water surfaces and things like that.
• It is memory-conservative: You only store in memory the current frame of the object.

Disadvantages:

• It could be slow, for example when you press ALT+A to animate the scene, it can be a lot of overhead unloading/loading meshes, but there are several solutions to fix that and to improve performance.

Future work:

• it could be possible to read an alembic file and load the keyframes of the object and to put them in the "Dope sheet" for the user to interact with them?

So, what do you think? I guess that in my spare time, after the work, I could read the current code and start to implement this.

Let me know your point of view about this fact, I understand that there are limitations but ir could be a good starting point.

Jorge.

Hej Jorge,

that is a nice proposal, but this is actually how alembic (or any pointcache for that matter) works. Only the current (or which ever you ask for) frame is loaded (otherwise you could run out of ram pretty quickly). Someone please correct me if I'm wrong.

cheers

A Linux build thread is on BA and a build is available:
http://blenderartists.org/forum/showthread.php?399763-Dev-Linux-build-Alembic-I-O

In T48075#376165, @Thomas Volkmann (knekke) wrote:

Hej Jorge,

that is a nice proposal, but this is actually how alembic (or any pointcache for that matter) works. Only the current (or which ever you ask for) frame is loaded (otherwise you could run out of ram pretty quickly). Someone please correct me if I'm wrong.

cheers

Actually it is advantageous to have all the frames loaded in memory so you can get better playback. That's kind of the whole point of cacheing in the app and leaving it that way (as opposed to just using it for import/export). Imagine you have a complicated character where it's deformations are causing the scene to drag. This is a case where you place a cache modifier on top of the stack and cache it. You can then disable all the modifiers below and let the cache play instead. In this case you need to have all the frames loaded in memory.

Now, I don't know if this is actually the way our implementation works or not. But this is at least part of its intended use use.

Perhaps kévin can implement a toggle in the UI for pre-loading all the frames?

Thank you Thomas and Galen for your comments :-)

You are right, keeping only one frame in memory allows to save a lot of memory, and keeping all of them allows a smooth and a fast playback. I know about a third approach, that consists in a multithread prefetching of frames, this technique uses one or more additional threads that load the next N frames before are needed, the advantages are that the playback should be fast and smooth but only keeping a few frames in memory, not all. Some months ago I implemented something similar for a network server, and its the way that Boost Asio libraries are used to load files.

Unfortunately, I have not taken a look to the Alembic code yet, but I would be very happy if I can check the current status of the point cache, any clues?

Thank you in advance.

OSX build also available and so far tested to be on par with linux:

http://blenderartists.org/forum/showthread.php?399763-Dev-Win-Linux-build-Alembic-I-O&p=3060405&viewfull=1#post3060405

Jens

Hi Kevin I'm testing your work with a Coke ad! :)
I'm get a Alembic from real flow... is working fine!

I found a error... the relative path to mesh modifier and constraint doesn't work for me...

Please tell me if you make a new build for Ubuntu.

Thanks!

@Eugenio Pignataro (oscurart) for new builds with fixes better follow the alembic thread on blenderartists: http://blenderartists.org/forum/showthread.php?399763-Dev-Win-Linux-build-Alembic-I-O

Some new builds are made every 2-3 days on average, depending on the development. You can also post bug reports there, though here is fine too. I'll check on that error later today.

Hello Kévin,

Here are 2 simple test file from houdini, exported from one scene.

• flat geo is a flat geometry export
• packed object contain only ref geo + transform matrix.

The point of using packed object is to get light weight abc file (prefered method for non deforming geometry (ie bullet sim, scene transfer..) and scene containing lots of instances).

The issue is blender slowdown while loading this kind of export (probably the Transform Cache constraint)

Here in the test files, flat geo reads at 60+ fps wherease packed object reach only ~2fps.

Thanks for your work !

https://drive.google.com/file/d/0B4Igx19SH9TlWXNXelFhUWZXdEE/view?usp=sharing

@ Boris
Tested your files in last linux build:
Flat geo reads at 60+ fps whereas packed object reach ~24fps.
So packed IS slower but somewhat acceptable on an actual highend cpu ?

Is this hdf5 based files ? We build hdf5 without threads atm. cause it would need mpi.

Jens

Hey jens,

I guess it's ogawa (not sure)
Yep it is really slower. Not sure what you mean by "acceptable", but yeah, it still works as expected, just wanted to notice the slowdown (Kévin talked about profiling, that's why i give the files).

computer spec:
GPU: Quadro K4200
CPU: Bi intel Xeon 2.40Ghz E5-2630 v3
Ram: 128Go

Hi all,

Alembic 1.6.0 is released! https://github.com/alembic/alembic/releases

Here is a patch for Alembic/HDF5 cmake files that I've been using for 1.6.0:

These are the cmake options I used to build Alembic 1.6.0:

mkdir build; cd build

cmake \
-D CMAKE_BUILD_TYPE=Release \
-D CMAKE_INSTALL_PREFIX=/opt/lib/alembic-1.6.0 \
-D USE_TESTS=OFF \
-D ALEMBIC_LIB_USES_BOOST=OFF \
-D USE_HDF5=ON \
-DHDF5_ROOT:STRING=/opt/lib/hdf5 \
../

Notice: I have ALEMBIC_LIB_USES_BOOST=OFF to build it using C++11 since my gcc is v6, and Blender automatically uses C++11 when using that gcc version. When using gcc v5 or lower, set ALEMBIC_LIB_USES_BOOST=ON. Also, HDF5 support in Alembic 1.6.0 is disabled by default, so I enabled it and pointed cmake to my local install.

I said it before, and personally still hope that the HDF5 dependency can be dropped.
I'll try to bring this up in the blenderartist thread again, as I'm pretty sure all user examples in that thread are Ogawa (I haven't checked all though).

Anyways, thank you for the super work and progress @Kévin Dietrich (kevindietrich)!

@Ejner Fergo (ejnersan) Thanks for the notification!

Bumped Alembic to 1.6 in the branch (rBf2bb125a6a02) and dropped HDF5 support (it can still be enabled for the curious by using WITH_ALEMBIC_HDF5, but it is not officially supported from now on).

Some examples in the blenderartists thread are HDF5 (the RealFlow ones for instance).

I reconfigured my buildbot for the new alembic-1.6/hdf5-less method now, first of those type:

• PACKAGE-INFORMATION --

Branch: alembic_basic_io
Revision: rBf2bb125a6a02
Submodules: locale a87cfb8 addons 75fb4e0 addons_contrib 72c24a1
OS: GNU/Linux, Architecture: x86_64, GLIBC: 2.19
Builddate: Fr 1. Jul 11:19:19 UTC 2016
Filesize: 72255692 byte
Shasum: dfc8b6ad0cd98f91cca39239ef20a131fa71cab9
URL: http://www.jensverwiebe.de/Blender/b..._latest.tar.xz

The BA thread: https://blenderartists.org/forum/showthread.php?399763-Dev-Custom-Builds-Alembic-I-O&s=0fd0298c12ea1ceb53bdd8d279814aae&p=3069700&viewfull=1#post3069700

... to be continued ...

Jens

Hi @Kévin Dietrich (kevindietrich) just wondering why you didn't use the patch? I see that you updated FindAlembic manually, which is of course most important, and granted the rest of the changes I made are mostly cosmetic:

• Remove references to Samplerate
• Copyright year
• hdf5_hl requirement removed in Alembic 1.6.0

Last item is not super relevant anymore (thumbs up for that!), but if someone chooses to use HDF5 it might as well be fixed. That also means that the HDF5 build option "--enable-unsupported" is not needed, as that option is for hdf5_hl.

Updated patch since your last changes:

@Ejner Fergo (ejnersan), I quickly read the email notification from your previous post in my mailbox yesterday and must have not realized you attached a patch to it, so that's why I didn't use it :) Applied the changes from your new patch, thanks!

@Kévin Dietrich (kevindietrich) I just created a simple patch to fix building of Alembic 1.6.0 on Linux. I attached you as a reviewer since you're working on this branch. Although it's super simple, I hope it helps somehow. :)