Mesh Cache Sequence Modifier
Closed, ArchivedPublic

Description

First version of a mesh cache sequence modifier patch. There is still alot to do but it is currently working and i want to get some feedback.

The purpose of this Modifier is to read a sequence of 'mesh' files eg 'meshobject_[00001-00120].bin'

Currently Realflow mesh, and Realflow particle sequences are supported.

Documentation can be found at http://wiki.blender.org/index.php?title=User:Levon/mesh_cache_sequence_modifier

This is my first blender coding, so i expect there to be a few issues with the code. alot of the code has been copied from the fluidsim read cache.

I welcome any feedback.

Details

Type
Patch

Ignore the previous patch.

I should note that i have taken out the OBJ sequence code as it needs a total rewrite. OBJ sequence is still in the menu, but it does nothing

Hey thanks for the patch. Can it support .obj(since it is a very common format) in the future ?

Hey thanks for the patch. Can it support .obj(since it is a very common format) in the future ?

yep, i will be adding OBJ files next.

Sergey Sharybin (sergey) triaged this task as "Normal" priority.Sep 25 2014, 9:32 AM

Only did quick view through the patch, some feedback so far:

  • Seems in quite a few places you break indentation of an existing code, this makes reading the patch more difficult. Blender uses tabs instead of spaces for indentation, you might want to check for that in some parts of the patch.
  • I'm not sure, can't it be a part of existing Mesh Cache modifier? Just it's gonna to be a bit weird to have "Mesh Cache", then also "Mesh Cache Seq" and somewhere in the future "Mesh Cache Alembic"? Would be cool if some general system for such a caches is laid down.
  • I don't really thing it's limited to adding a modifier, it should also be some changes to the modifier stack itself. So if there's a cached file for the current frame you don't evaluate modifiers which goes prior to the cache modifier.
  • What are the plans about invalidating the cache?

Adding @Campbell Barton (campbellbarton) here as well.

  • I'm not sure, can't it be a part of existing Mesh Cache modifier? Just it's gonna to be a bit weird to have "Mesh Cache", then also "Mesh Cache Seq" and somewhere in the future "Mesh Cache Alembic"? Would be cool if some general system for such a caches is laid down.

Maybe it should be renamed. But I think it is a bad idea to mix them in one modifier.

Mesh Cache modifier is a kind of deform modifier. Vertices count must be the same in base mesh and cache.
It have a lot of options. A modifier stack can mix several mesh cache modifiers.
Mesh Cache Sequence modifier purpose is to be able to use cache with variable vertices count for fluid growing meshes or alive/born/died particles.
It is mesh replacement. Base mesh could be a suzanne. You cannot add a mesh cache modifier after this one.

Only did quick view through the patch, some feedback so far:

  • Seems in quite a few places you break indentation of an existing code, this makes reading the patch more difficult. Blender uses tabs instead of spaces for indentation, you might want to check for that in some parts of the patch.

had QT creator set up incorrectly, ive fixed this now and ill upload a new patch once i have checked all modified files.

  • I'm not sure, can't it be a part of existing Mesh Cache modifier? Just it's gonna to be a bit weird to have "Mesh Cache", then also "Mesh Cache Seq" and somewhere in the future "Mesh Cache Alembic"? Would be cool if some general system for such a caches is laid down.

as zeauro said. the Meshcache modifer is 'eModifierTypeType_OnlyDeform' and uses the deformVerts (method??) ,no vertices or faces are added or deleted. i originally tried to add my code to the meshcache modifier, but was unable to get it working this way. if you can suggest a way of combining them into one, i will give it a go.

Alembic does allow whole scenes to be baked into a single file. multiple objects, and object types. which doesnt fit into the way blender modifiers are designed, ie, 1 mesh in, 1 mesh out.
however alembic can be uses for a single object bake in the same way the bpys files currently work, a single cache for 1 particle cache, a single cache for 1 cloth/softbody cache.

i havent looked into how the rigid body cache works in blender yet.

  • I don't really thing it's limited to adding a modifier, it should also be some changes to the modifier stack itself. So if there's a cached file for the current frame you don't evaluate modifiers which goes prior to the cache modifier.

That is a good idea, and similar to how Houdini works. the filecache node will bake out bgeo, and then read bgeo, with out computing the nodes above it.

  • What are the plans about invalidating the cache?

could you explain to me what Invalidating the cache means?

i believe the long term goal is to replace blenders cache system with alembic?
while alembic is becoming the standard in cache's i think this modifier will act more as a bridge to other software's cache files in C, for speed.

i had tried originally to convert real flow into blenders fluid cache format in python. It worked fine, but it had added overheads. when you have a 1gb file per frame, converting was taking a few seconds on a fairly fast computer, and also doubles the data storage. each time the cache is exported from Realflow, another convert is required.

Thanks for your feedback.

@ronan ducluzeau (zeauro), with such an approach we're gonna to have like 3-4 cache modifiers? We will go with alembic sooner or later and it'll be good to land some generic system for the caching, for which alembic, obj files etc would be just a backend. I don't really see huge benefit of having loads of decoupled modifiers.

If the existing modifier is limited for the goals, it's still possible to extend it's functionality?

@levon hudson (levon), by invalidating the cache i mean freeing the cache when it's no longer relevant. Imagine, you've got baked scene which plays real fast because it's all baked. And then you re-pose some character. or remove the object from the scene, or add new one, or change mesh topology. Such changes shouldn't really take artists attention all the time. Such an invalidation is the task of the dependency graph.

@Sergey Sharybin (sergey), if it is used for physics cache, it should not be exposed to the user as a modifier.
It should just be the format of files registered in blendcache folder. So, it would be relative to cache panel of any physics modifier.
Maybe, it would be a cache panel in scene or world tab or it would be a output node of nodal particles.
But the common use of alembic is to import/export a whole scene.
Ideally, alembic's place is in File->Import, File->Export menus.

If alembic is used as a registration of a modifier stack, I am not sure it is the best idea to put it in only one modifier. It should be added in modifiers stab but at same level that add button.

The fact is it make no sense to merge both modifiers that have different goals and use in one.
Actual mesh cache modifier was made to read same polycount and deform.
Mesh cache sequence modifier is to only read variable polycount. (its purpose is not to use it like a cache panel. Its purpose is to read mesh animation created from other software. Absolutely, no invalidation is wanted, here.)
An alembic modifier would be to register, read and deform.

The patch made available one more limited cache modifier.
But it would confuse the user to have 2 modes with different limitation inside one modifier.
It only make sense to keep one modifier, if Alembic modifier was already made.

hello,
when a compiled version of Blender will come out with the patch?
Can you help me or give me the link to download blender for 64 bit windows with RealFlow ?

Thank you in advance

Amazing modifier, but I can not activate patch , someone could release a build of blender with the patch , please ...

I built for Linux 64 with this patch but have not tested yet (opens fine though).
Link: https://www.dropbox.com/s/mx8171j5iss3ru2/blender_mesh_seq.tar.gz?dl=0

Have fun,
Varanda

mvaranda hello, is that the patch works on mac os x? I use Blender 2.7 on mac and am very interested in this patch to import the RealFlow. thank you very much in advance,

Ignore the previous patch.

I should note that i have taken out the OBJ sequence code as it needs a total rewrite. OBJ sequence is still in the menu, but it does nothing

Hello all, I have never done "patch" on software, but do any of you could tell me how to do on mac os x to install this patch in my blender and to halp me to import RealFlo... thank you for any help.

sorry for the lack of updates. ive moved for work and have very limited access to internet.

I do have a mac OSX build but i cant upload at the moment. i should be able to upload in the new year.

@Thierry ROUGERON (digitoopix)

its fairly simple to build blender on OSX just follow : http://wiki.blender.org/index.php/Dev:Doc/Building_Blender/Mac

then download the patch, place it in your blender_source dir and type

patch -p1 < meshsequence_v002.patch

Hello and Thanks a lot... i go to try it Levon.

Hi and thank you for the work!

I tried to make a build on a Windows PC and on a Mac but it failed both time.. Am I the only one with this problem?

Note: before adding the patch the building process work just fine.

@Lukas Toenne (lukastoenne) is working on alembic support, and think at this point we should focus energy there for mesh-cache functionality.

Closing for now. if there is some utility/useful logic in this patch it can always be used... but for now think its not really the best way forward compared to alembic.

Yeah I agree @Campbell Barton (campbellbarton).

I have not done anymore work on this as Lukas is working on allembic.

It really makes a lot more sense caching the alembic cache into ram for better playback rather than loading each frame.

i have compiled with current Master and uploaded a OSX version to http://www.graphicall.org/1150

Hello Levon

I am not a developer but a user and have downloaded your build in http://www.graphicall.org/1150 for OS X
I find it works very well in importing reflow bin files and would like to carry on using it but I have found that all the add ons and cycles have been disabled, they are all present in the blender executive file.
Could you tell me how to activate them or if I can extract the mesh sequence modifier add on and use on another blender build ?

Thank you

Aliakin

heya, sorry my bad, i compiled blender with most things turned off in my dev branch. ill recompile on the weekend with cycles.

Hey Levon!

Is it possible to make a build also for win 64bit?

heya, sorry my bad, i compiled blender with most things turned off in my dev branch. ill recompile on the weekend with cycles.

Hello Levon,

Thanks for your awesome work, I just tested the OS X version and it works like a charm. But can we please have a Win 64-bit build too

https://vimeo.com/134824292
our last project were had to import every frame of Realflow's OBJ manually and animated the render visibilty of the meshes.

Thanks again.

rebased against blender 2.75a release.

Im working on getting a windows build

Windows 64bit of 2.75a with mesh cache Sequencer patch applied thanks to @Martin Felke (scorpion81)

having trouble uploading to graphicall so its on dropbox until then,

https://dl.dropboxusercontent.com/u/14364171/blender_meshseqcache_realflow_win64.7z

Mac OSX 2.75a with mesh cache Sequencer patch applied
http://graphicall.org/1150

There is still one bug that you need to animate the Frame Offset, due to something in the new Depsgraph. im not going to be fixing this as the patch is closed.

Levon!

Thank you so much for this build! You just saved us countless ours of importing realflow meshes by hand.

:D :D :D

This project will be available in Trunk or it's "dead"? There are information at OS X Build (graphicall) "This is going to be the last build available, as im no longer working on it."

This is greate project giving lot of cool possibilities for blender, I'm very like to see it in official builds :)

levon hudson (levon) added a comment.EditedAug 14 2015, 4:00 PM

Project won't go in official builds. Alembic will be coming soon to blender and realflow can output alembic. Which is why I am not working on this patch anymore.

I never finished coding OBJ and RFC particles.

I don't have time to work on this anymore as it won't make it into master, and it got the job done at the time.

thanks
i hope you develop it again or someone do it
because sometimes we need to control parameters of bin file when create meshes ; like in others realflow plugins for 3dsmax ; lightwave .....
and in the last webinar of realflow team they showed blender in the list of plugins available ; and i never found any plugin except yours

Levon Hudson, thanks for a wonderful build, but I think your on to something here. Honestly Blender has been waiting for Alembic for years now and will continue to wait. Though I think this gets the job done, what about the particle support? I'm glad I can get .bin particles into blender then theres nothing to do with them?

My point is as a professional VFX artist using blender Reflow integration is a must and Alembic might not ever come around for blender.

@Strider Moore (stridermax) as I said in my previous post, I don't have any need for this anymore.
I don't use realflow anymore.

You can use particles with the particle texture for volumetrics for foam. If you want a solid mesh, you need to mesh your particle's inside realflow. There is no API for realflow to do the meshing inside Blender. You would have to speak to next limit to get that inside Blender.

@levon hudson (levon) hudosn (levon) please compile for us

Mesh Cache Sequence Modifier with ( vray for blender 2.73)

@levon hudson (levon) hudosn (levon) please compile for us

Mesh Cache Sequence Modifier with ( vray for blender 2.73)

I'm not too familiar with the vray build, but I think you just have to copy the Python add-on into this build.

@levon hudson (levon) hudson (levon) brother help me to copy python add-on to this , i can,t copy .patch or diff to blender. need your great suggestion. thnx for reply

Hi,

It's related, I just can't get it to work. The build from dropobox posted above, just don't work. Here's wider description:

http://blenderartists.org/forum/showthread.php?398565-Blender-Mesh-Cache-Sequence-Modifier&highlight=

Anyone? Please help, I really need it :)

Regards,
Kamil.