Fracture Modifier 2.8 port
Open, NormalPublic

Description

This is a summary of my design thoughts for porting the Fracture Modifier to 2.8
based on https://lists.blender.org/pipermail/bf-committers/2018-February/049150.html and following.

It is intended to open a hopefully fruitful discussion about what to consider during the Fracture modifier port and even the cache design.

For now, I assume Alembic will be used as cache backend, like described in https://developer.blender.org/T54690
There duplis are mentioned to represent streamed geometry in blender.

The Fracture Modifier should hold internally some representation of streamed geometry too, but return a regular mesh datablock, so it can be used as a regular modifier in the stack.

Using duplis here on object level would fix the geometry streaming to the base object thus being like always the first modifier.
But it is useful to have other modifiers before the FM, to be able to fracture this content and use it nondestructively.
Generally the geometry cache just shall avoid continouous refractures which are computationally expensive.
For example this is bmesh boolean and bisect for less suitable / nonmanifold geometry.

Modularization of Geometry Generation

Maybe it is wise decouple the actual "cutting" method ? For example just feed in raw voronoi cells, "process" them and write result to alembic geometry cache.
Process is boolean or bisect or maybe something else.
Even more generally spoken, decouple geometry generation from storage.
The generation process can be nonvoronoi too, like island decomposition or even vdb-based volume fracturing.
(just to make sure design is modular and extendible here)

Base / previous modifier Geometry → geometry generator operator (voronoi shatter (+intersect provider), island decompose, packing separate objects) → FM

Modularization of Cache

It is important in case we decide to have different backends / or backend has major update.
Suggestion would be to split the cache logically to a Geometry Cache and a Transform Cache.

Geometry Cache

The storage engine should be encapsulated in a "geometry storage provider".
(can be abc, can be different format for static geometry atleast, just to decouple it)
And how to keep geometry "parts" apart ? natural mesh islands, user selectable facemaps, generated facemaps
vertexgroups (something which isnt awfully slow, since it will be needed to advance the simulation)

storage provider needs to be dynamic, as in must be able to store more geometry (or less) per frame, without doubling the existing geometry, which is there on subsequent frame ranges
as in change contents at change events, but keep indifferent pieces only once. kinda smart geometry storage (vs Alembic dumping all in each frame)
Thats why each cache item should have an own frame range, but additionally we quickly need to query what is on frame x (lookup structure, update during simulation, slower at sim, faster playback)
for faster sim maybe option for "dumb/ big storage" ? and "smartify" in an additional post step to reduce persistent file footprint

Transform Cache

The simulation cache should be a "transformation provider", which can transform the geometry parts. for rigidbody this has been the traditional blender pointcache,
but each geometry item could get an own loc/rot or even transform matrix (and even more custom data, velocity, acceleration, etc)
But as transform changes over time, this needs to be a sequence per item and its valid frame range.
For the dynamic fracture case, make sure to take over last old transform to new "child" shards, setting a suitable start transform.
As backend we could have alembic here too, transformation provider(item) would write xform property data to the abc file.

So the abc file has as seen from FM a geometry provider part(store geometry once per update step, nothing inbetween...
prefracture is 1 initial step, dynamic are multiple) and a transform (sequence) provider

FM → geometry cache (untransformed) → abc

FM → transform sequence cache(per item, for frame range) → abc

FM → transformed geometry provider (cachestream) → Mesh

disabled FM, empty FM(before first fracture) → Mesh passthrough

Fitting in the simulation provider

Here we have the rigidbody bullet wrapper, decouple this from "Object" and object group in world.
The rigidbody world maybe needs to know contained objects, or even only the rigidbody parts....
since those are the cached items... shape per item is generated from geometry provider,
bullet updates loc/rot and feeds into transform provider (making matrix transform, writing to abc)
item could have as customdata an object name/ index + local object shard index.

FM → simulation provider → shape geometry, transform → rigidbody wrapper → bullet
bullet → rigidbody wrapper → transform update, rigidbody object → FM
FM → transformed geometry provider (mesh output),
FM → transform provider (item) (write to sequence); dynamic (new initial transform)
FM → geometry provider (item) (dynamic, new items)

Update (27.05.2018)

(just a rough outline)

typical fracturing case: pointcloud → voronoi (bbox) → bisect / boolean → shard
shard holds an own mesh, own customdata

(other cases, cutter plane based fracturing for example, but we get shards too there)

we only need indexes to polys for quick access (instead of storing parts of the mesh elsewhere like in mesh islands → vertices)
so:

struct MIsland {
	int polystart;
	int polycount;
	//it is a contiguous area of polys... well except for split shards !!!
}

so better keep an array of polys → can we use facemaps for it ? is Lookup quick enough ? like... get faces of map x... vertexgroup lookup was awfully slow before (2.7x) for example
would like to avoid a custom structure

or... just an int cd layer per poly... meshisland_index.... extract Meshisland structure from it at load time and store as runtime structure in FM ?
another layer (or 2) per island... rigidbody index in world ←→ which object / island is this rigidbody ?
to avoid searches... a lookup map ... now we have cache_index_map / cache_offset_map in the world
would rather give each rigidbody that "address" info directly (ob / island) and each island the rb index (another cd poly int layer)

The FM, holds a Mesh representing the current state of geometry (with current transforms, and maybe the start transform too)

this is a case for Alembic, Islands should be stored like objects are, + transforms over time... start transform is transform on start frame then

each island should have a defined range of validity, like frame 10 to 50 or so, so each shard is stored only once and is has valid transforms in that range

(each island could have some kind of ID, too.. a Name, an unique number, or both) → useful for later python exposure.. this is island X, this is island Y... etc

struct MeshIsland {
	int ID;
	int startframe, endframe;
	int polycount;
	int* poly;
	int rigidbody_index; //in the world
	//should be stored when the rigidbody is created, and the rigidbody gets current object index (from collection) and island index it belongs to
	//rigidbody will hold loc,rot
	//mi should make a matrix for transforming its verts from it (more easily)
	float [4][4] transform;

	float last_loc[3];
	float last_rot[4];
}
FractureModifierData {
	//ListBase islands; // maybe an array... since mostly this is a fixed number, but, it might grow in dynamic, but lookup still is faster for stuff to be removed

	MeshIsland **islands;
	int island_count;

	AbcWriter, AbcReader....
	Abcfilename...

	//if not cached yet... need some logic to check whether frame has been simulated
	writeIslands(frame) → do we have duplicates, if valid, do not write again, only write transform... if not valid anymore... delete island from current frame, if new, write geometry assign new id
	update islands listbase (append), need quick way to lookup islands to remove

	readIslands(frame) → if not valid, try to simulate this frame !!!
	else "stream" from Abc, transforming the existing islands (so store geometry in prefracture case only on startframe, with validity till endframe, also take care of start / endframe updates
	update start and end accordingly...

	if island_count different, rebuild mesh (better on validity events / changes), else just re-calc transform, in the modifier (un-apply old trans, re-apply new)

	Fracture operator will invoke the modifier once (or just trigger the actual fracturing code, the modifier should just update transforms + the mesh)
	Dynamic fracture events will trigger the fracture code too
	Fracturing will update the meshislands array and re-initialize the writer (to write new islands, new start transforms)
}

FM "queries" the rigidbody each frame and updates the matrix if necessary (keep lastloc, lastrot for quick comparision, to avoid matrix ops)
currently the FM geometry flickers / looks broken because the rb sim loop updates the geometry (a bad approach)

Proposal: remove the old Rigidbody system... replace by FM, each classical rigidbody could be an FM with 1 island
also handle constraints properly... cases: 1:1 1:n n:n... FM should handle this, but empties cant carry modifiers.... → use "empty mesh" here, hidden / not rendered, maybe with the axis shape of empty
in this case we need to be able to edit individual constraints (even if FM holds multiple, they just must not be objects !!!!, but pickable somehow

alternative, keep objectbased constraints, but its duplicate code...
further considerations: shard-space texture space... like duplis but for islands (since the object stays in place, only the mesh moves... for correct displacement, where the texture moves with each shard)

Details

Type
Design

whoops, this was automagically added to the codequest workboard... just wanted to tag it properly with "code quest"...

Using Alembic as a cache format has a big downside: it can't be updated. Alembic performs certain compression and deduplication operations while writing to the file, which disallows partial updates to an already-existing file. If we want to be able to read frames 1-100 from the cache and then write 101-200 to it, we need either a way to split those ranges into separate Alembic files, or not use Alembic.

hi,teacher!
The Fracture Modifier will support blender2.8?

we need either a way to split those ranges into separate Alembic files, or not use Alembic.

Well, or we just don't care and for baking always write the entire frame range. That's fine by me too.

Sybren A. Stüvel (sybren) triaged this task as Normal priority.