If a group is appended from another file, it is impossible to remove the group datablock, as well as the object/mesh/etc. datablocks because the original objects are not in the scene and can't be deleted. Even if the group datablock has no duplicator objects using it, and no objects in any scenes are members of it, it doesn't get purged like other 0-user datablocks in blender on file save. This also prevents all of the datablocks used by the group from being purged.
This bug was reported in 2006:
And was incorrectly closed due to being marked as archived. This behavior still exists.