While typing the question, I came up with a work-around. If I make the voronoi with the text into editable, then the crumbling works when I apply the hard body tag to the grouped pieces that are created. But there should be a way to do it while keeping the text object, right?