-
Import a piece of data from Excel for each comp? (and other data questions)
Hi there. I’m a bit in new territory here, so I’ve got a couple questions about using Excel data.
I need to place about 50,000 points of data into a world map. (location markers) I’ve started playing with the script, “Locations” to see if that will be a solution, but I’m worried that 50k points is going to be out of the realm of what AE can handle. How should I approach such large data sets?
In addition, I have a date field that needs to offset the lifespan of the marker comp. I was thinking I’d try to clean up the date column in Excel (e.g. 13-Feb-12) to something numeric like we did in Flash, where the Date.time object could express the date in milliseconds since January 1, 1970 (e.g. 1447175866). Then, I’d use an expression to offset the start time of the comp. Any thoughts on the feasibility of this? I’m not sure I can get a date in that format, but hoping some fancy text editor like Sublime or an Excel formula can reformat it for me.
Then, I also need to adjust the AE location marker’s scale by a unique data field from the same Excel set. (range: 0-100,000). I can probably rig up a script to attenuate it to 1-100%, but I’m not sure how or where to bring that data in. Is there a way to make every comp have a variable (maybe an Expression-Slider control?) that receives it’s unique data point?
Thanks for your input! I’m looking forward to this job, but need to make sure it’s all feasible!
-Greg -
3 Replies
-
Just to update my post…
50,000 data points turned into comps kills AE (which wasn’t surprising). I don’t think a comp can really manage more than a couple hundred items, so I’ll need to find another solution there. AE may not be the best tool for this job… -
Sounds like you will need a particle system. Do you want to display the same item at each location at specific times and scale?
Not that I have done anything with this amount of layers, but one possible route to investigate is Plexus with nulls. I haven’t used Plexus, but from what I’ve read, it should be able to place particles based on nulls. I would assume that 50000 nulls takes much less toll on the system than 50000 comps. A null could be born at a specific time and location and have a scale. Then if you can have particles on a single Plexus layer be based on the nulls, that might work.
…of course there is still the scripting question of how to get the nulls created automatically.
Anyways, might be worth a try creating a comp with 50000 nulls and see if AE still responds fine.
-
You probably already tried Flash then?
Another Particular-based option occurred to me: How many points maximum are born at the same time (in the same frame — the same “day” or whatever your minimum time frame is)? If it is a manageable number, you could have that many Particular Emitters being controlled by expressions reading your table data for position and size, emitting one particle for each frame (up to as many particles as are born for that frame) and moving to a new point at each frame.
Reply to this Discussion! Login or Sign Up