EDIT: last posts updates
Latest version of the document can be accessed on the official website.
I think one of the most important parameters we have to agree upon is the simulation step.
When you say that you aim for a control frequency of 500 Hz, it implies that the time step is either 2 ms or 1 ms (the Webots minimum).
Even with quite powerful hardware, I think this could be difficult to simulate in close to real time since 9 perspectives (8 robots + observer) have to be rendered at 8*100 + ~30 frames per second.
I made a quick test on one of our well equipped workstations and got a realtime factor of 0.1 when running 4 robot controllers at the same time. (Granted, there are some optimizations in our image pipline which would need further improvement, but I mostly want to point out, that this can be quite a big issue)
To summarize: I think we need to really test the capabilities, especially regarding image transfer of the simulation. I think that control frequency or simulation step is a parameter that should be defined relatively soon, as it is quite crucial for teams to adapt their software to it. Some teams may use a fixed time step in their software which might be quite a bit of work to adapt.
Thank you very much for the feedback.
I agree that the 500Hz available update for robots is quite ambitious and definitely requires performance testing. Currently we are aiming high and the next step is to define what needs to be changed (note that for some adult size robots, 500Hz is still below their update rate).
Moreover, there are two points I want to stress out:
Moreover, a few points that need to be stressed out given what I understood of your comments are the following:
Could you confirm the resolution you used and if the robot control software was running on the same workstation as the simulator?
Finally, while I do agree that we need to test the capabilities as soon as possible, while the server specifications have not been decided and some aspects of the simulation are still under evaluation, it’s not possible to define those parameters clearly. I do understand that this might be very inconvenient for teams, but we will likely not be able to set in stone such elements before beginning of April.
My team has been discussing the 100MB/s/team communication restriction and we are concerned that this limitation is overly restrictive.
On our robots we use a pair of cameras in a stereo setup. Each camera provides us with a 1280x1024 bayer format image at up to 170fps. So, per robot, we have a camera data stream of
2 * 1280 * 1024 * 170 / (1024 * 1024) = 425 MB/s (although we likely are running closer to 150fps which results in 375MB/s).
If we were to severely constrain our systems image requirements to a 640x480 RGB image at 30fps (RGB because webots can’t provide bayer images) we would still only be able to field a single robot on our team
2 * 640 * 480 * 3 * 30 / (1024 * 1024) ~= 53 MB/s
If I recall correctly, it was mentioned during the meeting on 9th March that it was not anticipated that the network would be a bottleneck during the competition, and this was the reason
why the cameras images were not going to be compressed. However, if the network is not likely to be a bottleneck why is there a 100MB/s/team communication cap?
If we allow for image (JPEG) compression, then a 1280x1024 RGB image occupies 1.5MB and a stereo pair at 30fps requires
2 * 1537477 * 30 / (1024 * 1024) ~= 88 MB/s
and a 640x480 RGB image occupies about 350KB with a stereo pair at 30fps requiring
2 * 360943 * 30 / (1024 * 1024) ~= 21 MB/s
So, if we can allow image compression in the simulator, we would at least be able to field a full team of 4 robots with a 640x480 stereo camera at ~25fps.
Thanks for your contribution, I do fully understand that this limitation might be considered as overly restrictive.
One thing that has changed since the workshop on March 9th is that we decided to make sure that running the competition would be possible using classical 1 Gbps Ethernet connection. This was the reason that based our decision of allowing JPEG compression with a custom level of compression (see message setQuality on the latest document). This is also allowing teams to control the tradeoff between quality of images (compression level) and frequency/resolution.
However, given this choice we have still strong restrictions based on the GPU consumption to generate the images seen by the cameras. Currently 50MB/s/team is what we can guarantee as available for teams.
Given the concern you raised, I would be willing to run more in depth tests on the platform that we will use for the competition with a complete world to try to increase the limit on sensors (but not on bandwidth, so you are right, compression would be necessary anyway to achieve stereo VGA at 30fps). Those results should be available in around 2 weeks from now and might allow us to increase this limit.
I hope this clearly answers your question and do not hesitate to ask if you feel it still needs clarification.