[DISCUSSION] Future for communication protocol

Rules and Roadmap Rules
1

At RoboCup 2019, teams shared their enthusiasm toward having a common protocol for communication, but their concern toward the Mitecom protocol which is not flexible and do not allow custom data.

In this topic, I would like to discuss two different points: the protocol we should use and what kind of incentive we should use to make it more common that teams uses it.

If you have other proposals for communication protocols or suggestion for other kind of incentive, feel free to propose them.

Protocols

Incentive

  • Only official protocol is allowed
    • Forces teams to update their existing software
    • Makes cooperation easier
    • Adding fields require maintenance
  • No additional incentive
    • Teams who want to cooperate can still implement the protocol
    • More flexibility for teams
    • Slower change toward a common protocol
2

I really like the idea of a single protocol for communication as it would enable presenting information to the public about what the robots think.

The implementation effort to integrate this into for example ROS is quite minimal.

How would we handle which messages are standardized for the humanoid league?

How can we prevent teams from using the opponents information they provide against the opponents? Do we even want to prevent this?

Do we require teams to use the standard messages, when they could include the same information in a specialized message for all the information they broadcast between players? How could this be enforced?

3

How would we handle which messages are standardized for the humanoid league?

I am not sure I understand this question. I think that we have to start with a basis at some point (such as the one I proposed, the mitecom one or other proposals). Then if multiple teams show interests for one type of message, we should standardize it.

How can we prevent teams from using the opponents information they provide against the opponents? Do we even want to prevent this?

This is a major problem which is already occuring. There are two different approches to this:

  1. We consider that information sent can be assimilated to speaking and there is no reason to prohibit other teams to use them in their strategies.
  2. We consider that the information are strategical and makes no sense with respect to the gameplay (e.g. using other robots estimated location to replace perception). In this case, we need to separate the two networks (which makes sense anyway from my point of view). This would make ‘using other team information’ more difficult (although not impossible). But I think it would be sufficient.

Do we require teams to use the standard messages, when they could include the same information in a specialized message for all the information they broadcast between players? How could this be enforced?

I think this is definitely an argument which goes in favor of proposing tools building up on standardized messages rather than to try to enforce it. Any possible scheme to try to enforce the use of standardized messages would be either difficult to apply or it would restrict highly the possibilities to send custom messages.

4

I am sorry that was unclear. What I meant was that if we use the solution you provided (protobuf based), we need to specify the messages we send over the Network in order to be able to analyze them for example for visualization of gamestate such as where which robot sees the ball for the public. If every team just uses their own messages, this would not be possible. There should be a standard set of messages which are defined before it is required or suggested for teams to use so that teams do not just implement their own messages.What i specifically mean by messages are the .proto files here hl_communication/proto at master · Rhoban/hl_communication · GitHub.

A different possibility is giving out encryption keys to teams. All the keys could be on the visualization or game controller computer. Teams could still try to steal the encryption keys from these computers but I think that is highly unlikely. I do not know how hard this would be to implement or if there is already a feature for that in protobuf.

We could propose a set of messages which we believe to be quite common or useful and require teams to use these messages if they want to send this specific information. If they want to send information that is not modeled in by the messages, they may use their own message but they need to make a pull request to the main repository so the new message is included. This way we can get an overview of which messages the teams want to use.

5

I haven’t worked with protobuf in depth yet but from what I could gather there is no way of simply turning on encryption for protobuf.

Nonetheless it is still possible to wrap protobuf communications in an encrypted envelope. This should also not be too difficult as long as the team-communication-protocol specifies what encryption to use because basically any language has encryption utilities.

6

The NUbots are in favour of a standard communication protocol.

We feel that a protobuf based implementation is preferable as this allows for a standard set of message fields to be enforced while also allowing teams to add to this set without breaking conformance.
Teams can utilise the standard protocol buffer definition which can be used by all teams, and then add their own fields on top of this as extensions to allow better communication between teams.

This protocol should be communicated over a UDP channel with a single message per packet. UDP is desirable because it’s an unreliable protocol which prevents old data from building up, allows single sender multiple receiver via either multicast or broadcast and it is easier to process each packet as a message meaning it will not require a streaming protocol with synchronisation bytes.

Ideally this proposal should be more about the communication protocol (how we communicate) and less about what is said (the contents of the message). We feel at this time it would be better to get a message protocol defined with a minimalist set of fields. This way the league as a whole can try out new ideas of what needs to be shared between teams, and if a majority of the teams begin to adopt these custom messages they can be added to the standard protocol buffer later. This will promote a more organic evolution of communication and only the fields that are used will end up in every team. If we attempt to define every possible scenario now the protocol buffer will be burdensome to implement for teams and will inevitably fail to cover all scenarios.

In terms of enforcing, this becomes quite difficult. If we require all communication to be broadcasted/multicasted on the network then the game controller could implement a packet sniffer and decode all received packets to ensure conformance, but this doesn’t prevent teams from sending direct communication between their own players, thereby avoiding this conformance check. We feel that this is the best we could hope for in terms of conformance, unless we want to implement the conformance checks at the router level.

Adding encryption seems like it would be more of a headache than it is worth and also seems like it would provide little gain. We also feel that these messages would be akin to speaking and we see no need to prevent the opposition from hearing it. However, if we really want to separate team communications we can assign a multicast group IP to each team (with the game controller in both groups), in this way each team can see their own multicast packets, but they can’t see the other teams packets.

In terms of message content, we propose the following information initially:

  • Player ID + Team ID (same as the game controller packet)
  • Robot position measured from center of field (x, y, theta)
  • Ball position measured from center of field (x, y, z)
  • Game mode, phase, and penalty reason (taken from game controller data)
  • Current walk command (x velocity, y velocity, turning velocity)

Both robot and ball position should be measured in SI units and should (optionally) be accompanied with a covariance measurement. If we add a “message” field (PENALISE, UNPENALISE, ALIVE) to this then we can use this protobuf message to replace the current game controller reply packets allowing teams to only need to implement a single message for sending data to the game controller.

With all of this information we could create a display that looks like this NUbots dashboard display

The circles with the wedges on the field are robot positions and orientations, the curved black arc is the robots walk command, the orange circle is where that robot thinks the ball is, and the arrow leaving the ball is the direction the robot intends to the kick ball in (kick direction could be left out of the message).

The bottom panels are specifics of each robot. “Mode”, “Phase”, and “Penalty” are repeats of game controller data (this data could be used as a minor check for teams implementing the protocol - send the robot game state information and have it repeat it to you). “Walk Command” is the numerical version of the black arc on the field. “Behaviour” and the battery percentage would be removed in this scenario. The camera, ball, and goal images at the very bottom are indications of how long it has been since the robot last saw those things.

A possible implementation of our proposed protobuf message could be

package robocup.humanoid;

/// Needed in proto3 to handle extensions
import "google/protobuf/any.proto";

/// A vector of three floats
message fvec3 {
    float x = 1;
    float y = 2;
    float z = 3;
}
/// A matrix of three vectors
message fmat3 {
    fvec3 x = 1;
    fvec3 y = 2;
    fvec3 z = 3;
}
/// The current playing state of the robot
enum State {
    UNKNOWN = 0;
    UNPENALISED = 1;
    PENALISED = 2;
}
message Message {
    /// ID of the team that is sending this message
    uint32 team_id = 1;
    /// ID of the player that is sending this message
    uint32 player_id = 2;

    /// The position of the robot on the field according to the following convention
    /// x meters along the field with 0 at the centre of the field and positive towards opponent goals
    /// y meters across the field with 0 at the centre of the field and positive to the left
    /// θ orientation of the robot (anti-clockwise from the x axis from above the field)
    fvec3 field_position = 3;
    /// The covariance measure of the robots [x, y, θ] values or NaN if not available
    fmat3 field_covarinace = 4;

    /// The position of the ball on the field according to the following convention
    /// x meters (along the field with 0 at the centre of the field)
    /// y meters (across the field with 0 at the centre of the field)
    /// z meters (up from the field with 0 at ground level)
    fvec3 ball_position = 5;
    /// The covariance measure of the balls [x, y, z] values or NaN if not available
    fvec3 ball_covariance = 6;

    /// The current walk speed of the robot in it's local [x, y, θ] coordinates
    /// positive x being forwards, positive y being strafing to the left, and positive θ being anti-clockwise
    fvec3 walk_command = 7;
    /// The robots current state
    State state = 7;

    /// extensions is only available in proto2
    extensions 100 to 1000;

    /// in proto3 we could do something like
    repeated google.protobuf.Any extras = 100;
}

EDIT: Alternatively, rather than using Any or extensions, we could just use parsePartial.

7

Thanks all for your contribution!

I will try to separate between elements where we have a consensus and elements which are still discussed. I might have misinterpreted some messages so do not hesitate to correct me.

Consensus reached

  • Having a common protocol is a good point
  • Using a protobuf based protocol is preferable over using Mitecom
  • Protocol should not be enforced, it should rather be easy and interesting to use it
  • Encryption of the messages would make implementation more difficult, it might be used in long-term, but not in short term

Controversial points

  • Should the proposed protocol be minimal or should it be exhaustive
    • Simplified protocol makes entry cost loser
    • Exhaustive protocol allows more option for strategies and extraction of high-level stats

Personnal comments on Bidski proposal

I do understand that my proposal might seems too exhaustive at first sight and difficult to apprehend at first. However, it is not mandatory to use all of the fields, and most of them are the results of elements identified as important when discussing with other teams.

The use of the any replaces advantageously the optional bytes free_field in my proposal

I think that the best solution would be to get the best side of the two proposals by using the following scheme:

  • We propose two different protocols: simple and exhaustive
    • simple: based on Bidski’s proposal with a few important fields added
      • Timestamp synchronizing all the messages require to establish a common clock
      • Ball velocity almost required since teamplay is more dynamical
      • Opponents required to assess the quality of opponent detection for rule evolution
      • TargetPose: WalkCommand might be too much short-sighted, target pose on field gives a better understanding
    • exhaustive : extends the simple version using elements from my proposal with key modifications
      • All messages could be gathered in the same file to make it easier to understand
      • Simple examples of setup could be shown

If my proposal sounds acceptable, the first thing we should need to do is implement a common proposal quickly. I think that in any case, the protocol should be available before end of December in order to allow teams to adapt.

8

In response to Ludovics further proposal, the NUbots feel that the simple protocol is a good idea and that this should be made the official standard. We feel that the exhaustive protocol should be an unofficial source of extra message content. Aiming for easy adoption should be paramount at this initial stage.

Based on Ludovics proposal, we have edited our initial proposal to the following

package robocup.humanoid;
import "google/protobuf/timestamp.proto";

/// A column vector of three floats
message fvec3 {
    float x = 1;
    float y = 2;
    float z = 3;
}

/// A matrix of three vectors
/// Specified as column vectors
message fmat3 {
    fvec3 x = 1;
    fvec3 y = 2;
    fvec3 z = 3;
}

/// The detected team colour of the robot
enum Team {
    UNKNOWN = 0;
    BLUE = 1;
    RED = 2;
}

/// The current playing state of the robot
enum State {
    UNKNOWN = 0;
    UNPENALISED = 1;
    PENALISED = 2;
}

message Robot {
    /// The position of the robot on the field according to the following convention
    /// x meters along the field with 0 at the centre of the field and positive towards opponent goals
    /// y meters across the field with 0 at the centre of the field and positive to the left
    /// θ orientation of the robot (anti-clockwise from the positive x axis from above the field)
    fvec3 position = 1;

    /// The covariance measure of the robots [x, y, θ] values or 0 if not available
    fmat3 covarinace = 2;

    /// Robot team colour, if known
    Team team = 3;
}
message Ball {
    /// The position of the ball on the field according to the following convention
    /// x meters along the field with 0 at the centre of the field and positive towards opponent goals
    /// y meters across the field with 0 at the centre of the field and positive to the left
    /// z meters above the field with 0 at the surface of the field and positive up
    fvec3 position = 1;

    /// The velocity of the ball in meters/second using the same convention as for position
    /// Set to 0 if not available
    fvec3 velocity = 2;

    /// The covariance measure of the balls [x, y, z] position values or 0 if not available
    fmat3 covariance = 3;
}
message Message {
    /// Timestamp of message creation
    google.protobuf.Timestamp timestamp = 1;
    /// ID of the team that is sending this message
    uint32 team_id = 2;
    /// ID of the player that is sending this message
    uint32 player_id = 3;

    /// Position, orientation, and covariance of the player on the field
    Robot current_pose = 4;

    /// Position, velocity, and covariance of the ball on the field
    Ball ball = 5;
    
    /// Position, orientation, and covariance of detected robots on the field
    repeated Robot others = 6;

    /// The current walk speed of the robot in it's local [x, y, θ] coordinates
    /// positive x forwards
    /// positive y strafing to the left
    /// positive θ being anti-clockwise from the positive x axis above the field
    fvec3 walk_command = 7;

    /// Position and orientation of the players target pose on the field
    Robot target_pose = 8;

    /// The robots current state
    State state = 9;

    /// All IDs less than 101 are reserved for official use only
}

In terms of extensions we should officially reserve the first 100 IDs (protobuf doesn’t let us do this, so it will have to be an agreed upon rule). All unofficial extensions can then be added from IDs 101 onwards. The game controller doesn’t need to do anything special, it can just use the official protobuf file to decode the message as normal, any unspecified IDs will be ignored by protobuf (we were mistaken in our parsePartial comment earlier).

This method avoids the headaches of using Anys and free form bytes fields (you can use whatever types you want without needing to pack them into some unspecified format).

We feel that UDP communication with multicast groups should be a must for the standard protocol. This will allow for ease of use and implementation (for both the teams and the game controller).

We would also like to propose that the game controller be updated to use protobuf for all communications (this may need to be a discussion for another time though).

9

I think this should include the player_id

We could then remove team_id and player_id here and just have a field for the robot itself.

10

Thanks for your effort Bidski, I have the feeling that we are converging toward something and I agree that we should aim for easy adoption at this stage.

It seems quite clear to me that you have more experience with protobuf than me, therefore, I think that relying on your proposal makes sense.

However, there are a few points which I still think might be discussed/improved:

  • It might be interesting to have timestamp of both, emission of the message and reception of the message (filled by the computer receiving the message). This can easily be used to improve sync of different streams after the games
  • Including player_id for the Robot message (which is likely to be set to 0, meaning unknown)
  • I think that adding kick target (direction + distance) when a robot intend to kick is a simple but essential information for teamplay since it allows player receiving the pass to receive it. Do you think it’s too much?
  • Finally for the convention proposed, it is a ‘team based convention’ (x points toward the opposite goal). In my proposal, I preferred a ‘global convention’ (x points right of the referee table). In my experience, ‘team based convention’ makes it cumbersome to draw any information which is global, because it always need to be inverted at any point (and ground truth always has a global referential), ‘global convention’ just forces team to invert the coordinate when they receive the message. Do you have any strong argument to keep it ‘team based’ or is it just a matter of preference?

I think it would be great if you can manage to provide a git repository in the next few days? This would allow to provide the link for the next rule discussion. Examples of usage or extension of the protocol would be an awesome addition.

11

Here is the link to our repository containing some example code and our proposal. The repository currently only provides an example for how the extensions to the protocol could be used. For extensions that are just primitive data types they will default to 0 if not specified. If the extension is a a message type then you can check for its existance (when decoding) by using the .has_x() API. We can add an example of this if needed.

The repository currently does not include any examples on the multicast networking that we suggested, if this is needed then we can add that as well.

We have moved player_id into the Robot message and removed player_id and team_id from Message as suggested by @Bitbots_Jasper.

We don’t understand why the receiving timestamp should be included as part of the protocol as the teams/players will never be sending this. The receiving computer can, of course, take note of the time it receives messages, but this is logic that is external to the protocol itself.

We have added the kick target as @Ludovic has suggested, but we feel that if we are seriously considering adding this information then we really need to enforce team separation using either the multicast method that we have previously suggested or some other method. This is the sort of information that would make it very easy for teams to circumvent autonomous behaviour and just move to intercept the broadcasted kick targets from the other team.

Finally for the convention proposed, it is a ‘team based convention’ (x points toward the opposite goal). In my proposal, I preferred a ‘global convention’ (x points right of the referee table). In my experience, ‘team based convention’ makes it cumbersome to draw any information which is global, because it always need to be inverted at any point (and ground truth always has a global referential), ‘global convention’ just forces team to invert the coordinate when they receive the message. Do you have any strong argument to keep it ‘team based’ or is it just a matter of preference?

The ‘team based’ convention is a preference for our team. It simplifies our calculations and we don’t need to rely on a ‘global’ reference point existing. So as long as we know which side of the field is “ours” we always kick in the +x direction. Basically, whichever side of the field we place the robots on becomes “our” side, so no extra configuration is needed.

If we make the coordinate convention a ‘global’ convention then we will need to provide a consistent method of expressing the ‘global’ reference point to all players, the game controller I think would need to be responsible for this.

Whichever way we do it, there will need to be an transformation performed either by the game controller, or by each player in order to get into the correct coordinate frame. We have no strong opinions about which convention should be used. For the time being, I have left the coordinate as it was in our previous proposal, if it is deemed that we should use a ‘global’ convention then we can update that.

12

Thank you very much for your work.

I think that your approach is really satisfying and that it can be more easily spread.

Regarding the receiving timestamp, it is for recording of the games if we want to make public datasets. What often happens is that the clock of the players has an offset with the clock for the gamecontroller. Saving emission timestamps (in emitter referential) and reception timestamps (in receiver referential) when logging is an easyway to diagnose this kind of issues and sometimes to recover it. But I’m fine to have it as an additional data field for unofficial repositories.

Regarding the need to enforce the team separation, I agree that it is something required (especially), but position of the robots themselves can already be used to have a cheap opponent detector, so the problem was already present. Using the multicast method might be a satisfying option. We were hoping to have someone taking in charge a proposal for monitoring network and filtering messages in the last call for projects for league development. If you can propose something regarding this next year, I have no doubt that this would be strongly supported by the TC.

Regarding the ‘team-based convention’ that’s fine by me. It’s true that it will be easier for teams. On our side, we used the position of the referee table to set the global referential. I understand the fact that pushing the ‘task’ of doing the transform in external tools is reasonable, because it leaves it easier for most people.