Based on the AR Foundation Samples reference implementation.
You want two (or more) iPhones to see the same virtual objects in the same real-world positions. Player A places a virtual cube on a table. Player B, looking at the same table from across the room, sees that cube in the correct spot.
This requires solving two sub-problems:
ARKit solves problem 1. MultipeerConnectivity solves problem 2. The sample code wires them together.
Each AR device creates its own world coordinate system the moment the AR session starts. The origin (0, 0, 0) is wherever the phone happened to be at startup. Two phones in the same room will have completely different origins, different axis orientations, different scales of drift.
This means if Phone A says "I put a cube at position (1.2, 0.5, -3.0)", that position is meaningless to Phone B. Phone B's (1.2, 0.5, -3.0) is a completely different point in the real world.
To share AR content, the devices need to establish a common reference frame -- a shared understanding of how their individual coordinate systems map to the same physical space. ARKit provides two mechanisms for this:
| Mechanism | How it works | Best for |
|---|---|---|
| ARCollaborationData (real-time) | Continuous stream of visual feature data exchanged between devices | Live collaborative sessions |
| ARWorldMap (snapshot) | One device serializes its entire world understanding to a file; another device loads it | Persistence, async sharing |
When you enable collaboration on an ARKitSessionSubsystem, ARKit starts producing ARCollaborationData objects. These are opaque blobs that contain:
You don't need to understand the internal format. You treat it as a black box: serialize it, send it over the network, deserialize it on the other side, and feed it to the other device's AR session. ARKit handles the rest internally.
Each piece of collaboration data has a priority:
This distinction maps directly to network send modes (explained in section 10).
In Unity/C# (from CollaborativeSession.cs):
// Get the ARKit-specific subsystem
var subsystem = m_ARSession.subsystem as ARKitSessionSubsystem;
// Check support (requires iOS 13+)
if (ARKitSessionSubsystem.supportsCollaboration)
{
// This single flag tells ARKit to start producing collaboration data
subsystem.collaborationRequested = true;
}
Once enabled, the subsystem starts queuing ARCollaborationData objects. You dequeue them each frame:
while (subsystem.collaborationDataCount > 0)
{
using (var collaborationData = subsystem.DequeueCollaborationData())
{
// collaborationData.priority tells you Critical vs Optional
// collaborationData.ToSerialized() gives you the bytes to send
}
}
And when you receive data from a remote peer:
var collaborationData = new ARCollaborationData(receivedBytes);
if (collaborationData.valid)
{
subsystem.UpdateWithCollaborationData(collaborationData);
}
That's it. ARKit internally merges the remote data into the local session, aligning coordinate systems behind the scenes.
MultipeerConnectivity (MC) is Apple's framework for peer-to-peer communication between nearby Apple devices. It works over Wi-Fi and Bluetooth simultaneously, with no server needed. It handles:
MC uses Bonjour (Apple's zero-configuration networking protocol, based on mDNS) to advertise and discover services on the local network.
There are two roles that each device plays simultaneously:
MCNearbyServiceAdvertiser) -- Broadcasts: "I exist and I'm running service type ar-collab"MCNearbyServiceBrowser) -- Listens: "Is anyone out there running service type ar-collab?"The service type is a string identifier (max 15 chars, lowercase ASCII + hyphens + numbers) that acts like a channel name. Only devices advertising and browsing the same service type will find each other.
invitationHandler(YES, m_Session))MCSession is established between the two peersSince both devices are simultaneously advertising AND browsing, both will try to invite each other. MultipeerConnectivity handles the deduplication -- you won't get duplicate connections.
| Term | What it is |
|---|---|
MCPeerID |
A unique identity for a device in the session. Created with a display name (the sample uses SystemInfo.deviceName). |
MCSession |
The live connection between peers. You send data through this. |
MCNearbyServiceAdvertiser |
Broadcasts this device's presence using a service type string. |
MCNearbyServiceBrowser |
Scans for other devices advertising the same service type. |
MCSessionSendDataMode |
Either .reliable (TCP-like: guaranteed, ordered) or .unreliable (UDP-like: fast, may drop). |
| Service Type | A string like "ar-collab" that acts as a channel. Devices must match to find each other. |
The sample has three layers stacked on top of each other:
┌───────────────────────────────────────────────────┐
│ Layer 3: Unity MonoBehaviour │
│ CollaborativeSession.cs │
│ - Dequeues ARCollaborationData from ARKit │
│ - Sends it via MCSession │
│ - Receives remote data via MCSession │
│ - Feeds it back to ARKit │
└────────────────────┬──────────────────────────────┘
│ calls C# methods
┌────────────────────▼──────────────────────────────┐
│ Layer 2: C# P/Invoke Bridge │
│ MCSession.cs, NSData.cs, NSString.cs, etc. │
│ - C# structs wrapping native Obj-C pointers │
│ - [DllImport] declarations mapping to C functions │
│ - IDisposable for proper memory management │
└────────────────────┬──────────────────────────────┘
│ calls native C functions
┌────────────────────▼──────────────────────────────┐
│ Layer 1: Native Objective-C Plugin │
│ MultipeerDelegate.m + C Bridge files │
│ - Creates MCSession, MCPeerID, etc. │
│ - Handles all delegate callbacks │
│ - Manages thread-safe received data queue │
│ - C bridge functions for P/Invoke compatibility │
└───────────────────────────────────────────────────┘
[DllImport]).MultipeerDelegate.h / MultipeerDelegate.mThis is a single Objective-C class that implements three Apple delegate protocols:
@interface MultipeerDelegate : NSObject
<MCSessionDelegate, // Handles session events (data received, peer state changes)
MCNearbyServiceAdvertiserDelegate, // Handles incoming invitations
MCNearbyServiceBrowserDelegate> // Handles discovered peers
initWithName:serviceType:)Creates everything needed for a MultipeerConnectivity session:
m_PeerID = [[MCPeerID alloc] initWithDisplayName: name];
m_Session = [[MCSession alloc] initWithPeer:m_PeerID
securityIdentity:nil
encryptionPreference:MCEncryptionRequired];
m_Session.delegate = self;
m_ServiceAdvertiser = [[MCNearbyServiceAdvertiser alloc] initWithPeer:m_PeerID
discoveryInfo:nil
serviceType:serviceType];
m_ServiceBrowser = [[MCNearbyServiceBrowser alloc] initWithPeer:m_PeerID
serviceType:serviceType];
Key details:
- securityIdentity:nil -- No certificate-based authentication. (For a game jam, this is fine.)
- MCEncryptionRequired -- Data is encrypted over the wire. This is the default Apple recommends.
- discoveryInfo:nil -- No metadata is broadcast during discovery. You could add key-value pairs here to filter peers.
When enabled is set to true:
[m_ServiceAdvertiser startAdvertisingPeer]; // Start broadcasting "I'm here"
[m_ServiceBrowser startBrowsingForPeers]; // Start scanning for others
When set to false, advertising and browsing stop, and the received data queue is cleared.
When a peer is found by the browser, it immediately invites them:
- (void)browser:(MCNearbyServiceBrowser *)browser foundPeer:(MCPeerID *)peerID ...
{
[browser invitePeer:peerID toSession:m_Session withContext:nil timeout:10];
}
When an invitation is received by the advertiser, it's automatically accepted:
- (void)advertiser:(MCNearbyServiceAdvertiser *)advertiser
didReceiveInvitationFromPeer:(MCPeerID *)peerID ...
invitationHandler:(void (^)(BOOL, MCSession *))invitationHandler
{
invitationHandler(YES, m_Session); // Always accept
}
This creates a fully automatic mesh -- any device running the same service type will connect to all others without user intervention.
When data arrives from a peer, MultipeerConnectivity calls didReceiveData: on a background thread. Unity's Update loop runs on the main thread. The bridge between them is a @synchronized queue:
// Called on background thread by MC framework
- (void)session:(MCSession *)session didReceiveData:(NSData *)data fromPeer:(MCPeerID *)peerID
{
@synchronized (m_Queue) {
[m_Queue addObject:data]; // Thread-safe enqueue
}
}
// Called on main thread by Unity via P/Invoke
- (NSData*)dequeue
{
@synchronized (m_Queue) {
NSData* data = [m_Queue objectAtIndex:0];
[m_Queue removeObjectAtIndex:0];
return data; // Thread-safe dequeue
}
}
MultipeerDelegate-C-Bridge.m)Unity's [DllImport] can only call C functions, not Objective-C methods. The bridge file converts between the two:
// Unity C# calls this C function via [DllImport("__Internal")]
ManagedMultipeerDelegate UnityMC_Delegate_initWithName(void* name, void* serviceType)
{
// Which calls the Objective-C method
MultipeerDelegate* delegate = [[MultipeerDelegate alloc]
initWithName:(__bridge NSString*)name
serviceType:(__bridge NSString*)serviceType];
return (__bridge_retained void*)delegate;
}
__bridge_retained transfers ownership of the Objective-C object to the C# side (prevents ARC from deallocating it). The C# side later calls CFRelease when it's done with the object.
MCSession.csA C# struct that holds a single IntPtr -- a pointer to the native MultipeerDelegate Objective-C object:
[StructLayout(LayoutKind.Sequential)]
public struct MCSession : IDisposable, IEquatable<MCSession>
{
IntPtr m_Ptr; // Points to the native MultipeerDelegate
Each method maps directly to a native C function:
// C# method you call from Unity
public void SendToAllPeers(NSData data, MCSessionSendDataMode mode)
{
using (var error = SendToAllPeers(this, data, mode))
{
if (error.Valid)
throw error.ToException();
}
}
// The native function it calls
[DllImport("__Internal", EntryPoint="UnityMC_Delegate_sendToAllPeers")]
static extern NSError SendToAllPeers(MCSession self, NSData data, MCSessionSendDataMode mode);
The EntryPoint string maps to the C function name in the bridge file. "__Internal" means "this function is statically linked into the app binary" (which is how iOS plugins work -- no dynamic libraries).
NSData.csWraps Apple's NSData (an immutable byte buffer). The critical method:
public static unsafe NSData CreateWithBytesNoCopy(NativeSlice<byte> bytes)
NoCopy is important -- it wraps the existing byte buffer without allocating new memory. Since ARCollaborationData.ToSerialized() returns a NativeSlice<byte>, this avoids a copy when sending. The using statement in the caller ensures the NSData wrapper doesn't outlive the source buffer.
MCSessionSendDataMode.csA simple enum:
public enum MCSessionSendDataMode
{
Reliable, // Guaranteed delivery, in order (like TCP)
Unreliable // Fire-and-forget, may drop (like UDP)
}
All native wrapper structs implement IDisposable and call CFRelease:
public void Dispose() => NativeApi.CFRelease(ref m_Ptr);
This releases the Objective-C object that was retained by __bridge_retained in the C bridge. The sample uses using blocks everywhere to ensure cleanup:
using (var serializedData = collaborationData.ToSerialized())
using (var data = NSData.CreateWithBytesNoCopy(serializedData.bytes))
{
m_MCSession.SendToAllPeers(data, ...);
}
// Both serializedData and data are disposed here
CollaborativeSession.csThis is the only file you interact with as a Unity developer. It's a MonoBehaviour that goes on the same GameObject as your ARSession.
[RequireComponent(typeof(ARSession))]
public class CollaborativeSession : MonoBehaviour
One serialized field in the Inspector:
[SerializeField]
string m_ServiceType; // e.g., "ar-collab"
This is the Bonjour service name. All devices must use the same string to find each other. Rules: - Max 15 characters - ASCII lowercase letters, numbers, hyphens only - Must not start or end with a hyphen
Awake -- Creates the native MC session:
void Awake()
{
m_ARSession = GetComponent<ARSession>();
m_MCSession = new MCSession(SystemInfo.deviceName, m_ServiceType);
}
OnEnable -- Starts everything:
void OnEnable()
{
subsystem.collaborationRequested = true; // Tell ARKit to produce collaboration data
m_MCSession.Enabled = true; // Start advertising + browsing
}
OnDisable -- Stops everything:
void OnDisable()
{
m_MCSession.Enabled = false;
subsystem.collaborationRequested = false;
}
OnDestroy -- Frees native memory:
void OnDestroy()
{
m_MCSession.Dispose();
}
Every frame, Update() does two things:
1. Send outgoing collaboration data:
while (subsystem.collaborationDataCount > 0)
{
using (var collaborationData = subsystem.DequeueCollaborationData())
{
if (m_MCSession.ConnectedPeerCount == 0)
continue; // No one to send to
using (var serializedData = collaborationData.ToSerialized())
using (var data = NSData.CreateWithBytesNoCopy(serializedData.bytes))
{
m_MCSession.SendToAllPeers(data,
collaborationData.priority == ARCollaborationDataPriority.Critical
? MCSessionSendDataMode.Reliable
: MCSessionSendDataMode.Unreliable);
}
}
}
2. Receive incoming collaboration data:
while (m_MCSession.ReceivedDataQueueSize > 0)
{
using (var data = m_MCSession.DequeueReceivedData())
using (var collaborationData = new ARCollaborationData(data.Bytes))
{
if (collaborationData.valid)
{
subsystem.UpdateWithCollaborationData(collaborationData);
}
}
}
That's the entire networking loop. ARKit handles all the spatial alignment internally.
Here's exactly what happens when Device A places a virtual object and Device B sees it:
Device A Device B
──────── ────────
1. ARKit detects features,
produces ARCollaborationData
(priority: Critical for anchors,
Optional for feature updates)
2. CollaborativeSession.Update()
dequeues the collaboration data
3. data.ToSerialized() → NativeSlice<byte>
(opaque binary blob)
4. NSData.CreateWithBytesNoCopy()
wraps bytes in Obj-C NSData object
5. MCSession.SendToAllPeers()
→ C bridge → Obj-C sendData:toPeers:
→ Apple MC framework sends over
Wi-Fi/Bluetooth
6. MC framework receives data
on background thread
7. MultipeerDelegate.didReceiveData
enqueues to @synchronized queue
8. CollaborativeSession.Update()
checks ReceivedDataQueueSize > 0
9. DequeueReceivedData() returns NSData
10. new ARCollaborationData(bytes)
deserializes the blob
11. subsystem.UpdateWithCollaborationData()
feeds it to ARKit
12. ARKit internally merges the data:
- Matches visual features to its
own camera observations
- Computes the transform between
the two coordinate systems
- Resolves shared anchors into
local coordinates
13. Shared anchors now appear in
Device B's ARAnchorManager
with correct local positions
Step 12 is where the magic is. ARKit uses visual-inertial odometry data from both devices to find overlapping feature points. Once it finds enough matches, it computes a rigid transformation (rotation + translation) that maps Device A's coordinate system into Device B's. This happens incrementally -- accuracy improves as more data is exchanged and as both devices observe more of the shared environment.
Both devices must be observing some of the same physical space for this to work. If they're in different rooms, alignment will never converge.
The sample maps ARKit's priority levels to network send modes:
| ARKit Priority | Network Mode | Behavior |
|---|---|---|
Critical |
Reliable |
Guaranteed delivery, in-order. Retransmits on failure. Higher latency. |
Optional |
Unreliable |
Fire-and-forget. No retransmission. Lowest latency. May be dropped. |
Critical data includes anchors and key spatial reference points. If a critical packet is lost, the receiving device will have gaps in its spatial understanding -- anchors might never appear, or coordinate alignment might fail. These are infrequent but important, so the overhead of reliable delivery is acceptable.
Optional data is incremental feature point updates produced nearly every frame. Sending all of these reliably would: 1. Overwhelm the connection with retransmission overhead 2. Introduce latency as packets queue up waiting for acknowledgment 3. Be pointless, since the next frame's data supersedes the current frame's
Sending them unreliably means "send it, move on, the next one is coming in 16ms anyway."
If you're sending your own game data (player positions, actions, etc.) on top of collaboration data, apply the same principle:
- **Game state that must be correct** (scores, item pickups, game-over events) → Reliable
- **Ephemeral state** (player position every frame, cursor location) → Unreliable
Starting with iOS 14, apps that use local networking (Bonjour/mDNS) must declare:
1. Why they need local network access (NSLocalNetworkUsageDescription)
2. Which Bonjour services they will use (NSBonjourServices)
Without these entries, MultipeerConnectivity silently fails -- no crash, no error, just no peers found.
The sample includes CollaborationBuildPostprocessor.cs, which runs automatically during the Unity iOS build process:
class CollaborationBuildProcessor : IPostprocessBuildWithReport, IPreprocessBuildWithReport
It does two things:
Pre-build: Scans all scenes for CollaborativeSession components and collects their serviceType values.
Post-build: Modifies Info.plist in the Xcode project:
// Adds the permission prompt string
root["NSLocalNetworkUsageDescription"] = new PlistElementString("Collaborative Session");
// For each service type (e.g., "ar-collab"), registers both TCP and UDP Bonjour services
bonjourServices.AddString("_ar-collab._tcp");
bonjourServices.AddString("_ar-collab._udp");
Add these manually to your Info.plist:
<key>NSLocalNetworkUsageDescription</key>
<string>This app uses the local network to find nearby players for collaborative AR.</string>
<key>NSBonjourServices</key>
<array>
<string>_your-service._tcp</string>
<string>_your-service._udp</string>
</array>
Replace your-service with whatever string you set as your service type.
The sample also includes ARWorldMapController.cs, which demonstrates a different approach: serializing an entire ARWorldMap to disk.
An ARWorldMap is a snapshot of a device's entire spatial understanding at a point in time: - All detected feature points - All anchors - The relationship between the device's coordinate system and the physical environment
Saving (Device A):
var request = sessionSubsystem.GetARWorldMapAsync();
// ... wait for completion ...
var worldMap = request.GetWorldMap();
var data = worldMap.Serialize(Allocator.Temp);
// Write data to file
Loading (Device B):
var data = /* read bytes from file */;
ARWorldMap.TryDeserialize(data, out ARWorldMap worldMap);
sessionSubsystem.ApplyWorldMap(worldMap);
When Device B applies Device A's world map, ARKit attempts to relocalize -- it looks at what its camera currently sees and tries to match it against the feature points in the loaded map. Once it finds a match, both devices share the same coordinate system.
Before saving, check sessionSubsystem.worldMappingStatus:
| Status | Meaning |
|---|---|
NotAvailable |
Not enough data yet. Don't save. |
Limited |
Some spatial data, but not enough for reliable relocation. |
Extending |
Good data, actively expanding. Save is possible but may be incomplete. |
Mapped |
Best quality. The visible area is well-mapped. Best time to save. |
| ARCollaborationData | ARWorldMap | |
|---|---|---|
| Communication | Real-time, continuous stream | One-time snapshot |
| Network | Requires live P2P connection | Can be transferred any way (file, cloud, AirDrop) |
| Latency | Low -- data flows every frame | High -- must save, transfer, load, relocalize |
| Persistence | Session only; lost when app closes | Can be saved to disk and reused later |
| Best for | Live multiplayer sessions (game jam!) | Persistent AR (leave content, come back later) |
| iOS requirement | iOS 13+ | iOS 12+ |
For a game jam: use ARCollaborationData. It's real-time, automatic, and requires almost no code beyond what's in the sample.
Scene setup: Create a scene with AR Session and AR Session Origin (or XR Origin) as usual.
Add CollaborativeSession: Attach it to the same GameObject as your ARSession. Set a service type string in the Inspector (e.g., "gamejam25").
Copy the Multipeer plugin: You need the entire Assets/Scripts/Runtime/Multipeer/ folder. This is the native bridge code. Without it, there's nothing to link MCSession.cs to.
Copy the build postprocessor: CollaborationBuildPostprocessor.cs goes in an Editor/ folder. This handles Info.plist automatically.
Place shared content using ARAnchorManager: When a player taps to place an object, create an ARAnchor at that position. ARKit's collaboration system automatically shares anchors. On the remote device, the anchor will appear in ARAnchorManager.trackablesChanged once the coordinate systems are aligned.
The collaboration data handles spatial alignment. For game-specific data (player actions, scores, game state), you have two options:
Option A: Piggyback on the existing MCSession
The sample's MCSession exposes SendToAllPeers. You could extend it to send custom game messages alongside collaboration data. You'd need to:
- Add a message type header (1 byte) to distinguish your data from collaboration data
- Modify the receive side to check the header and route accordingly
- Be careful about serialization (use BinaryWriter/BinaryReader or a library like MessagePack)
Option B: Use a separate networking layer
Use Unity Netcode for GameObjects, Mirror, Photon, or any other networking solution for game logic, while keeping MultipeerConnectivity solely for AR collaboration data. This is cleaner but adds a dependency.
NSBonjourServices or NSLocalNetworkUsageDescription silently kills discovery on iOS 14+..m files are included in the Xcode build. Check that their import settings in Unity have "iOS" checked as the target platform.Unity.XR.Samples.Multipeer.asmdef is set to iOS only and allows unsafe code. Make sure your project settings allow unsafe code for iOS builds.using statements and NoCopy variants everywhere to minimize allocations. If you modify the code, maintain this discipline.sendData:toPeers:m_Session.connectedPeers). This should work out of the box for 3+ devices.All files in the sample related to multiplayer AR, organized by role:
| File | Purpose |
|---|---|
Assets/Scenes/ARKit/ARCollaborationData/CollaborativeSession.cs |
Main script. Wires ARKit collaboration data to MCSession. |
Assets/Scenes/ARKit/ARCollaborationData/CollaborationNetworkingIndicator.cs |
Debug UI showing data flow (green/red indicators). |
| File | Purpose |
|---|---|
Assets/Scripts/Runtime/Multipeer/MCSession.cs |
C# wrapper for the native MultipeerDelegate. |
Assets/Scripts/Runtime/Multipeer/MCSessionSendDataMode.cs |
Enum: Reliable / Unreliable. |
Assets/Scripts/Runtime/Multipeer/NSData.cs |
C# wrapper for NSData (byte buffers). |
Assets/Scripts/Runtime/Multipeer/NSString.cs |
C# wrapper for NSString. |
Assets/Scripts/Runtime/Multipeer/NSError.cs |
C# wrapper for NSError. |
Assets/Scripts/Runtime/Multipeer/NSErrorException.cs |
Converts NSError to C# Exception. |
Assets/Scripts/Runtime/Multipeer/NativeApi.cs |
CFRelease wrapper for memory management. |
| File | Purpose |
|---|---|
Assets/Scripts/Runtime/Multipeer/NativeCode/MultipeerDelegate.h |
Obj-C header. |
Assets/Scripts/Runtime/Multipeer/NativeCode/MultipeerDelegate.m |
Obj-C implementation: MCSession + discovery + thread-safe queue. |
Assets/Scripts/Runtime/Multipeer/NativeCode/MultipeerDelegate-C-Bridge.m |
C functions that call Obj-C methods (for P/Invoke). |
Assets/Scripts/Runtime/Multipeer/NativeCode/NSData-C-Bridge.m |
C bridge for NSData operations. |
Assets/Scripts/Runtime/Multipeer/NativeCode/NSString-C-Bridge.m |
C bridge for NSString operations. |
Assets/Scripts/Runtime/Multipeer/NativeCode/NSError-C-Bridge.m |
C bridge for NSError operations. |
| File | Purpose |
|---|---|
Assets/Scenes/ARKit/ARCollaborationData/Editor/CollaborationBuildPostprocessor.cs |
Auto-adds Bonjour service entries to Info.plist during iOS build. |
| File | Purpose |
|---|---|
Assets/Scripts/Runtime/Multipeer/Unity.XR.Samples.Multipeer.asmdef |
Marks the Multipeer folder as iOS-only, enables unsafe code. |
| File | Purpose |
|---|---|
Assets/Scenes/ARKit/ARCollaborationData/ARCollaborationData.unity |
The sample scene with everything wired up. |
| File | Purpose |
|---|---|
Assets/Scripts/Runtime/ARWorldMapController.cs |
Save/load ARWorldMap to disk (non-realtime approach). |