Unity 2.6 Flash Webcam to Unity Code
Lab: Streaming flash webcam via sockets to Unity3d 2.6
Finally after a year, here is the code I used to Stream the Flash webcam data to my Unity app via sockets, remember that both plug-ins will have to sit on your webpage for this to work and you will have to build in Unity 2.6 unless you deal with the new Socket Security Policy in Unity v3, shouldn’t be too hard.
Link to an online demo is here. Another reason why this code is a bit defunct is that Unity are building an export into Flash Molehill, so you should natively be able to pass bitmap data to a texture 2D with Flash Actionscript shortly, roll on the tedious amounts of “new” augmented reality solutions, however if you get a Unity build working with SURF at a good speed the let me know!
I can’t verify this code i’m mid-project and had loads of requests, so I’ve literally found the code I think worked and pasted here, will update later if found not to work. AS3 code and the Unity Code below:- (Remember if you find this usefull Donate it helps make future code freely available)
ACTIONSCRIPT AS3
import flash.display.*;
import flash.events.*;
import flash.geom.*;
import flash.media.*;
import flash.net.*;
import flash.utils.*;
import flash.events.IEventDispatcher;
var videoDisplay:Video;
var sendstring:String;
var firstrun:Boolean;
var WEBCAM_WIDTH:int;
var TRY_CONNECT_FREQUENCY:int;
var SERVER_HOST:String;
var camera:Camera;
var WEBCAM_HEIGHT:int;
var SendTimer:Timer;
var getconnection:Timer;
var mydataout:ByteArray;
var mydatain:ByteArray;
var SERVER_PORT:int;
var mySocket:Socket;
WEBCAM_WIDTH = 320;
WEBCAM_HEIGHT = 240;
var mybitmp:BitmapData = new BitmapData(WEBCAM_WIDTH,WEBCAM_HEIGHT);
camera = Camera.getCamera();
if (camera != null) {
camera.addEventListener(ActivityEvent.ACTIVITY, activityHandler);
camera.setMode(WEBCAM_WIDTH, WEBCAM_HEIGHT, camera.fps);
videoDisplay = new Video(WEBCAM_WIDTH, WEBCAM_HEIGHT);
videoDisplay.attachCamera(camera);
} else {
trace("You need a camera.");
stop();
}
camera.setMode(WEBCAM_WIDTH, WEBCAM_HEIGHT, camera.fps);
videoDisplay = new Video(WEBCAM_WIDTH, WEBCAM_HEIGHT);
videoDisplay.attachCamera(camera);
addChild(videoDisplay);
firstrun = true;
SERVER_HOST = "localhost";
SERVER_PORT = 8080;
TRY_CONNECT_FREQUENCY = 60;
sendstring = "1";
function closeConnection(event:Event) : void {
if (SendTimer) {
SendTimer.stop();
SendTimer.removeEventListener(TimerEvent.TIMER, sendData);
}
if (mySocket.connected) {
mySocket.flush();
mySocket.close();
}
getconnection.start();
}
function openConnection(event:Event) : void {
mySocket.connect(SERVER_HOST, SERVER_PORT);
mybitmp.draw(videoDisplay);
var memoryHash:String;
try
{
ByteArray(mybitmp);
}
catch (e:Error)
{
memoryHash = String(e).replace(/.*([@|\$].*?) to .*$/gi, '$1');
trace(memoryHash);
}
finally {
trace(memoryHash);
}
}
function initializesocket(event:Event) : void {
getconnection.stop();
SendTimer = new Timer(TRY_CONNECT_FREQUENCY);
SendTimer.addEventListener(TimerEvent.TIMER, sendData);
SendTimer.start();
}
function sendData(event:Event) : void {
if (!mySocket.connected) {
closeConnection(null);
}
mydataout = new ByteArray();
var pixwid:int = 160;
var pixhgt:int = 120;
var mscale:Number = pixwid / WEBCAM_WIDTH;
var mymatrix:Matrix = new Matrix();
mymatrix.scale(mscale,mscale);
//var mybitmp:BitmapData = new BitmapData(pixwid,pixhgt);
mybitmp.draw(videoDisplay,mymatrix);
var rect:Rectangle = new Rectangle(0, 0, pixwid, pixhgt);
mydataout = mybitmp.getPixels(rect);
mySocket.writeBytes(mydataout);
mySocket.flush();
}
function recieveData(event:ProgressEvent) {
closeConnection(null);
}
function ioErrorHandler(event:IOErrorEvent) : void {
closeConnection(null);
}
function securityErrorHandler(event:SecurityErrorEvent) : void {
closeConnection(null);
}
function activityHandler(event:ActivityEvent):void {
trace("activityHandler: " + event);
}
import flash.events.*;
import flash.geom.*;
import flash.media.*;
import flash.net.*;
import flash.utils.*;
import flash.events.IEventDispatcher;
var videoDisplay:Video;
var sendstring:String;
var firstrun:Boolean;
var WEBCAM_WIDTH:int;
var TRY_CONNECT_FREQUENCY:int;
var SERVER_HOST:String;
var camera:Camera;
var WEBCAM_HEIGHT:int;
var SendTimer:Timer;
var getconnection:Timer;
var mydataout:ByteArray;
var mydatain:ByteArray;
var SERVER_PORT:int;
var mySocket:Socket;
WEBCAM_WIDTH = 320;
WEBCAM_HEIGHT = 240;
var mybitmp:BitmapData = new BitmapData(WEBCAM_WIDTH,WEBCAM_HEIGHT);
camera = Camera.getCamera();
if (camera != null) {
camera.addEventListener(ActivityEvent.ACTIVITY, activityHandler);
camera.setMode(WEBCAM_WIDTH, WEBCAM_HEIGHT, camera.fps);
videoDisplay = new Video(WEBCAM_WIDTH, WEBCAM_HEIGHT);
videoDisplay.attachCamera(camera);
} else {
trace("You need a camera.");
stop();
}
camera.setMode(WEBCAM_WIDTH, WEBCAM_HEIGHT, camera.fps);
videoDisplay = new Video(WEBCAM_WIDTH, WEBCAM_HEIGHT);
videoDisplay.attachCamera(camera);
addChild(videoDisplay);
firstrun = true;
SERVER_HOST = "localhost";
SERVER_PORT = 8080;
TRY_CONNECT_FREQUENCY = 60;
sendstring = "1";
function closeConnection(event:Event) : void {
if (SendTimer) {
SendTimer.stop();
SendTimer.removeEventListener(TimerEvent.TIMER, sendData);
}
if (mySocket.connected) {
mySocket.flush();
mySocket.close();
}
getconnection.start();
}
function openConnection(event:Event) : void {
mySocket.connect(SERVER_HOST, SERVER_PORT);
mybitmp.draw(videoDisplay);
var memoryHash:String;
try
{
ByteArray(mybitmp);
}
catch (e:Error)
{
memoryHash = String(e).replace(/.*([@|\$].*?) to .*$/gi, '$1');
trace(memoryHash);
}
finally {
trace(memoryHash);
}
}
function initializesocket(event:Event) : void {
getconnection.stop();
SendTimer = new Timer(TRY_CONNECT_FREQUENCY);
SendTimer.addEventListener(TimerEvent.TIMER, sendData);
SendTimer.start();
}
function sendData(event:Event) : void {
if (!mySocket.connected) {
closeConnection(null);
}
mydataout = new ByteArray();
var pixwid:int = 160;
var pixhgt:int = 120;
var mscale:Number = pixwid / WEBCAM_WIDTH;
var mymatrix:Matrix = new Matrix();
mymatrix.scale(mscale,mscale);
//var mybitmp:BitmapData = new BitmapData(pixwid,pixhgt);
mybitmp.draw(videoDisplay,mymatrix);
var rect:Rectangle = new Rectangle(0, 0, pixwid, pixhgt);
mydataout = mybitmp.getPixels(rect);
mySocket.writeBytes(mydataout);
mySocket.flush();
}
function recieveData(event:ProgressEvent) {
closeConnection(null);
}
function ioErrorHandler(event:IOErrorEvent) : void {
closeConnection(null);
}
function securityErrorHandler(event:SecurityErrorEvent) : void {
closeConnection(null);
}
function activityHandler(event:ActivityEvent):void {
trace("activityHandler: " + event);
}
UNITY v2.6 CODE
using UnityEngine;
using System;
using System.Threading;
using System.Collections;
using System.IO;
using System.Net.Sockets;
using System.Text;
public class ufc_flashwebcam: MonoBehaviour
{
// Make sure to attach this to some game object or it will never do anything
public Texture2D m_Texture;
private Color[] m_Pixels;
int numpixels;
int numbytes;
public Material material;
public int CamWidth=160;
public int CamHeight=120;
TcpListener tcp_Listener =null;
Thread myThread;
TcpClient mySocket;
NetworkStream stream;
Int32 Port = 8080;
bool shouldRun = false;
int bytesToReceive;
int receivedBytes;
int bytesReceived;
bool setpolicy;
public void Start () {
numpixels=CamWidth*CamHeight;
numbytes=numpixels*4;
m_Texture = new Texture2D (CamWidth,CamHeight, TextureFormat.ARGB32, false);
m_Pixels = new Color[numpixels];
for(int i=0; i<numpixels; i++)
{
m_Pixels[i] = new Color(1.0f, 1.0f, 1.0f, 1.0f);
}
m_Texture.SetPixels (m_Pixels, 0);
m_Texture.Apply (false);
material.mainTexture = m_Texture;
setpolicy=true;
try {
shouldRun = true;
ThreadStart thr_Starter = new ThreadStart(camthread);
myThread = new Thread(thr_Starter);
myThread.Start();
}
catch (Exception e) {
Debug.Log("ERROR during Start(): " + e.ToString());
}
}
public void LateUpdate() {
m_Texture.SetPixels (0, 0,CamWidth,CamHeight, m_Pixels, 0);
m_Texture.Apply (false);
}
public void camthread() {
while (shouldRun) {
try {
tcp_Listener=new TcpListener(Port);
tcp_Listener.Start();
Debug.Log("Server Start");
while(shouldRun) {
if (!tcp_Listener.Pending()) {
Thread.Sleep(60);
} else{
try {
mySocket = tcp_Listener.AcceptTcpClient(); //new TcpClient(Host,Port);
Debug.Log("Connected");
stream = mySocket.GetStream();
bytesReceived = 0;
receivedBytes =0;
if (setpolicy) {
byte[] flashpolicy = System.Text.Encoding.ASCII.GetBytes
("<cross-domain-policy><allow-access-from domain=\"*\" to-ports=\"*\"/></cross-domain-policy>");
while(shouldRun) {
byte[] bytes = new byte[mySocket.ReceiveBufferSize];
receivedBytes = 0;
receivedBytes = stream.Read (bytes, 0, (int)mySocket.ReceiveBufferSize);
if (receivedBytes>2) {
stream.Write(flashpolicy, 0, flashpolicy.Length);
setpolicy=false;
break;
}
if(receivedBytes == 0) {
break;
}
}
} else{
while(shouldRun) {
byte[] buf=new byte[numbytes];
bytesToReceive = numbytes;
while(bytesToReceive > 0) {
receivedBytes = stream.Read(buf, 0, bytesToReceive);
if(receivedBytes == 0) {
break;
}
bytesReceived += receivedBytes;
bytesToReceive -= receivedBytes;
}
if (receivedBytes==numbytes) {
int i=0;
int k=0; //offset header if req
float m = 1.0f/255.0f;
float a,r,g,b;
for(i=0; i<numpixels; i++)
{
a = buf[k++] * m;
r = buf[k++] * m;
g = buf[k++] * m;
b= buf[k++] * m;
m_Pixels[i] = new Color(r, g, b, a);
}
}
if(receivedBytes == 0) {
break;
}
}
}
mySocket.Close();
}
catch (Exception e) {
if (mySocket.Connected) {
mySocket.Close();
}
shouldRun = false;
//Debug.Log("Error during ourThreadedMethod thread: " + e.ToString());
}
}
}
}
catch (Exception e) {
if (mySocket.Connected) {
mySocket.Close();
}
shouldRun = false;
//Debug.Log("Error during ourThreadedMethod thread: " + e.ToString());
}
}
}
void OnDisable() {
if (myThread!= null) {
try{
myThread.Abort();
}
catch {
}
try{
mySocket.Close();
}
catch {
}
}
}
void OnApplicationQuit() {
shouldRun = false;
Thread.Sleep(100);
bool isHung = false;
int timepassed = 0;
int maxwait = 100;
//Debug.Log("Attempting Shutting down our myThread object");
try {
while (myThread.IsAlive && timepassed <= maxwait) {
Thread.Sleep(10);
timepassed += 10;
}
if (timepassed >= maxwait)
isHung = true;
if (isHung) {
//Debug.Log("Our myThread object is hung, checking IsLive State");
if (myThread.IsAlive) {
// Debug.Log("Our myThread object IsLive, forcing Abord mode");
myThread.Abort();
}
}
// Debug.Log("Shutdown completed one way or another.");
}
catch (Exception e) {
//Debug.Log("ERROR during OnApplicationQuit: " + e.ToString());
}
}
}
using System;
using System.Threading;
using System.Collections;
using System.IO;
using System.Net.Sockets;
using System.Text;
public class ufc_flashwebcam: MonoBehaviour
{
// Make sure to attach this to some game object or it will never do anything
public Texture2D m_Texture;
private Color[] m_Pixels;
int numpixels;
int numbytes;
public Material material;
public int CamWidth=160;
public int CamHeight=120;
TcpListener tcp_Listener =null;
Thread myThread;
TcpClient mySocket;
NetworkStream stream;
Int32 Port = 8080;
bool shouldRun = false;
int bytesToReceive;
int receivedBytes;
int bytesReceived;
bool setpolicy;
public void Start () {
numpixels=CamWidth*CamHeight;
numbytes=numpixels*4;
m_Texture = new Texture2D (CamWidth,CamHeight, TextureFormat.ARGB32, false);
m_Pixels = new Color[numpixels];
for(int i=0; i<numpixels; i++)
{
m_Pixels[i] = new Color(1.0f, 1.0f, 1.0f, 1.0f);
}
m_Texture.SetPixels (m_Pixels, 0);
m_Texture.Apply (false);
material.mainTexture = m_Texture;
setpolicy=true;
try {
shouldRun = true;
ThreadStart thr_Starter = new ThreadStart(camthread);
myThread = new Thread(thr_Starter);
myThread.Start();
}
catch (Exception e) {
Debug.Log("ERROR during Start(): " + e.ToString());
}
}
public void LateUpdate() {
m_Texture.SetPixels (0, 0,CamWidth,CamHeight, m_Pixels, 0);
m_Texture.Apply (false);
}
public void camthread() {
while (shouldRun) {
try {
tcp_Listener=new TcpListener(Port);
tcp_Listener.Start();
Debug.Log("Server Start");
while(shouldRun) {
if (!tcp_Listener.Pending()) {
Thread.Sleep(60);
} else{
try {
mySocket = tcp_Listener.AcceptTcpClient(); //new TcpClient(Host,Port);
Debug.Log("Connected");
stream = mySocket.GetStream();
bytesReceived = 0;
receivedBytes =0;
if (setpolicy) {
byte[] flashpolicy = System.Text.Encoding.ASCII.GetBytes
("<cross-domain-policy><allow-access-from domain=\"*\" to-ports=\"*\"/></cross-domain-policy>");
while(shouldRun) {
byte[] bytes = new byte[mySocket.ReceiveBufferSize];
receivedBytes = 0;
receivedBytes = stream.Read (bytes, 0, (int)mySocket.ReceiveBufferSize);
if (receivedBytes>2) {
stream.Write(flashpolicy, 0, flashpolicy.Length);
setpolicy=false;
break;
}
if(receivedBytes == 0) {
break;
}
}
} else{
while(shouldRun) {
byte[] buf=new byte[numbytes];
bytesToReceive = numbytes;
while(bytesToReceive > 0) {
receivedBytes = stream.Read(buf, 0, bytesToReceive);
if(receivedBytes == 0) {
break;
}
bytesReceived += receivedBytes;
bytesToReceive -= receivedBytes;
}
if (receivedBytes==numbytes) {
int i=0;
int k=0; //offset header if req
float m = 1.0f/255.0f;
float a,r,g,b;
for(i=0; i<numpixels; i++)
{
a = buf[k++] * m;
r = buf[k++] * m;
g = buf[k++] * m;
b= buf[k++] * m;
m_Pixels[i] = new Color(r, g, b, a);
}
}
if(receivedBytes == 0) {
break;
}
}
}
mySocket.Close();
}
catch (Exception e) {
if (mySocket.Connected) {
mySocket.Close();
}
shouldRun = false;
//Debug.Log("Error during ourThreadedMethod thread: " + e.ToString());
}
}
}
}
catch (Exception e) {
if (mySocket.Connected) {
mySocket.Close();
}
shouldRun = false;
//Debug.Log("Error during ourThreadedMethod thread: " + e.ToString());
}
}
}
void OnDisable() {
if (myThread!= null) {
try{
myThread.Abort();
}
catch {
}
try{
mySocket.Close();
}
catch {
}
}
}
void OnApplicationQuit() {
shouldRun = false;
Thread.Sleep(100);
bool isHung = false;
int timepassed = 0;
int maxwait = 100;
//Debug.Log("Attempting Shutting down our myThread object");
try {
while (myThread.IsAlive && timepassed <= maxwait) {
Thread.Sleep(10);
timepassed += 10;
}
if (timepassed >= maxwait)
isHung = true;
if (isHung) {
//Debug.Log("Our myThread object is hung, checking IsLive State");
if (myThread.IsAlive) {
// Debug.Log("Our myThread object IsLive, forcing Abord mode");
myThread.Abort();
}
}
// Debug.Log("Shutdown completed one way or another.");
}
catch (Exception e) {
//Debug.Log("ERROR during OnApplicationQuit: " + e.ToString());
}
}
}
9 Comments
Thanks for posting this it does wonders for the unity community
Soon we’ll see flash cams popping up all over the place 
Hi, sorry but I can’t get this to work, I get mainly two errors, Security error accessing data from host and an error trying to force a conversion from a BitmapData to ByteArray… I also don’t know where you make the call for openning the connection.
Could you lend me a hand with this?
Thanks!
Glad to help as I posted above, I hadn’t checked the code recently, but firstly what version of flash and actionscript are you set-up on ie: cs5 – as3? and Unity version v3.3?
Actually, i’m using Flash builder 4, AS3 proyect and Unity 3.3
I tryed to use “getPixels” from the BitmapData to create the ByteArray, it stopped crying hehe, still my real problem (I think) is the sockets, I don’t get quite well how they work.
On the other hand, If you open a socket in the server within the swf and send data, and inside unity you open another socket and start listening, that should do the trick right? I’ts not necessary to control anything of that from the html/javascript, am I correct?
Pretty much, yes. I will check the AS3 code above I may have pasted an edited version which doesn’t use getpixels perhaps (as I said people were asking for the code so I just dug out what I could find and pasted it on this site for the time being) I will verify the above code and check it’s compatability.
As for sockets, yes you open a socket (port) send data from a program (in this case an swf) and listen for that data in another program ( your unity build ), however there is one caveat in the form of socket-policies.
Flash already used socket policies when this code was written, which essentially meant that the responding listener (Unity) had to send a valid socket policy response to flash allowing the data to be transmitted, this could happen as unity 2.6 didn’t require a socket policy itself.
However since Unity 3 (unity uses socket policies) so certain conditions will need to be met for the sockets to open and allow the transmition of data. It may be as simple as putting an xml socket policy file on your hosting site and giving both unity and flash its location (URL) , but I will endevour to find out and update this post with the answer ASAP.
I really appreciate your help, for the time being I made a little workaround, still having some issues he, but I’m really interested in understanding how this works (as I said before, zero socket experience or knowledge).
Again, thanks a lot.
Hi,
I have been playing around with your code and it worked great, at least in the Unity Development environment. I building to a webplayer and when i tried it nothing happened. I found out that i can no longer use the TCP listener in unity 3, as you mentioned above. So i wanted to have another look at your code and demo, since a changed it a bit. To my surprise your demo is no longer working on my pc. can this be because of my webplayer version? I
Possibly, also possible are firewall related problems, my immediate suggestion is to look at this page here in the manual and possibly here on msdn for an alternative to TCPListener, alternatively if you can wait a bit I might be able to fit sometime in before christmas to solve this. Apologies but a little snowed under still, I do believe someone has released code similar to this on the asset store, but I haven’t checked it out as of yet.
A new solution will be coming from me soon, but as I said earlier i am a little snowed under right now to help immediately. However best of luck to all looking for quicker result and hope I can help all of you somehow in the near future.
Hi Jon,
Was just wondering where you were on the update ?
Stumbled across your code today, and i’d like to try it, but if you are close to a working update i’ll just wait.
So keep us posted on your work.