The networking infrastructure is formed through the interoperability of Data Sources, Listeners, and a Central Server. Reliable communication between these units is required to ensure the proper data generation and representation. Through library the data is able to be sent and received from a vast variety of clients without any changes make to the server. This allows flexibility in the uses of this infrastructure for future research projects as any data to the server to be received by all clients to facilitate interaction. Any client can register with the server to receive the blob stream to do any number of representations or gather statistics on the data. The server used to run all of the data through is a nodejs server that communicates through and TCP. The job of the server is to transform all of the data received into useful blob information and broadcast it to all interested clients. The server is designed to be lightweight and modular so that the data can be received by the clients as fast as possible. Besides delivering the data received to every interested client the server also runs a Blob Manager that allows modification to the sent blobs.

Details: • id: The id of this blob • cameraID the specific camera this Id came from (together with id these uniquely identify a blob) • origin: The center of the blob, specified in (x, y, z) coordinates. Coordinates of blobs in the building should consider these coordinates relative to the learning lab observation suite pillar next to the door. • orientation: The orientation of the blob, specified as an (x, y, z) vector and a rotation around that vector in radians. Blobs data coming from the cameras in the building currently have no way of determining orientation, and will always have an orientation of (0.0, 0.0, 0.0, 0.0). • source: The source of this blob's data. For example a Matlab/OpenCV or a specific virtual client. • updatedTime: The last time this blob's data was updated. currently set to always 0 in Matlab • creationTime: The time when this blob first entered the scene and was created currently always set to 0 in matlab. • boundingBox: The bounding box of blobs from the camera tracking system, specified as the (x, y) coordinate of the top left corner, as well as the width and height of the rectangle. The bounding box also includes the image height and image width of the actual image used to create the blobs. These are used by the blob manager to help determine global coordinates.

The blob manager is a sub-module within the server. Its constructor requires a callback that will be used whenever the blob manager deems a specific blob needed to be sent (function BlobManager(sendBlobCallback)). This callback will have a single parameter that is the blob to send to interested clients. The purpose of the blob manager is to preform CPU intensive tasks, this means in the future if the tasks become intensive enough the blob manager should be run in a separate thread or process from the server's front end to allow the nodejs event loop to run smoothly, permitting the continuous sending and receiving of data. The interface to hand off a blob to the blob manger for processing is the process blob method(method.processBlob = function (blob)). Once this blob has been sent to the manager through this function call it is no longer the responsibility of the calling class. The blob manager will then run any computations it needs to on the blob and if it is determined it needs to be sent, the blob will be returned to the calling class through the function callback given in the constructor with the intention of this blob being sent to the clients.

The blob manager currently serves the function of transforming each blobs coordinates from the perspective of each individual camera in the building to a global coordinate system that does not require each client to be knowledgeable of the specific locations and orientations of each camera, and also makes the computation only have to happen once instead of on every individual client.Each camera has an x, y, z, width, height, and theta associated with it. These are all determined by physically measuring the respective qualities from a predetermined origin. To aid in the addition of new cameras and more easily keep the server able to support all cameras, the data is read in from a CSV file containing all of the relevant information. The specification for the CSV file can be found here: (Eric Williamson insert link to CSV file spec once it is finalized). The use of these global coordinates allows the easier mapping of movements of blobs without having to worry about the properties of each individual camera. For some specialized clients the specific cameraID will remain on the blobs so they can ignore blobs that have originated from other cameras than the ones the are concerned about.

Additional features can be added to the blob manger as needed without requiring changes to the part of the server sending and receiving the data.

Computer Vision



Research Infrastructure