BIM, 3D Photospheres and AR R&D Project Update (2)

3D Photospheres
Altuity and the University of Bristol Democratizing BIM Project

In our previous post we described progress at the 3 month stage of our R&D project with the University of Bristol. This research project, which is co-funded by the UK Government Innovation Agency (Innovate UK), aims to democratise access to BIM within operations and maintenance by using 3D photorealistic environments and augmented reality.

Achieving this will provide the education sector (schools, MATs etc.) and the commercial sector with access to pragmatic, affordable and leading edge 3D technology to help maintain buildings, estates and assets which up until now has been unaffordable and/or required specialist skills.

Watch this short video (just over a minute long) to see the concepts explained below in action. It shows how you can:

  • capture 360º photospheres
  • calibrate the photospheres to floor plans within a browser (no CAD required)
  • mark up features of interest in the photosphere e.g. record a defect
  • access information in the office or on-site via augmented reality (AR)

After 6 months work we’re approaching the point where we’ll be making the software available to a limited number of early adopters for evaluation and feedback.

3D Photosphere project update
The R&D team reviews progress in Nov 2017

Capturing 3D Photospheres

Capturing a 3D photosphere is simple and cost effective which is why we’ve chosen to use this technology. However given that they’ll be used for asset and maintenance management purposes there’s a need for a best practice guide which we’ve now written. For example, there are two key areas to consider to get the most from the photospheres:

1. Distance and angle of the camera to the assets
Distance and camera angle are important considerations given the different sizes of assets that might need to be marked up in a photosphere. For example, there’s an optimum distance to visually recognise a fire alarm compared to a larger asset or feature such as a door or window.

Our testing showed that the optimal distance from asset to camera should be approximately 3m with an angle of 40º.

2. Room size and shape
The room size and shape determine how many photospheres are needed to get the required visual coverage of the space. A small rectangular or square shaped room will require fewer photospheres than a larger space such as a hall, gym or ‘L’ shaped space. With multiple photospheres in use they need to be linked together to facilitate virtual room walking as well as being able to virtually walk an entire building.

Calibrating Photospheres to 2D Floor Plans

While viewing and managing data related to assets or facilities requests in the photosphere is very beneficial it does not entirely replace the need for a 2D view in order to see it on, for example, a floor plan. Calibration is therefore required to integrate 3D photospheres to 2D floor plans.

The calibration process involves identifying the camera position on the 2D plan and then matching room features with the plan such as the room corners. Once the calibration has been completed markers can be positioned on the 2D plan and they appear in the right location in the 3D photosphere or alternatively added to the photosphere and then they’ll appear on the 2D plan.

Calibrating 3D photospheres and 2D floor plans
Calibrating 3D photospheres and 2D floor plans to provide a 3D and 2D immersive experience
Visual data management using photospheres
Visual data management using photospheres

The calibration process has been built into the software to make it quick and easy so that users can undertake this process themselves.

 

Photospheres and floor plans
Photospheres located on floor plan

 

Once calibrated data can be managed from both the floor plan and photosphere environments.

 

Delivering a Reliable Augmented Reality Experience

A reliable AR experience (AR) is essential in an operational environment. Although AR technology is advancing at a rapid pace we looked at what was required from the user perspective rather than purely from a technological one

This led us towards adopting a meta-AR approach. This involves using the photospheres on a mobile device with the photospheres automatically aligning themselves. We are then able to reliably overlay data onto the photosphere on-site rather than augmenting it via the camera view. The synchronisation of the photosphere and device is so good that it is often indistinguishable from the camera view as illustrated below.

Using meta-AR provides a better and more reliable user experience
Using meta-AR provides a better and more reliable user experience

Photospheres, Stakeholders and Income Generation

During our discussions with business managers and facilities teams they’ve identified that as well as using the photospheres for operational asset and maintenance purposes they’re much better than photographs in providing stakeholders with an interactive visual perspective of a room or space. Typical uses would be:

 At school governor meetings to better visualise and provide a virtual experience of the condition of school buildings needing refurbishment or replacing;
 To show new or refurbished facilities to help promote a school to prospective parents;
 To promote a school’s facilities for use by external organisations to help generate income.

They’ll be another final(!) update on the completion of the project in a couple of months’ time. At this point this functionality will be available in AltoSites.

Please check back regularly; complete the sign up form or use the contact us form to receive notifications of updates or to express interest in participating in our early adopter feedback programme. This gives you an opportunity to test and evaluate the technology for free in exchange for providing feedback and suggestions.

University R&D Collaboration

We’re an SME with ambitious plans supported by on-going R&D in our software which helps managers responsible for maintaining built environment assets. We wanted to extend our R&D capability by collaborating with a university and maybe you’re an SME wondering if this might help your company – well here’s our experience so far.

Key Requirements

  • Undertake R&D while not compromising existing software development.
  • Connect to academics to take advantage of the expertise, experience and different perspectives they offer.
  • Provide an opportunity for a student to gain some useful work experience.
  • Deliver meaningful results to our business.

We initially considered Innovate UK’s Knowledge Transfer Partnership programme. However, this is suited to projects of 12 months or more and for our first foray into working with a university a shorter timescale was more appropriate. In addition this R&D project only required a period of about 3 months although it might then be extended.

The Work

We approached the Faculty of Engineering at the University of Bristol who explained the potential to work with a paid intern; supervised by the faculty, over a 3 month period. This was ideal from our perspective.

We jointly scoped an R&D project to investigate mobile localisation and visualisation techniques that could help, for example, facilities and school business managers, bursars and site teams responsible for looking after assets and buildings.

Harry Whiskard, an engineering student, expressed interest in the work and joined the project. Harry, supported by the Faculty, undertook proof-of-concept R&D into the use of:

  • Tango™ from Google
  • Photospheres
  • Image recognition.
Google Tango
Tango device

He then tested the outcomes at a school thanks to the support of Alan Neale, School Business Manager, at Churchill Academy and Sixth Form in North Somerset.

Photosphere at Churchill Academy
Photosphere of Design&Technology room at Churchill Academy and Sixth Form

The Result

Harry’s work was extremely valuable in helping to scope what was possible and in identifying the practical limitations of some of the technology.

Floor plan, photosphere augmented marker integration.
Prototype showing a floor plan linked to a photosphere of the gym at Churchill Academy accessible through a smartphone. Two simple markers have been augmented onto the real world view – note the use of perspective to indicate where the markers are in relation to the viewer.
Automatic visual recognition
Trialling automatic visual recognition between physical features and a visual image.

These outcomes have influenced the scope of a new R&D project which is due to start soon and which will see us working with the University of Bristol on a longer project.

As well as the direct outcomes from the R&D project there were other indirect benefits. Working with a university provides an opportunity to step outside the purely commercial world for a while and having open conversations at the university keeps us fresh and receptive to new ideas.

It was also good to offer a student the opportunity to develop their skills and to gain some practical experience which will hopefully help in the future. In Harry’s own words:

“I thoroughly enjoyed my time working with Altuity, it was a very rewarding experience. This project provided me with the opportunity to learn new skills such as software development which otherwise I would not have been able to experience in my degree. It also provided me with insight into the importance of research and how ideas can be developed in the commercial world.”

No matter what the size of your company if you need some assistance and access to wider knowledge then why not contact a university?

Explore what might be possible in terms of working together. There’s plenty of enthusiasm within universities to do this and properly managed it’ll be the classic ‘win-win’.

One final thought – if you do employ an intern do the right thing and pay them!

We’ll be posting an update shortly on the follow up R&D project Harry’s work helped set the scene for…

(Project Tango is a registered trademark of Google Inc.)