BIM, 3D Photospheres and AR R&D Project Update (2)

3D Photospheres
Altuity and the University of Bristol Democratizing BIM Project

In our previous post we described progress at the 3 month stage of our R&D project with the University of Bristol. This research project, which is co-funded by the UK Government Innovation Agency (Innovate UK), aims to democratise access to BIM within operations and maintenance by using 3D photorealistic environments and augmented reality.

Achieving this will provide the education sector (schools, MATs etc.) and the commercial sector with access to pragmatic, affordable and leading edge 3D technology to help maintain buildings, estates and assets which up until now has been unaffordable and/or required specialist skills.

Watch this short video (just over a minute long) to see the concepts explained below in action. It shows how you can:

  • capture 360º photospheres
  • calibrate the photospheres to floor plans within a browser (no CAD required)
  • mark up features of interest in the photosphere e.g. record a defect
  • access information in the office or on-site via augmented reality (AR)

After 6 months work we’re approaching the point where we’ll be making the software available to a limited number of early adopters for evaluation and feedback.

3D Photosphere project update
The R&D team reviews progress in Nov 2017

Capturing 3D Photospheres

Capturing a 3D photosphere is simple and cost effective which is why we’ve chosen to use this technology. However given that they’ll be used for asset and maintenance management purposes there’s a need for a best practice guide which we’ve now written. For example, there are two key areas to consider to get the most from the photospheres:

1. Distance and angle of the camera to the assets
Distance and camera angle are important considerations given the different sizes of assets that might need to be marked up in a photosphere. For example, there’s an optimum distance to visually recognise a fire alarm compared to a larger asset or feature such as a door or window.

Our testing showed that the optimal distance from asset to camera should be approximately 3m with an angle of 40º.

2. Room size and shape
The room size and shape determine how many photospheres are needed to get the required visual coverage of the space. A small rectangular or square shaped room will require fewer photospheres than a larger space such as a hall, gym or ‘L’ shaped space. With multiple photospheres in use they need to be linked together to facilitate virtual room walking as well as being able to virtually walk an entire building.

Calibrating Photospheres to 2D Floor Plans

While viewing and managing data related to assets or facilities requests in the photosphere is very beneficial it does not entirely replace the need for a 2D view in order to see it on, for example, a floor plan. Calibration is therefore required to integrate 3D photospheres to 2D floor plans.

The calibration process involves identifying the camera position on the 2D plan and then matching room features with the plan such as the room corners. Once the calibration has been completed markers can be positioned on the 2D plan and they appear in the right location in the 3D photosphere or alternatively added to the photosphere and then they’ll appear on the 2D plan.

Calibrating 3D photospheres and 2D floor plans
Calibrating 3D photospheres and 2D floor plans to provide a 3D and 2D immersive experience
Visual data management using photospheres
Visual data management using photospheres

The calibration process has been built into the software to make it quick and easy so that users can undertake this process themselves.

 

Photospheres and floor plans
Photospheres located on floor plan

 

Once calibrated data can be managed from both the floor plan and photosphere environments.

 

Delivering a Reliable Augmented Reality Experience

A reliable AR experience (AR) is essential in an operational environment. Although AR technology is advancing at a rapid pace we looked at what was required from the user perspective rather than purely from a technological one

This led us towards adopting a meta-AR approach. This involves using the photospheres on a mobile device with the photospheres automatically aligning themselves. We are then able to reliably overlay data onto the photosphere on-site rather than augmenting it via the camera view. The synchronisation of the photosphere and device is so good that it is often indistinguishable from the camera view as illustrated below.

Using meta-AR provides a better and more reliable user experience
Using meta-AR provides a better and more reliable user experience

Photospheres, Stakeholders and Income Generation

During our discussions with business managers and facilities teams they’ve identified that as well as using the photospheres for operational asset and maintenance purposes they’re much better than photographs in providing stakeholders with an interactive visual perspective of a room or space. Typical uses would be:

 At school governor meetings to better visualise and provide a virtual experience of the condition of school buildings needing refurbishment or replacing;
 To show new or refurbished facilities to help promote a school to prospective parents;
 To promote a school’s facilities for use by external organisations to help generate income.

They’ll be another final(!) update on the completion of the project in a couple of months’ time. At this point this functionality will be available in AltoSites.

Please check back regularly; complete the sign up form or use the contact us form to receive notifications of updates or to express interest in participating in our early adopter feedback programme. This gives you an opportunity to test and evaluate the technology for free in exchange for providing feedback and suggestions.

BIM, 3D Photospheres and AR R&D Project Update

In our previous post we announced the kick off of our R&D project with the University of Bristol. This research project, which is co-funded by the UK Government Innovation Agency (InnovateUK), aims to democratise access to BIM within operations and maintenance by using 3D photorealistic environments and augmented reality.

Achieving this will provide the education sector (schools and MATs etc.) and the commercial sector with access to pragmatic, affordable and leading edge technology to help maintain buildings and assets.

This post describes our progress so far with 5 months still to run.

Capturing 3D Photospheres

Our first activity was to visit several schools to collect a range of photospheres from rooms and spaces of differing sizes, shapes and lighting conditions.  We also gathered excellent ideas and feedback from school business managers on how the project could be used to assist with managing and maintaining school buildings and assets.

Collecting Photospheres
Fig 1 Collecting photospheres at The King’s School

From this work we’ve been able to assess the optimum number of photospheres needed in order to provide proper fields of view for an entire room which might not be a simple square or rectangular shape and also in many instances will have furniture or equipment obscuring the field of vision.

Importantly this process has also shown how quick, easy and cost-effective it is to collect the photospheres using a 3D camera with a connected smartphone. This is essential as we want our users to have the capability to do this work themselves without having to use external resources although this is a future service we’ll be providing for those that want it.

We’ve also shown that multiple, inter-connected photospheres will be required so that a user can fully explore a class room or office by virtually walking through it whether that’s a relatively small room or a larger space such as the assembly hall in figure 2.

Photosphere
Fig 2 Churchill Academy Assembly Hall photosphere

This is particularly important in order to provide the right level of coverage to ensure it’s possible to zoom into a feature or asset anywhere in the room e.g. a fire alarm or lighting switch.

This work also showed the need to assess the ability to read augmented reality markers in a room based on size, angle and distance from the smartphone’s camera. The results of these tests are currently being evaluated.

Integrating Photospheres and 2D Floor and Site Plans

Photospheres have been integrated into 2D plans within AltoSites. This means users can collect the photospheres and position them on a floor or site plan.

 

Photosphere marker integrated into a 2D floor plan
Fig 3 Photosphere marker integrated into a 2D floor plan

The extract below shows the ease of displaying a photosphere linked to a 2D plan.

With this integration in place we are now assessing how assets and maintenance/service request details located on the 2D plan can be augmented onto the photosphere for that room or space.

Stakeholder Engagement – Early project win!

During our discussions with business managers they’ve identified that as well as using the photospheres for operational asset and maintenance purposes they’re much better than photographs to provide stakeholders with an interactive visual perspective of a room or space.  Typical uses would be:

  • At school governor meetings to better visualise and provide a virtual experience of the condition of school buildings needing refurbishment or replacing.
  • To show new or refurbished facilities to help promote a school to prospective parents;
  • To promote the school’s facilities for use by external organisations to help generate income.

Given the interest in this we’re launching a service to provide:

  1. the capability for a school or organisation to take photospheres themselves (or we can do this as a service) and
  2. software which you can integrate into your website to display the photospheres

For more information on this new service please contact us.

If you’d like to receive notification of further project updates please register via the ‘Subscribe for email updates’ window, or provide your contact  details or simply check back in a couple of months’ time!

Democratising BIM through light weight on-site imaging

BIM, photospheres and augmented reality
Current 2D plan and map functionality extended to BIM, AR and photo-realism

In a previous post I described how we undertook a joint research project with the University of Bristol (UoB) to assess how various technologies, including Google Tango™, can help the education sector manage and maintain assets, buildings and estates. This has now led to a nine month R&D project with the UoB, co-funded by the UK Government Innovation Agency (InnovateUK).

This post is the first in a series of bi-monthly updates on our progress.

Why are we doing this R&D?

Currently the vast majority of education and smaller organisations managing buildings and assets do not use Building Information Modelling (BIM) in their day-to-day operations.  There are various reasons for this such as:

  • The need for specialised and expensive hardware/software, skills, expertise and software.
  • BIM data simply isn’t available because of the age of the buildings being maintained.
  • The complexity involved is off-putting to many.
  • Current BIM processes tend to be focused on design whereas our users’ interests are in the maintenance of assets and buildings.

This is why our strategy of ‘Bringing Simplicity from Complexity’ and applying this to asset and maintenance management is key to wider adoption. We achieve this simplicity by using a highly visual approach. For example, AltoSites our asset and maintenance software, uses maps and floor and site plans.

A class room showing augmented maintenance data
Prototype illustrating how augmented data can be overlaid onto a real world class room.

This project extends our existing mobile capability by offering users a more immersive experience – one that allows them to easily see and record data as illustrated to the left.

 

The Technology (Mobile, 3D Environments and Augmented Reality)

Using 2D plans, photospheres and augmented markers.
Using 2D plans, photospheres and augmented markers.

This R&D will bring a disruptive step change to mobile data collection and will also encourage wider adoption of BIM. Harnessing various technologies, combined with new software tools which are easy to use and affordable, will encourage widespread use within an organisation – the democratisation effect referred to in our project’s title. For example, this will help a building’s site maintenance staff and other stakeholders such as teaching staff in a school or university record and access information.

Our ambition is to leap frog existing state-of-the art mobile apps by providing suitably accurate onsite data capture and information access through the creation of interactive user environments that combine visually-realistic onsite imaging and BIM, asset and maintenance information. Innovations in augmented reality and mobile device innovations, such as the capture and display of 3D environments, allow us to explore how users can easily access and interact with combined digital and physical information.

What will it mean to you?

If you’re inspecting assets and buildings and need to record data and access information across the estate it’ll make your work easier. Augmented reality provides enhanced visualisation combined with locational context meaning you get to focus on the data that’s relevant to your position and you can also access other information e.g. from BIM, plans or internal reports.

This new software solution will lead to a step change in productivity. There will be improvements in operational efficiency such as staff utilization and productivity. No manual data collection nor re-keying of data will be required reducing inaccuracies, data redundancy and saving time. Better maintenance planning and works will lead to more productive and safer, fit-for-purpose workplace.

This R&D will also ensure that the solution is uniquely capable of being adopted by non-specialists. This means building users as well as site maintenance staff can use it. For example, school administration and teaching staff could report maintenance issues or make service requests. Putting this capability in the hands of front line staff improves overall quality and responsiveness and is consistent with the “uberisation” trend seen in many industries.

Want to participate or find out more?

We’re very interested in hearing from schools, colleges, universities and private sector organisations who’d like to participate in a short beta programme in October/November, 2017.  No technical experience necessary – simply an open mind and a willingness to provide feedback. To find out more contact us.

If you’d like to receive notification of further project updates please register via the ‘Subscribe for email updates’ window, or provide your contact  details or simply check back in a couple of months’ time!

University R&D Collaboration

We’re an SME with ambitious plans supported by on-going R&D in our software which helps managers responsible for maintaining built environment assets. We wanted to extend our R&D capability by collaborating with a university and maybe you’re an SME wondering if this might help your company – well here’s our experience so far.

Key Requirements

  • Undertake R&D while not compromising existing software development.
  • Connect to academics to take advantage of the expertise, experience and different perspectives they offer.
  • Provide an opportunity for a student to gain some useful work experience.
  • Deliver meaningful results to our business.

We initially considered Innovate UK’s Knowledge Transfer Partnership programme. However, this is suited to projects of 12 months or more and for our first foray into working with a university a shorter timescale was more appropriate. In addition this R&D project only required a period of about 3 months although it might then be extended.

The Work

We approached the Faculty of Engineering at the University of Bristol who explained the potential to work with a paid intern; supervised by the faculty, over a 3 month period. This was ideal from our perspective.

We jointly scoped an R&D project to investigate mobile localisation and visualisation techniques that could help, for example, facilities and school business managers, bursars and site teams responsible for looking after assets and buildings.

Harry Whiskard, an engineering student, expressed interest in the work and joined the project. Harry, supported by the Faculty, undertook proof-of-concept R&D into the use of:

  • Tango™ from Google
  • Photospheres
  • Image recognition.
Google Tango
Tango device

He then tested the outcomes at a school thanks to the support of Alan Neale, School Business Manager, at Churchill Academy and Sixth Form in North Somerset.

Photosphere at Churchill Academy
Photosphere of Design&Technology room at Churchill Academy and Sixth Form

The Result

Harry’s work was extremely valuable in helping to scope what was possible and in identifying the practical limitations of some of the technology.

Floor plan, photosphere augmented marker integration.
Prototype showing a floor plan linked to a photosphere of the gym at Churchill Academy accessible through a smartphone. Two simple markers have been augmented onto the real world view – note the use of perspective to indicate where the markers are in relation to the viewer.
Automatic visual recognition
Trialling automatic visual recognition between physical features and a visual image.

These outcomes have influenced the scope of a new R&D project which is due to start soon and which will see us working with the University of Bristol on a longer project.

As well as the direct outcomes from the R&D project there were other indirect benefits. Working with a university provides an opportunity to step outside the purely commercial world for a while and having open conversations at the university keeps us fresh and receptive to new ideas.

It was also good to offer a student the opportunity to develop their skills and to gain some practical experience which will hopefully help in the future. In Harry’s own words:

“I thoroughly enjoyed my time working with Altuity, it was a very rewarding experience. This project provided me with the opportunity to learn new skills such as software development which otherwise I would not have been able to experience in my degree. It also provided me with insight into the importance of research and how ideas can be developed in the commercial world.”

No matter what the size of your company if you need some assistance and access to wider knowledge then why not contact a university?

Explore what might be possible in terms of working together. There’s plenty of enthusiasm within universities to do this and properly managed it’ll be the classic ‘win-win’.

One final thought – if you do employ an intern do the right thing and pay them!

We’ll be posting an update shortly on the follow up R&D project Harry’s work helped set the scene for…

(Project Tango is a registered trademark of Google Inc.)