Posted on

System updates – June 2022

Clutter not LiDAR

We’ve been busy as always improving our service. Here’s a visual roundup of updates we’ve pushed recently for API version 2.9 and UI version 2.8, current as of the 13th June 2022.

For the latest updates click the versions in the corner of the interface.

Software change log shortcuts

New features

Variable building heights

Our global building data has had a lot of attention recently from 5G operators and (counter) drone companies.

Previously you could use a mixture of sources to define a building from manipulating your clutter profile values to adding your own as GeoJSON or KML. Our third party buildings which we offer as an enrichment option in the Landcover menu were lacking height data so we used a user defined value from the clutter profiles. For many users on default settings, this resulted in buildings at 6m height, adequate for a suburb with similar architecture but no use for a vertical city like Chicago.

Our building data now has unique heights and we’ve modified our clutter system to match it. Our SLEIPNIR engine could already handle user defined height data due to previous modifications around custom clutter.

In the CloudRF UI, you can define the 9 system codes and only 9 custom codes.

Customers with private servers can define up to 255 clutter codes in clutter files on their system.

Variable building heights

Diffraction for 3D clutter

Previously we modelled single knife edge diffraction only for the surface model, and considered through building attenuation for the clutter but now we can do it for clutter.

In the screenshot below, the COST model is used with 3D clutter for a suburb with a treeline along a train track. Both the buildings and trees are displaying diffraction, recognised by a shadow beyond an obstacle which gradually comes back into coverage as the obstacle angle decreases.

Clutter not LiDAR
This image does not use LiDAR. It is 10m landcover with 2m buildings on a 30m DTM.

DTM option instead of LiDAR

We’ve worked with LiDAR in modelling for years and its great but has its limitations, especially with non line of sight communications where LiDAR will show a strong signal for the roof of a building and produce a very conservative “shadow” immediately behind an obstacle until knife edge diffraction kicks in.

Being conservative in RF planning isn’t a bad thing but with proper building height data we can enable more accurate through-building attenuation in cities, worldwide.

Nothing beats LiDAR for LOS analysis so we’ll still consider requests for uploading public LiDAR but for most of the world where there isn’t LiDAR, we now have an improved solution.

For rooftop planning, we recommend LiDAR but for through building/tree modelling at ground level use calibrated clutter instead. You can enable this option in the Clutter menu.

You can access this mode in the interface “Clutter” menu or in the JSON API with {clm: 2} in the environment object.

Clutter on DTM
LiDAR DSM

Delete-all custom clutter

API users can now delete all their custom clutter by requesting to delete id=0. This will soon be added to the UI with a button.

Added my-metrics endpoint for API usage

We’re now logging metrics for all the analytics APIs which count against your API use. We have an API to generate charts and will soon add client side charts so you can see how your API use breaks down by tool and time.

Automated testing

Testing is a critical part of maintaining our quality of service which is becoming increasingly complex. We’ve added automated workflows to our development environment to help catch bugs earlier and complement the manual interface testing.

This is implemented for the API and UI repositories at the code function level as well as our regular regression testing at the API level and now third party automated exception handling.

Improvements

Conditional terrain smoothing

We’re smoothing terrain for those super sampled locations. For example if you went to Africa which is mostly 30m DTM, and requested a 2m plot, we would be super-sampling the DTM by a factor of 15 which used to result in ugly artefacts on hillsides. We’ve fixed that and are able to have smooth hills and precision 2m clutter in remote areas 🙂

Rwanda with super sampled 30m DTM and 2m buildings with diffraction

Up to 255 clutter codes

In the CloudRF UI, you can define the 9 system codes and only 9 custom codes but the backend system supports up to 255 unique classes of clutter. You set the height and density for each so this could be skyscrapers for a city or crops.

Customers with private servers can define up to 255 clutter codes in clutter files on their system.

Diffraction loss adjusted down by 3dB

Following feedback from Mountain rescue teams using our service who were surprised to find coverage in modelled dead spots, we investigated our diffraction model and found it was too conservative by at least 5dB. We’ve adjusted the loss it applies down by 3dB so it’s now more optimistic, but still cautiously conservative.

Extended southern limit to -89N

You can model on Antarctica now. We don’t have DTM there so it’s flat as far as our service is concerned but if you are using the Satellite tool, this now works on the continent for testing for the horizon on a route etc.

Fixes

  • Replaced source for 3D buildings from a commercial supplier to Openstreetmap.org
  • Points requests was failing to handle some responses from the engine
  • Returning correct HTTP status codes for errors now
  • Template list is returned as JSON in the UI
  • Template authentication has been upgraded to the header “key”
  • Credits balance was reported incorrectly for some API calls
  • Remote tile fetching was corrupting some local tiles
  • Sanity check unworkable paths before passing into engine (eg. 1m or 1000km)
  • Some JSON responses malformed
  • Splicing of points near the Rx in a points request works better for a figure of 8 route.
  • Preferences was breaking if no lat or lon set
  • Performance improvements to “sea tiles” used for offshore planning.
  • Changed noise floor validation to -130 to -50dBm
  • Fixed issue with some interference API calls missing id values

Posted on

Enhancing accuracy with environment profiles

Clutter profile manager

In radio planning, accurate terrain data is only half the story.

The other data you need, if you want accurate results, is everything above the surface such as buildings and trees.

This is known as land cover or in radio engineering; clutter data.

Clutter data

Clutter manager with 18 bands

In October 2021, the European Space Agency released a global 10m land cover data set called WorldCover with 9 clutter bands.

In our opinion, the ESA data is a sharp improvement on a similar ESRI/Microsoft 10m land cover data set also published this year.

The land cover can be used to enhance coarse 30m data sets to distinguish between homes and gardens, or crops and rivers. It’s space mapped so has every recent substantial building unlike community building datasets which can be patchy outside of Europe.

This data was previously very expensive. A price reflected in the eye watering pricing of legacy WindowsTM planning tools.

The data has 9 bands which have been mapped to 9 land cover codes in Cloud-RFTM. Combined with our recent 9 custom clutter bands, we have 18 unique bands of clutter which you can use simultaneously.

Read more about the codes in the documentation here.

Explore the data we have on the ESA WorldCover viewer application here:

https://viewer.esa-worldcover.org/worldcover/

Custom clutter enrichment

We have integrated the 10m data into our SLEIPNIRTM propagation engine which as of version 1.5, can work with third party and custom clutter tiles simultaneously, in different resolutions.

This is significant as it means you can have a 30m DSM base layer, enhanced with a 10m land cover layer, enriched further with a 2m building which you created yourself. Effectively this gives you a 10m global base accuracy with potential for 2m accuracy if you add custom obstacles. The interface will let you upload multiple items as a GeoJSON or KML file.

Demo 1 – The Jungle

Always a tricky environment to communicate in, and model accurately due to dense tree canopies. In this demo, a remote region of the Congo has been selected at random for a portable VHF radio on 75MHz with a 3km planning radius.

This area has 30m DSM which out of the box produces an unrealistic plot resembling undulating flat terrain. This is because the thick tree canopy is represented as hard ground and the signal is diffracting along as if it were bare earth. The result therefore is that 3km is possible in all directions.

By adding our “Tropical.clt” clutter profile, calibrated for medium height, dense trees, we get a very different view which shows the effective range through the trees to be closer to 1km, or less, with much better coverage down the river basin, due to the lack of obstructions.

Demo 2 – A region without LiDAR

Scotland has very poor public LiDAR compared with England which has good coverage at 1m and 2m.

For this demo, Stirling was chosen which has 30m DSM only. A cell tower on a hill serving the town produces an optimistic view of coverage by default but when enhanced with a “Temperate.clt” clutter profile, calibrated for solid and tall town houses and pine forests (eg. > 50N, Northern Europe, Northern USA) we get a much more conservative prediction. As a bonus, the base resolution has improved three fold to 10m.

Demo 3 – A region with 2m LiDAR

You might think if you’ve got high resolution LiDAR data that’s enough. Wrong. Soft obstacles like trees especially will produce excessive diffraction as if they were spiky terrain. This manifests itself as optimistic ‘great’ coverage due to the diffraction coverage. By adding our “Temperate.clt” profile again we make trees absorb power and see where there are nulls in our coverage – beyond the houses and woods.

Despite our land cover being only 10m resolution, we are able to benefit from the full LiDAR resolution with 2m accuracy.

Inspecting a profile

The path profile tool will now show you colour coded land cover as well as custom clutter and 3D buildings. Crops are yellow, grass is green(!), Trees are dark green, built-up areas are red, 3D buildings are grey, water is blue…

The most significant feature in this image isn’t the coloured land cover, or the custom building (as both are features we’ve done before), or the fact we know the tidal river Severn sits lower than the man-made Canal beside it, It’s the fact that both are being used in the same model at the same time. They are different sources, different resolutions, different densities…

Path profile for 860m link showing 2m LiDAR, 10m Land cover and 2m custom building

Using and editing a profile

Clutter menu with 3D buildings enabled

Once you’ve got the hang of switching profiles you may find it needs optimising for your region. With the clutter manager in the web interface, premium customers can create their own profile based on field measurements for highly accurate predictions. After all no two forests or neighbourhoods are the same density.

Create your perfect profile and save it to your account. The system has 5 regional profiles ready for all users and you can add your own.

To use them, pick from the Clutter > Profile menu and ensure “Landcover” is set to “Enabled”.

If you have created custom clutter and want to use that, set Custom clutter to “Enabled” to blend it in.

For more see the web interface clutter section in the documentation.

Using clutter from the API

We played with a few designs before settling on this very simple template method where you set a profile within the environment menu as follows. This is a new value “clt” and you can still use the existing “cll” and “clm” values to manage the system clutter and custom clutter layers.

JSON request excerpt for a temperate “European” profile, with custom clutter, with 3D buildings and a 3D building density of 0.25dB/m

  "environment": {
        "clt": "Temperate.clt",
        "clm": 1,
        "cll": 2,
        "mat": 0.25
    },

Example for Jungle profile, without custom clutter, without 3D buildings.

  "environment": {
        "clt": "Jungle.clt",
        "clm": 0,
        "cll": 1,
        "mat": 0
    },

Further reading:

CloudRF API on Postman: https://docs.cloudrf.com/

OpenAPI reference: https://cloudrf.com/documentation/developer/swagger-ui/

What’s next?

Now that we have highly configurable environment profiles. it’s time to tune them with field testing. We’ve bought a heap of comms equipment and will be using it to optimise these profiles with real world measurements.

Posted on

RF penetration demonstration

During infantry training, soldiers are shown firsthand the impact of different weapons upon different materials to help them make better decisions about good cover versus bad cover. Spoiler: The railway sleeper doesn’t make it 🙁

As tactical radios have moved several hundred megahertz up the spectrum from their cold-war VHF roots, material attenuation is a serious issue which needs demonstrating to enable better route selection and siting. Unlike shooting at building materials it’s hard to visualise invisible radio signals, and therefore teach good siting, but equally important as ground based above-VHF signals are easily blocked in urban environments.

This blog provides a visual demonstration of the physical relationship between different wavelengths and attenuating obstacles only. It does not compare modulation schemas, multi-path, radios or technologies.

Bricks and wavelengths

Clutter data refers to obstacles above the ground such as trees and buildings. Cloud-RF has 9 classes of clutter data within the service which you can use and build with. Each class (Bricks +) has a different attenuation rate measured in decibels per metre (dB/m). This rate is a nominal value based upon the material density and derived from the ITU-R P.833-7 standard and empirical testing with broadcast signals in European homes.

A signal can only endure a limited amount of attenuation before it is lost into the noise floor. In free space attenuation is minimal but with obstacles it can be substantial. This is why a Wi-Fi router in a window can be hard to use within another room in the house but the same router is detectable from a hill a mile away.

The attenuation rate is an average based upon a hollow building with solid walls.

Common building materials attenuate signals to different amounts based on their density and the signals wavelength.

A higher wavelength signal such as L band (1-2GHz) will be attenuated more than VHF (30-300MHz) for example.

A long wavelength signal like HF will suffer minimal attenuation making it better suited to communicating through multiple brick walls.

The layer cake house

A brick house is not just brick. It’s bricks, concrete blocks, glass, insulation, stud walls, furniture and surfaces of varying absorption and reflection characteristics. Modelling every building material and multi-path precisely, is possible, given enough data and time due to the exponential complexity of multi-path but wholly impractical.

A trade-off for accurate urban modelling is to assign a local attenuation value. It’s local since building regulations vary by country and era so a 1930s brick house in the UK has different characteristics to a 1960s timber house in Germany. Taking the brick house we can identify the nominal value by adding up the materials and dividing it by the size.

For example, 2 x solid 10dB brick walls plus a 5 dB margin for interior walls and furniture would be 25dB. Divide this by a 10m size and you have 2.5dB/m. Using some local empirical testing you can quickly refine this for useful value for an entire city (assuming consistent architecture) but in reality the *precise* value will vary by each property, even on a street of the same design, due to interior layouts and furniture.

Range setup

We created nine 4 metre tall targets using each of the 9 clutter classes in attenuation order from left-to-right, measuring 10x10m and fired radio-bulletsTM at them from a distance of 300m using the same RF power of 1W.

The following bands were compared: HF 20MHz, VHF 70MHz, UHF 700MHz, UHF 1200MHz, UHF 2.4GHz. SHF 5.8GHz.

The ITU-R P.525 model was used to provide a consistent reference.

Only the stronger direct-ray is modelled. Multipath effects mean that reflections will reach into some of the displayed null zones, with an inherent reflection loss for each bounce, but these are nearly impossible to model accurately and in a practical time.

Here are the results.

HF 20MHz

VHF 70MHz

UHF 700MHz

UHF 1200MHz

UHF 2.4GHz

SHF 5.8GHz

Findings

  • Dense materials, especially concrete, attenuates higher frequency signals more than natural materials like trees
  • Lower UHF signals perform much better than SHF with the same power
  • Higher frequencies with low power can be blocked by a single house, even after only 300m
  • HF eats bricks for breakfast!

Summary

Modern tactical UHF radios, and their software eco-systems, are unrecognisable from their cold-war VHF ‘voice only’ ancestors in terms of capabilities but have an Achilles heel in the form of material penetration. To get the best coverage the network density must be flexed to match the neighbourhood.

This is obvious when comparing rolling terrain with a urban environment but the building materials and street sizes in the urban environment will make a significant difference too. Ground units which communicated effectively in a city in one country may find the same tactics and working ranges ineffective in another city with the same radios and settings. Understanding the impact of material penetration will help planning and communication.

Posted on

DIY clutter

DIY clutter
In this video, a Port in west Africa poorly served by high resolution data is enhanced with DIY clutter. The result shows substantial attenuation from the shipping containers which due to their dynamic nature would not be current in commercial data.

Summary

High quality clutter data is necessary for accurate radio planning but it’s not always available when and where you need it. Using the new ‘My clutter’ feature at CloudRF you can define your own and use it in seconds. The data can be layered on top of existing data, regardless of resolution, to enhance accuracy with material attenuation conforming to ITU standards for forests.

Clutter data

Clutter data in modelling refers to objects on the earth’s surface. In radio this is typically buildings and trees which attenuate signals. These must be factored in to deliver accurate predictions. It’s normally very expensive and the market for this data is worth billions due to demand by global telecommunications firms. This puts it out of reach of most small businesses and organisations.

Material attenuation

Different materials attenuate RF in different ways. The impact depends upon the wavelength (eg. WiFi can’t go through thick walls) and the material (concrete absorbs more RF than wood). For more on this subject see the land cover blog here.

How

Use the form in the ‘Model’ menu to either define your own polygons and lines or upload your own bulk clutter as a KML file containing polygons.

Why

Here’s a few reasons why DIY clutter is necessary:
  • Based on market pricing it would cost over a billion dollars to purchase ‘commercial’ clutter data for the earth.
  • Based on experience, the lead time for clutter in Africa can be 6 weeks.
  • The expensive clutter data is out of date by the time you buy it. Shipping containers, construction, transport will change and they affect RF coverage.
  • Commercial clutter data doesn’t let you model future construction projects eg. a new building
Posted on

Modelling trees and buildings

3D buildings Land Cover or ‘clutter’ data describes obstacles on the earth’s surface a radio wave will have to negotiate like trees or buildings. The Land cover data is layered on top of the terrain data which can be either a smooth(er) Digital Terrain Model (DTM) or a rougher ground-with-clutter ‘Digital Surface Model’ (DSM) . For DTM and DSM this will allow you to simulate attenuation from a forest or city where it might not otherwise be represented in the data resulting in much more accurate results. It also means you can enhance basic coarse terrain data with fresh high resolution land cover to reflect recent construction developments.

Obstacles and attenuation

Radio waves are attenuated differently by different materials like vegetation and man-made buildings. The impact varies by frequency with very short wavelength signals like WiGig at 60GHz struggling to penetrate a paper wall whilst a long wavelength VHF signal can breeze through multiple brick walls. For accurate modelling its essential that land cover is considered otherwise you run a risk of an unrealistic prediction which will bear little resemblance to real world results.

Trees

Trees attenuate differently with dense coniferous pine forests attenuating the most. An ITU standard, ITU-R P.833-7 “Attenuation in Vegetation” exists to describe the impact of a forest of different signals. There have been many academic studies into this subject but the summary of this standard for a mixed deciduous/coniferous forest is as follows:
Frequency MHzAttenuation dB/m
1060.04
4660.12
9490.17
18520.3
21180.34
No two forests are the same but if you err on the side of caution you can budget for their impact with a rule of thumb that 10m of mixed forest is equivalent to 2dB of attenuation at 1GHz, 4dB at 2GHz and 8dB at 4GHz. Trees are defined in the Land cover used by the system with attenuation values aligned to ITU-R P.833-7 which scale with wavelength so the same forest block will attenuate a WiFi signal more than a PMR446 signal. The resolution varies by region with 30m for CONUS, 100m for Europe and 500m for the rest of the world.

Buildings

Man-made buildings are even less predictable due to the variety in size, density and materials used. Many studies have been conducted into building attenuation but they are region specific due to construction materials and designs. A good reference is a UK paper by OFCOM which merges multiple research papers and has a useful table of attenuation by material and frequency on page 39.
MaterialAttenuation dB/m
at 1GHz
Attenuation dB/m
at 10GHz
Concrete24>50
Brick3232
Plasterboard11>50
Wood5>50
Glass344
Ceiling board110
Chipboard22>50
Floorboard4>50
The system currently has four classes of building attenuation for high to low intensity developments. The attenuation rate is 1% of the solid material attenuation rate (eg. Brick is 32dB/m so a brick house in CloudRF is 0.32dB/m) since most buildings are largely hollow.

Land cover data

To enhance DTM and DSM models with 3D clutter, Land Cover data can be layered on top of the terrain to apply representative attenuation. This Land Cover data has been sourced from a variety of sources with up to 30m resolution. The total possible resolution possible is determined by the highest resolution data so if you are in New York City for example where there is 2m LIDAR / DSM data available, your effective resolution will be 2m.

30m Digital surface model

30m Digital surface model plus 30m land cover

2m Digital surface model (LIDAR)

2m Digital surface model (LIDAR) plus 30m land cover

Propagation models

Propagation models vary in complexity from the simple ‘one liners’ like the free-space-path-loss model to the incredibly complex Irregular Terrain / Longley Rice model. Most models are simple and must be used within their parameter limits (especially with empirical ‘measured’ models) otherwise you could get wildly inaccurate results. A good example is the well known Hata model which was designed for elevated cellular base stations serving mobile subscribers which were lower than it. If you use this model at the bottom of a hill you can get some incredibly unlikely results as the simple model has no concept of terrain only A to B. By using Land cover, the output from these simple models can be enhanced greatly to provide a result which is sensitive to changes in the terrain along a given path, similar to how the ITM model works.
A UHF repeater at the foot of some hills with 20m DSM only. By default the Sleipnir engine will restrict coverage to line-of-sight for simple models like Hata.
With knife-edge-diffraction enabled, the Hata coverage is free to roam beyond line-of-sight. The coverage becomes very optimistic to the west up in the hills as Hata has no concept of terrain and expects a clear shot from the base station to the mobile station.
With knife-edge and 30m Land cover enabled, the optimistic Hata coverage is still free to roam beyond line-of-sight but is now severely constrained by the dense forests and urban developments without modifying the model itself.

Forest example

Modelling little forest blocks far away from your tower is easy with accurate DSM data but modelling a huge forest where your tower is within it is a harder problem. Heights are all defined as relative to the ground so if you have a 10m tall forest which is represented as raised earth in a DSM model and your tower is 12m tall you will end up with a tower which is in fact 22m above the ground – not ideal! Instead, when working with the 30m DSM you should define your height as the height above the canopy which is 2m. Here’s a comparison using 30m DSM and 30m Land cover in west Virginia.

Free space path loss prediction for an outdoor WiFi router, 30m DSM.

30m DSM plus 30m Land cover.

Urban example

To demonstrate the attenuation of buildings, this example has an emitter (LTE eNodeB on 800MHz) equidistant between a city and some countryside. The attenuation of the urban land cover becomes obvious once applied which contrasts with the open fields and water.

Free space path loss prediction for an LTE eNodeB, 30m DSM

Free space path loss prediction for an LTE eNodeB, 30m DSM with 30m Land cover

Summary

Land cover is essential for accurate planning and is now supported at 30m resolution. Coupled with high resolution data like 2m LIDAR, you can now accurately model attenuation of different materials in a cluttered environment. More land cover data is planned for the near future along with an upgraded ‘my clutter’ interface to allow you to define your own forests or housing developments for areas where data may not be available.