Our latest field test was focused on drive testing novel antennas by UK SME Far Field Exploits (FFX) around the Somerset countryside with Trellisware radios.

Previously, we validated diffraction models using LTE800 in the Mountains. The outcome of that cold test highlighted Deygout as the most accurate diffraction model when paired with empirical cellular models. For this much warmer antenna drive testing, we used lower frequencies and a lower mast in an area with many trees which presented a challenge for both legacy cellular models and LiDAR.

Testing highlights

• Average Root Mean Square Error of 7.4dB
• Average Modelling Error of 4.4dB
• Automated data collection with ATAK plugin
• New “General Purpose” model developed
• New “GP” clutter profile for use with GP model

Test setup

The test area was in and around the small town of Somerton in Somerset. This town sits in rolling countryside featuring farms, high hedgerows and blocks of trees. A railway line with road humpback bridges bisects the town. The town has a small housing estate under construction which did not feature in our buildings data.

The base station was a wide-band Omega panel elevated 5m above the ground and connected to a Trellisware spirit radio. The radio was operated across several UHF bands, each with 1.2MHz bandwidth, and live positions observed on WinTAK using cursor on target (CoT).

The antenna testing vehicle was fitted with a roof mounted magnetic antenna bracket which connected to a spirit radio. This mount allowed different antennas to be swapped out. As a result we were able to test both a Hascall Denke MPDP675X4 and a FFX Sigma 3.

Data logging

We know customers and OEMs like to voice opinions about radios, waveforms and antennas but without solid measurement data it’s just noise with a lot of bias and emotion.

Data beats emotions every day!

As an antenna OEM, FFX developed the ATAK spectrum survey app to streamline collection of field measurements for antenna testing in different environments.

The logging application used the Trellisware radio’s API to fetch link metadata from the local radio and save it to the SD card as a CSV file.

The ATAK plugin enabled a large quantity of high quality measurements to be efficiently collected. As a result we were able to execute several test cycles in a short space of time – just as well as it was hot (for the UK) and Harry had no air conditioning…

The CSV files were downloaded from the phone and loaded into the CloudRF calibration utility for analysis.

The survey data was filtered to remove results weaker than the theoretical noise floor at -113dBm.

We were planning to use a measurement error of 2dB for the high quality radios (a cell phone is 3dB) but owing to the high temperate of the mobile radio in the car we used 3dB as receiver performance degrades with temperature.

At first look

The first pass comparison of the data showed a ~15dB delta between modelling and field measurements with LiDAR, prior to tuning. Using the ITM model and a high reliability value (99%) this only reduced several decibels and clearly needed more work. Ideally the model should align within 10dB so clutter tuning can then be used to reduce this towards 6dB.

ITM uses the complex Vogler multi knife edge diffraction model which is accurate for hills but needs tuned clutter to handle soft obstacles. Using cellular models, as we did in LTE800 field tests, didn’t produce the same results due presumably to the lower mast height and frequencies, even when enhanced with Deygout diffraction.

A new model

Through curve fitting we identified alignment with the P.525 reference model and a 20dB constant representing observed system losses. When enhanced with the Deygout 94 diffraction model this produced excellent alignment with the more challenging beyond-line-of-sight areas. Many signal paths on the route had multiple obstructions so a multiple knife edge model (MKED) was essential.

We have created a new model from these settings called the General Purpose Model. It is frequency and height agnostic which makes it ideal for ground and air based links and much more versatile than empirical equivalents which must be operated within a restricted performance envelope. Like all our models it must be used in conjunction with a diffraction model and tuned clutter to deliver accurate beyond line of sight results.

In our opinion, modern developments in processing and clutter data especially have rendered legacy empirical models largely obsolete. The modern way to fit modelling to measurements is to focus on precise clutter data not old path loss curves.

In the screenshot below, the car drove up a hill where it fell off the network behind a prominent knoll before reacquiring the network later on. This knoll was the second of two obstructing hills for this section of the route. The modelling predicted more coverage due to the chosen receive threshold, -107dBm, which was based upon 6dB above the thermal noise floor which was -113dBm at 1.2MHz bandwidth. It is very likely local noise was slightly higher.

ITU clutter values

Without clutter, the General Purpose (GP) model will be optimistic in most ground environments. It will be accurate over bare earth but where obstacles are present, it needs land cover and a clutter profile. Prior to developing the GP model, we did most of the tuning in the model using reliability (%) and only fine tuned with the clutter.

This is why older CloudRF clutter profiles eg. Minimal.clt have low values such as 0.05 dB/m for trees. With the GP model, the model itself is very simple and most alignment takes place within the clutter (profile). As a result, the clutter values used for GP are much denser. Our GP profile, created for this test has trees with a density of ~0.5dB/m, aligning with ITU-R P.833, attenuation in vegetation.

Diffraction logic has been re-balanced to accommodate ITU clutter values. Users using either the default ITM model or models without land cover are not affected. Legacy clutter profiles such as Minimal have not changed but you are advised to try the new GP model and associated GP clutter and see the difference for yourself.

Test parameters

Bandwidth: 1.2Mhz

Feeder loss: 1dB

Receiver height: 1.5m

Receive sensitivity: -107dB (6db above noise)

Noise floor: -113 dB

Model: General purpose / ITM

Reliability: 60% / 90%

Context: Average

Diffraction: Deygout 94 / Vogler (ITM)

Clutter Profile: Buildings 3dB/m, Trees 10m @ 0.5dB/m

Radius: 6km

Resolution: 5m

Results

The following table of results were from measurements conducted with the same base station, vehicle and radios. Only the vehicle antenna, and frequency, were changed in between tests. Once calibration had been achieved the area covered was extracted from the modelling. This is typically inverse to the frequency so a low frequency has better coverage than a high frequency at the expense of bandwidth – and both matter.

There are two standout results from the data: First is the low RMSE accuracy for the new GP model with tuned clutter compared with LiDAR which is satisfying given the challenging terrain and the second is the performance of the Sigma 3 on a frequency it is not officially rated for as it has a bottom end of 350MHz. The best alignment with the same settings was found to be with -5dBi receive gain confirming the antenna can be operated lower, and at range.

Once again, DTM with clutter has proven to be superior to LiDAR.

Antenna test	Model + Diffraction	Clutter profile	DEM	Receive gain dBi	RMSE error	Modelling error	Modelling area covered km2	Modelling area covered %
Hascall Denke MPDP675X4 on 1.4GHz	GP (60%) + Deygout 94	GP	DTM + 10m Land cover + 2m Buildings	2	9.4	6.4	19.2	17
Hascall Denke MPDP675X4 on 1.4GHz	ITM (90%)	N/A	LiDAR	2	15.2	12.2	12.4	11
FFX Sigma 3 on 415MHz	GP + Deygout 94	GP	DTM + 10m Land cover + 2m Buildings	2	6.6	3.6	89.9	79
FFX Sigma 3 on 415MHz	ITM (90%)	N/A	LiDAR	2	18	15	72.7	64
FFX Sigma 3 on 287MHz	GP + Deygout 94	GP	DTM + 10m Land cover + 2m Buildings	-5	6.2	3.2	86.1	76
FFX Sigma 3 on 287MHz	ITM (90%)	N/A	LiDAR	-5	15.1	12.1	63	56

The scatter plot for the 1.4GHz data shows the simple GP model to align closer to field measurements than the much more complex ITM model. Our conclusion is that the ITM model, and it’s Vogler diffraction, developed in the 1960s, pre-dates developments in computing and precision clutter so provides good performance across multiple hills, at range, but is inadequate for macro planning at “street level” resolution where density of obstacles must be budgeted for.

ITM continues to be a solid UHF broadcasting model but it was designed for hard obstacles. Retro fitting it with soft clutter, as we have done can improve its performance several decibels but for maximum accuracy, the simple General Purpose model with tuned clutter provides superior results.

Results Gallery

Tuned coverage and survey data is displayed on the same map showing the RMSE and Mean error.

Look ahead

The General Purpose model will go live on CloudRF in early July 2024 following more testing and then into SOOTHSAYER 1.8 later in the year.

The problem with tunnels and stairs

Whenever there’s been a major incident involving emergency services in complex urban environments the inquiry report has consistently highlighted radio communications failure despite significant developments in radio communications and 3D technology since the infamous 1988 Kings Cross Fire on the London Underground. The following tragic incidents all featured tunnels, stairs and communications failure:

• The 1988 Kings Cross Fire report highlighted radio communications only worked when they were in line of sight.
• The 2007 London underground bombings report highlighted that radio communications from trains to network control were inadequate or non-existent.
• The 2017 Grenfell tower fire inquiry highlighted inadequate radio communications between the incident control on the ground floor and fire fighters higher up the tower.

Limitations of (2D) radio planning tools

Radio planning tools are not used in emergencies. They’re complicated, slow and require a lot of knowledge to produce an accurate output. Even if a skilled operator were able to model a site before the event, currently they would be expected to model each floor of a multi-story building in isolation due to the “floorplan” design of current software.

The problem is indoor planning tools are built for corporate clients to achieve seamless Wi-Fi in every corner of the office, not to help a fire chief deploy a mesh radio network down stairs and then along a tunnel. The top end tools can do limited multipath, slowly, but not as an API which can be consumed by a third party viewer…

Most radio planning tools on the market, ourselves included, have the following limitations when it comes to complex urban modelling which we will explore in detail:

Using LiDAR as a 2.5D surface model

The abundance of free LiDAR data has made this high resolution data the standard for accurate outdoor RF planning and for several Fixed Wireless Access (FWA) tools, including free LiDAR based path tools, it is their core feature. We started using LiDAR in 2015 and know its limitations well; for example when point cloud LiDAR has been rasterised into GeoTIFF then it’s no longer 3D, it’s a 2.5D surface model which is useful for building heights and unsuitable for bridges, arches and tunnels.

A bridge or arch in a rasterised LiDAR model extends to the ground like a wall. In the screenshot below, a large ferris wheel is blocking line of sight through it as well as the elevated rail bridge across the river which is casting a shadow much larger than it would in reality.

Using a floor plan to model a building

For indoor Wi-Fi planning tools, the start point is typically a floor plan. This does not scale well with multi-story buildings or support vertical planning as it produces a 2D image of a 2D plan.

Many tools present 2D images in a 3D viewer, as we do, but the output remains 2.5D, as with rasterised LiDAR. The significant Wi-Fi attenuation presented by solid floors makes this simplified 2D floor-by-floor planning viable for corporate clients in offices but not in challenging environments or where a floor plan does not exist.

Direct ray only

Modelling multipath, or fast fading, is much more complex than the direct ray. For this reason, most tools only do the more powerful direct ray and even then some cannot do diffraction or obstacle attenuation as we do already. For the previously mentioned Wi-Fi planning tools, the current standard is to model obstacle attenuation only. By doing this a tool is able to simulate most of the coverage quickly for a given floor but for complete accuracy it must be augmented by a walk survey, which isn’t so quick. For some customers, a walk survey is just not possible.

Multipath effects will increase coverage beyond a direct ray simulation and cause phase issues like signal dead-spots and doppler spread where reflections increase bandwidth and overall noise. This effect can be observed indirectly via customer reviews for urban WISPs where people state their once good link quality reduced as more neighbours subscribed.

A 3D multipath API for 2024

We’ve been working on this full 3D capability since the 2022 Grenfell inquiry with valuable input from firefighters, mining experts and MANET radio OEMs. The first version of the engine is done and we’re onto API integration now.

Our GPU based design takes a 3D model, simulates propagation in all directions irrespective of floors including configurable reflections, surface refractivity, material attenuation and crucially it outputs to the open 3D standard glTF. It scales from small rooms to suburbs and everything in between so will be used for tunnels, multi-story buildings and outdoor multipath.

It will be integrated into our API first so other standards compliant viewers can visualise it and will then be integrated into our own 3D user interface. We can’t say what interfaces people will be using in the future but are confident that by aiming for open standards APIs we will ensure compatibility with phones, glasses and holograms.

Commercial plan

The 3D engine API will be a new feature within CloudRF Gold plans and our SOOTHSAYER server at no additional cost. It requires a GPU. We’re aiming to get a beta up on CloudRF in May/June and to ship this with the next major SOOTHSAYER release, currently scheduled for September.

Users will be allowed to upload models within their storage limits and execution time / accuracy will be scaled to fit within a reasonable time. Limits will be relaxed on SOOTHSAYER.

We are partnering with open standards based companies to integrate this into different viewers. One exciting partner we are working with now is Avalon Holographics. Their revolutionary display is able to display our rich engine output in a hologram format so it can be explored in three dimensions for maximum spatial awareness without additional hardware for viewers.

If you would like to get our open standard glTF models into your viewer, get in touch. If you can bring challenging BIM models or LiDAR scans of real tunnels and large buildings we would really like to talk to you.

Demo video

Recently, we added advanced diffraction models to CloudRF to complement our existing models. To validate the performance of the new Bullington and Deygout models, we took a field trip to the Highlands of Scotland to collect UHF measurements over rugged mountain terrain and through forests.

With these measurements we have validated and optimised our new models for this environment. We already had single-knife-edge diffraction, based on Huygen’s formula, and the Irregular Terrain Model (ITM) which uses Vogler diffraction. The Vogler model is known to be good but single knife edge has its limits which we have pushed.

Summary

The testing validated our investment into the complex multi-obstacle models we have added.

Both new models offer a significant improvement in accuracy, with no loss in performance for Bullington. We were able to model diffraction with higher accuracy over multiple challenging obstacles such as gradual convex slopes, ridges and valleys. Modifications have been made to the CPU and GPU engines which will be updated on CloudRF and SOOTHSAYER in due course.

Our key findings include:

• Single-knife-edge was optimistic
• Deygout was the most accurate, but slower
• Bullington provided the best overall performance
• 7.6dB accuracy achieved, including receiver error
• 2.4dB improvement on single knife edge model

Test environment

We selected a famously cold and remote valley in the Cairngorms national park for our test which has cell towers in the valley and a variety of local repeaters for TETRA, VHF and UHF PTT services. The challenging terrain is notoriously difficult for radio communications making it ideal for our purposes.

Using a test phone with 3dB of measurement error attached to the Vodafone 4G network and a portable Rohde and Schwarz spectrum analyser, we collected a variety of VHF and UHF measurements along a 22km circular mountain route covering a wide variety of terrain. From the data collected, the 800MHz LTE measurements proved the best examples of signal failure so we focused our post-analysis on these.

Throughout the LTE testing the phone attached to multiple local cells and experienced prolonged signal failure as expected in a remote mountain valley.

We filtered the results to isolate 634 RSRP readings from a single physical LTE cell, PCI 460, from which we would calibrate modelling. This cell was located at the start of our test route and was a high power LTE band 20 (800MHz) base station with 10MHz of bandwidth.

Trees and attenuation

The first, and last, few miles of the circular route was a mature Scots pine forest. Unlike dense Scandinavian pine forests, this was sparse with a relatively high tree canopy. A lighter tree clutter profile was used to represent the attenuation from these trees which impact UHF propagation.

Convex hill and a loss of signal

Beyond the forest, the route gained altitude into a mountain plateau where line of sight was lost. The shape of the hill meant any diffraction formula would have to model a gradual convex shape versus a simpler knife-edge obstacle.

The ascent and re-acquisition

As the route ascended a spur leading toward the ridge, the signal was reacquired beyond the snowline. This signal gain was gradual, starting as a diffracted signal from the lower convex hill which eventually became a direct signal at the summit, 7km away from the cell.

Summit switcheroo

The route traversed a high ridge which featured many gaps in our cell coverage in the test data. These gaps were because the LTE modem performed a handover to stronger cells which appeared as soon as they were “visible”. Depending upon the position along the ridge, it occasionally reverted to the original “460” cell at over 7km.

Descent into darkness

The steep descent from the ridge entered a obscured valley not visible from the cell.

This resulted in a prolonged loss of signal for several miles until the signal was reacquired toward the trees at the foot of the valley.

Results analysis

The LTE survey data was prepared as CSV and loaded into the CloudRF web interface for use with the coverage analysis tool. This provided live feedback on accuracy with user generated heatmap layers so the correct settings could be identified first visually using a fine colour schema and then numerically by the reported average error in decibels.

Whilst the site location and frequency was known, the power output was not so the first task was to match line of sight positions, such as on the ridge-line, to establish the power without any obstacles. From there, a tree clutter profile was created to match the tree measurements and finally the best model and context were selected. For this task, the generic Egli VHF/UHF model was chosen as a basic model on which to base the diffraction comparison.

As settings matured, the reported Root Mean Square (RMS) error reduced accordingly until it was below 8dB (including 3dB of receiver error). This was slightly better than the 8dB we achieved on our last field test with LTE800 previously and given the extreme context, spanning a diverse mountain range, this was an excellent improvement.

Subtracting receiver error gives modelling error in the range of 4.6 to 7dB; an excellent result for difficult terrain.

Diffraction model	Mean error dB	RMSE error dB	Modelling error dB	Comment
Single knife edge	5.2	10	7	Optimistic. May show false positive coverage.
Deygout	-1.7	7.6	4.6	Good. Can be conservative and is 50% slower but gives high assurance.
Bullington	1.4	8.9	5.9	Good. Can be optimistic but is as fast as KED and relatively accurate.

Coverage results

The scatter plot for the ascent to the ridgeline shows measured and simulated values. The steep drop at 2.5km and gap in results after 3.3km matches closely for the critical beyond line of sight region. The results start again once we ascended toward the ridge where the new models were conservative by 10dB whilst the simple knife edge model tracked the path loss curve – which was to be expected. All models aligned once line of sight was achieved at 6.3km.

Recommendations

The outcome of this testing has improved the accuracy of our diffraction models, identified optimisations for our clutter profiles and proved a simple path loss model can be very accurate beyond line of sight with the right diffraction model.

The API settings we used for the LTE800 cell and RSRP output are here. Note the custom clutter profile and fine colour schema.

{
    "version": "CloudRF-API-v3.9.5",
    "reference": "https://cloudrf.com/documentation/developer/swagger-ui/",
    "template": {
        "name": "Lochnagar LTE800",
        "service": "CloudRF https://api.cloudrf.com",
        "created_at": "2024-01-16T13:15:02+00:00",
        "owner": 1,
        "bom_value": 0
    },
    "site": "Site",
    "network": "LOGNAGAR",
    "engine": 2,
    "coordinates": 1,
    "transmitter": {
        "lat": 57.003155,
        "lon": -3.327424,
        "alt": 15,
        "frq": 806,
        "txw": 15,
        "bwi": 10,
        "powerUnit": "W"
    },
    "receiver": {
        "lat": 0,
        "lon": 0,
        "alt": 2,
        "rxg": 0,
        "rxs": -129
    },
    "antenna": {
        "mode": "custom",
        "txg": 19,
        "txl": 0,
        "ant": 0,
        "azi": 180,
        "tlt": 0,
        "hbw": 120,
        "vbw": 20,
        "fbr": 19,
        "pol": "v"
    },
    "model": {
        "pm": 11,
        "pe": 2,
        "ked": 2,
        "rel": 60
    },
    "environment": {
        "obstacles": 0,
        "buildings": 0,
        "landcover": 1,
        "clt": "SCOT4.clt"
    },
    "output": {
        "units": "m",
        "col": "PLASMA130.dBm",
        "out": 6,
        "ber": 0,
        "mod": 0,
        "nf": -120,
        "res": 10,
        "rad": 8
    }
}

Disclaimer

Climbing mountains in winter to test radio networks is dangerous, hard work which requires fitness, experience, skill and dedication to RF engineering. Only do this if you are serious about improving accuracy!