Category: Self-hosted

Posted on by

SOOTHSAYER 1.6 released

The latest release of our private server, SOOTHSAYERTM is now available with bigger limits, better reference data, VSphere 8 enterprise GPU support, enhanced security controls and improved performance.

SOOTHSAYER-1.6-flyer-Download

GPU clutter attenuation

The GPU engine can now model clutter attenuation in the same way as the CPU engine, only much faster. This functionality works for our global land cover for forests, global buildings data, and even custom clutter for BYO obstacles. Clutter profiles for regions and seasons are configured through the interface as normal.

Global buildings

We have replaced OpenStreetMap buildings with Microsoft buildings, generated from machine learning trained on satellite imagery. This provides much better data for planning in rural areas and countries which traditionally have suffered from poor data coverage. In the comparison below in Khartoum, Sudan, MS buildings produce a much more conservative result owing to their comprehensive coverage with OSM’s crowd-sourced data lacks.

A SOOTHSAYER server now has global buildings which can be side loaded by country into the data folder or fetched on-demand via api.cloudrf.com when connected to the internet.

Antennas and utilities

Previously SOOTHSAYER’s antenna database was sparse but now it ships with over 500 patterns from major OEMs as well as improved tools to validate, convert different text formats, import ADF patterns and search. Pattern images have been retired for dynamic plots direct-from-source so what you see is what you get and the custom plots are indistinguishable from OEM patterns in both the UI and outputs.

The new look database lets you find patterns quicker than ever with responsive filters. Type the first few letters of what you’re looking for to reduce search results.

The rating feature lets your team score patterns so “the good ones” are always top of the list.

Enhanced MANET limits

Limits for the Multisite API and it’s flasgship MANET tool have been enhanced. You can now do 500km radius / 310 Mile MANET sites and networks that are 2000km wide.

If you try to create a network across the Atlantic Ocean this won’t work….yet.

VSphere 8 support

SOOTHSAYER has been successfully tested on enterprise servers (HPE) with VSphere 8 and a Nvidia T4 GPU. VSphere 8 uses UEFI firmware (instead of BIOS) by default which is becoming the norm as enterprise moves towards trusted platform architectures.

It has also been tested with Proxmox, Virtualbox and Azure environments. We recommend Proxmox for SOOTHSAYER for the widest GPU support (RTX etc) and VSphere for enterprise GPUs.

You can install the VM on a GPU laptop with Linux but this is only recommended for people who like a challenge. We have a tutorial video and instructions available for this.

ATAK plugin

It comes with a free TPC signed plugin for ATAK 4.8: https://github.com/Cloud-RF/SOOTHSAYER-ATAK-plugin

Simple data storage with a second drive

You don’t need to mount an SMB share for your data anymore! (You can if you want to though…)

Just attach a second disk to the VM and reboot. You decide the size. Your terrain data and user files will be saved to this disk. The database remains encrypted inside the first disk.

If you don’t then you can still use it but will get warnings telling you about disk limits it in the CLI:

Change log

1.6.0

API v3.8.10

3.8.10 (2023-08-16)

  • Improvement: New default dBm and dB colour keys
  • Improvement: Enhanced admin dashboard for server
  • Improvement: Auto-rotate function for misaligned patterns refined
  • Fix: A private server will fetch MS buildings from CloudRF before OSM
  • Fix: Path and route ATAK data packages not working in ATAK.
  • Fix: Downloading .ant returns NAN and INF.
  • Fix: GPU elevation pattern downtilt aligns with CPU

3.8.9 (2023-08-05)

  • Improvement: More verbose errors for engine/data failure
  • Improvement: Handling and logging corrupt DTM files

3.8.8 (2023-07-31)

  • Improvement: MANET KMZ now returns points and heatmap
  • Fix: Refactored antenna pattern parsing
  • Fix: Fixed bug in antenna auto-alignment
  • Fix: Sorting MANET files to use latest when requesting KMZ

3.8.7 (2023-07-25)

  • Improvement: MANET networks in the archive show average value for each of the columns.
  • Improvement: Added support for MS 2m buildings via buildings option
  • Improvement: Using buildings will now only go to OSM as a last resort
  • Improvement: System clutter profiles adjusted to make vegetation more dense
  • Fix: Remote tile download not working with DTM tiles.

3.8.6 (2023-07-21)

  • Improvement: Custom antenna optimisations
  • Feature: Redesign of antennas database to make use of ADF files.
  • Fix: Handling legacy .ANT patterns not referenced to 0dB PEP

3.8.5 (2023-07-05)

  • Improvement: GPU can now do 500km radius with a minimum 180m resolution
  • Improvement: Multisite limit increased to 2000km for wide networks
  • Improvement: Logarithmic custom antenna pattern radiation

3.8.4 (2023-06-23)

  • Improvement: GPU calcs can now use custom clutter
  • Fix: Storing antenna gain before feeder loss adjustment
  • Fix: area and coverage values returned in KMZ balloon not being calculated.
  • Fix: When using PPA tool network name is appended with _PPA to separate from other calculations in the archive.
  • Fix: Interference tool not working with GPU calculations.

3.8.3 (2023-04-24)

  • Feature: Added session endpoint to get current load state of API in the response of a PNG image.
  • Feature: Area calculation site and network parameters now accept Latin letters which are not in the ISO basic Latin alphabet.
  • Feature: Added remaining_calculations_since_last_purchase to my-metrics API.
  • Feature: Added edges argument to a BSA request to allow an area to be defined for a BSA area which is cut out during processing.
  • Feature: Requesting tiff file type on an area request returns both 3857 and 4326 projections.
  • Improvement: Requesting tiff file type via the archive is now compressed to reduce size and transfer time.
  • Improvement: Additional error reporting and logging.
  • Fix: Antenna search form now shows all patterns for a given OEM when name is selected even if search string not present in description/filename
  • Fix: site is not a required field.
  • Fix: Google Earth throws an error upon first loading a coverage layer in relation to the PPA tool. This was causing errors as the PPA tool is called immediately after the loading of the coverage layer of which both the transmitter and receiver locations were the same.
  • Fix: Some temporary files not being removed during cleanup process.
  • Fix: User clutter files being copied to wrong location when using GPU processing engine.
  • Fix: Google Earth throws an error upon first loading a coverage layer in relation to the PPA tool. This was causing errors as the PPA tool is called immediately after the loading of the coverage layer of which both the transmitter and receiver locations were the same.
  • Fix: User clutter files being copied to wrong location when using GPU processing engine.
  • Fix: Submitting receiver.lat or receiver.lon to area endpoint causes calculation failures.
  • Fix: Timings no longer output for error messages on production.
  • Fix: credits returned from preferences not factoring in requested calculations.
  • Fix: Loading KMZ into Google Earth would immediately fire off a PPA calculation.
  • Fix: Calculations made in Google Earth would always have TEST network name.

3.8.2 (2023-03-28)

  • Improvement: Polarity for custom patterns
  • Fix: Rx gain NULL after using interference tool
  • Fix: All KMZs contained the GE ppa tool
  • Fix: Removing gain from ADF patterns so all are referenced to 0dB PEP
  • Fix: Using Signal to Noise units with a receiver sensitivity of 0 dB caused an issue
  • Fix: Google Earth PPA tool could fail when both markers were stacked up. Now fails gracefully.
  • Fix: Antenna model and OEM search filters was using OR vs AND

3.8.1 (2023-03-02)

  • Improvement: GPU engine has improved Huygens diffraction
  • Improvement: Colour steps can now be any value between -30 and +30
  • Improvement: New buildings are now fetched synchonously so will block for request #1 up to a max of 30s
  • Improvement: Extended bandwidth lower limit from 10kHz (0.01MHz) to 1kHz (0.001MHz).
  • Improvement: Extended noise floor lower limit from -140 to -150 dBm.
  • Improvement: Requesting a network which doesn’t exist in a mesh calculation will indicate up to 3 network names which have been used recently.
  • Improvement: Chatbot authentication process simplified. The user ID will always be the ID of the Chatbot.
  • Improvement: Extended bandwidth lower limit from 10kHz (0.01MHz) to 1kHz (0.001MHz).
  • Improvement: Extended noise floor lower limit from -140 to -150 dBm.
  • Commercial: Ibiza is no longer the anonymous-full-spectrum-party-island due to abuse. A plan is required to test.
  • Fix: Tightened restrictions on users on free plan.
  • Fix: Interference tool failing to produce result.
  • Fix: Bug in SLEIPNIR JSON report for very low power levels eg. 0dBm with feeder loss. Fixed as 1.6.2
  • Fix: Points megapixel calculation ignores radius in favour of point distance. Hard limit of 64MP
  • Fix: Confusing error messages when running a mesh calculation.
  • Fix: Chatbot outputs any returned error messages when a multisite API request fails.
  • Fix: Tightened restrictions on users on free plan.
  • Fix: Interference tool failing to produce result.
  • Fix: Show the callsign of the device which is authenticated with Chatbot as this is more useful than a device ID.
  • Fix: Requesting 3857 projections for GPU calculations (viewshed, BSA, multsite) returning HTTP 404.
  • Fix: Selecting a bad antenna ID would throw a HTTP 500 with no output error message.
  • Fix: PPA URL outputs will return a secure encrypted URL so as to now expose information via the public URL.
  • Fix: Multisite endpoint reporting validation errors for some imperial units.
  • Fix: Multisite endpoint not honouring imperial units.

UI v3.8.8

3.8.8 (2023-08-15)

  • Update: Satellite database updated to 01/08/2023.
  • Fix: Calculate ERP after switching antenna pattern
  • Fix: BSA colour key no longer disabled when BSA fails

3.8.7 (2023-07-31)

  • Feature: Added a download KMZ button to the MANET tool.
  • Fix: Skips polygons and polylines with no coordinates when uploading KML as clutter.
  • Fix: Bit error rate and modulation drop downs missing in Google Earth.
  • Fix: GPU engine enabled no longer hides the play button to allow calculations to be made without changing settings.
  • Fix: Gracefully fail with a message when unable to load path profile results from the archive.
  • Fix: Gracefully fail with a message when an image could be loaded properly for some scenarios.
  • Fix: Multi-point results hover in the sky when turning off 3D terrain.

3.8.6 (2023-07-13)

  • Fix: Buildings on the Path Profile Chart were all being bunched up on the left when land cover was turned OFF.
  • Feature: “My patterns” antennas makes use of recent updates to API endpoints and draws pattern plots rather than outputting JPEG images.
  • Fix: Antenna menu labels tidied and widths standardised. Horizontal is now H, Vertical is now V.

3.8.5 (2023-06-23)

  • Fix: Handling blocked sentry script with link to adblocker FAQ
  • Fix: Metric/Imperial logic in Multisite tool
  • Fix: MANET links not showing if no preference set
  • Fix: ERP not recalculated when reloading from archive
  • Fix: Deleted archive images now throw a error

3.8.4 (2023-05-23)

  • Enhancement: Loading page with async loader for large JS assets
  • Enhancement: Added “Hide Node” functionality to the MANET context menu.
  • Enhancement: Sets the default antenna gain when selecting an antenna.

3.8.3 (2023-04-24)

  • Fix: Removed “Plane Earth Loss” from models list.
  • Fix: Using “Line of Sight” model automatically sets diffraction to “Off”.
  • Fix: Improved error handling and reporting.
  • Fix: Issue when changing the colour schema while best site analysis is processing a drawn polygon.
  • Feature: API load status light added in top navbar upon page load.
  • Fix: Marker pin on MANET nodes at top of plumbline rather than at ground level.

3.8.2 (2023-03-28)

  • Enhancement: Welcome screen while loading ui.
  • Enhancement: Added reference data import dialog for CSV,KML,GeoJSON.
  • Enhancement: Increased 1000 row limit for CSV to 60k (Make sure your device has lots of memory if you use this)
  • Enhancement: BSA KML polygons via reference data form
  • Enhancement: MANET KML locations via reference data form
  • Enhancement: KML reference points via reference data form
  • Fix: Issue when entering invalid DMS coordinates.

3.8.1 (2023-03-03)

  • Improvement: Upgraded CesiumJS to the latest version.
  • Fix: Mouse hover tooltip value lookup to use the correct schema for multiple layers with different schemas.
  • Fix: Error messages from API for MANET tool not being output.
  • Fix: MANET failed links are styled which is confusing, particular when zoomed out and dashed line is not visible.
  • Fix: Best server tool able to misfire on GPU area calculation upon click.
  • Fix: Allow for imperial units with MANET heatmap which adds locking to the dis value to prevent unit mismatch.

More information

To request more details see here, or to get a quote or a demo VM, get in touch from your official email.

Posted on by

System updates – 3.6 (Nov 22)

Here’s a roundup of what’s changed since June. As always it’s been a busy period of development which we’ve matched with a focus on maintenance and bugfixes. Our backlog of feature and change requests stands at 28 for the UI and 23 for the API with all priority issues resolved.

The deck is now clear for a dynamic network planning feature which will be integrated in December.

UI changes, June to November

400km radius
Custom WMS
JSON templates
Custom WMTS
Satellite data

With 19 new features/improvements and 69 bugfixes in this period, the user interface has matured considerably due in large part to user feedback from the community. We’ve said no to a few feature requests also which is necessary to keep the interface technology agnostic and easy to use. That doesn’t mean we can’t support complexity or special waveforms directly in the API.

  • Radius increased to 400km for aircraft (3.0)
  • GPU engine integrated with best server and CSV analysis tools (3.0)
  • New 300m resolution for airborne broadcasting (3.0)
  • Received Signal Received Power (RSRP) option added to measured units (3.1)
  • Account information dialog (3.2)
  • GPU Best Site Analysis (3.2)
  • Custom WMS and WMTS map sources (3.4)
  • Satellite database updated (3.4.2)
  • Multi azimuth antenna patterns (3.5)
  • Route, multipoint and path path tools updated for all possible measurement units (3.5.1)
  • JSON templates system (3.6)

The long surviving Google Earth interface lives on and has been given a fresh wind thanks to code compatibility libraries which mean we no longer have to worry about breaking it with new Javascript. Long live Google Earth!

UI Changelog

## Version 3.6.1 (2022-11-28)

- Fix: Layer from archive would not load when clicking on the layer name or selecting "Add tower(s) only to map" button.
- Fix: Processing modal window getting stuck when using the Interference or Super Layer tools.
- Fix: Display the power of items loaded from the archive in the selcted power unit.
- Fix: Dynamically move navigator controls down to make space for imagery layer list.
- Fix: Prevent crash when disabling path profile analysis before calculation completes.
- Fix: Refreshes displayed clutter after deleting all clutter using the clutter manager.
- Fix: Selecting a template in the Google Earth interface crashes.
- Fix: MANET nodes sometimes clipped by the terrain resulting in partially hidden markers.
- Fix: Best site analysis tool sometimes goes into "zombie" state under bad network conditions.

## Version 3.6 (2022-11-10)

- Feature: Templates functionality makes use of the new API endpoints and JSON files.
- Feature: Added button to download the JSON template for both custom and system templates.
- Feature: Added modal to add/remove favourite system templates.
- Feature: Dropdown select of templates indicates if the template is a system or custom template by prepending with either "System" or "Custom" respectively.
- Fix: Styling issues on custom template save/delete modal.
- Fix: Reorganised "My Account" modal as was becoming cluttered. Logout button moved to top and managers split into a single row. API usage moved to the bottom.
- Fix: Some API errors which are returned as an array of values would not be parsed as an array, instead the error displayed would be a generic "Request failed." message which wasn't indicative of the error.
- Fix: Don't fire a GPU request when loading a GPU template.
- Fix: Adjusting the `rxs` field manually would not update the receiver sensitivity slider.

## Version 3.5.1 (2022-10-31)

- Feature: Route, Multipoint and Path tools able to display SNR, dBuV and RSRP units as well as dBm.

## Version 3.5.0 (2022-10-18)

- Feature: Multi-azimuth antenna support by sending a comma-separated list of values, for example, `0,120,240`.

## Version 3.4.2 (2022-09-21)

- Update: Satellite database updated to 2022-09-22.
- Feature: Fly to PPA when loading from the archive.
- Fix: Custom pattern vertical beamwidth not visible in the Google Earth interface.
- Fix: Profile drop down not populated and opening the clutter manager in the Google Earth interface.
- Fix: Sensitivity is loaded when selecting a template.
- Fix: Map attribution interfered with satellite timeline controls.
- Fix: Loading PPA from archive and then hovering over chart would cause a crash.

## Version 3.4.1 (2022-09-14)

- Fix: Template saves coax type.
- Fix: Problem toggling the azimuth mode in the Google Earth interface.
- Fix: Problem loading a template in the Google Earth interface.
- Fix: Switching between templates with engine set to GPU and CPU runs area calculation with incorrect engine.
- Fix: Saving a custom map URL cuts off part of URL after an "&".
- Fix: Suspends error logging when library blocked by client.
- Fix: Handle missing preferences.

## Version 3.4.0 (2022-08-31)

- Feature: Custom WMS and WMTS URLs for imagery layers.
- Feature: Attribution text lists API providers.
- Fix: Pruned dead and unsuitable map sources, such as "Blue Marble".
- Fix: Problems with editing coordinates.
- Fix: Antenna patterns not loading properly in some browsers.

## Version 3.3.0 (2022-08-23)

- Feature: Sets free space model with diffraction off and shows warning for best site analysis.
- Feature: Select last network when opening archive to match the last row.
- Feature: Make clutter go to ground.
- Fix: Improved area calculation error handling.
- Fix: Selecting a network in archive drop down when some items have no network would cause a crash.
- Fix: Legacy colour key images not displayed.
- Fix: Super layer KMZ quick-download button fixed for "Merge network"

## Version 3.2.0 (2022-08-15)

- Feature: Account Information dialog.
- Feature: GPU Best Site Analysis.
- Fix: Updates to colour schemas to match recent changes to colour schema API.
- Fix: Buildings layer cannot be removed.
- Fix: Receiver sensitivity icons missing.
- Fix: Creating a clutter profile does not refresh the list.
- Fix: RSRP breaks MANET.
- Fix: Opacity slider doesn't work.
- Fix: MANET reload by name click and best server for imperial units mi/f.
- Fix: Crash clicking layer checkbox while layer is loading.
- Fix: Default RSRP schema not always selected.
- Fix: Interference tool is trying to map colours to colour key.
- Fix: Crash saving clutter polylines with less than 2 coordinates or polygons with less than 3 coordinates imported from kml.
- Fix: Crash downloading MANET metrics report when node evauation failed.
- Fix: Power unit in MANET node details dialog displayed as undefined.
- Fix: Node percentanges on MANET metrics report sometimes NAN.
- Fix: Points CSV validation no longer works.
- Fix: Entity already exist in collection when disabling MANET.
- Fix: Crash adjusting bsa sliders when result loading failed.
- Fix: BSA images not recoloured on colour schema changed.
- Fix: Custom antenna polar plot not displayed in Google Earth.

## Version 3.1.0 (2022-07-18)

- Feature: Added Received Signal Received Power (RSRP) as an option to the "Measured Units" dropdown.
- Fix: Deleting multiple selected networks will now give the name of the networks rather than the ID.
- Fix: Selecting a PSK/QAM modulation will only show supported PSK/QAM bit-error-rate values, and likewise when selecting a LoRa modulation.
- Fix: Deleting a network from the archive now gives a better modal message rather than a browser `alert()` window.
- Fix: Occasional crash when computing azimuth and tilt for the mouse tootltip for satellite mode when the position has not been computed yet.
- Fix: Can't find variable `Map` in Google Earth interface.
- Fix: Crash in Google Earth when logging error information.
- Fix: Prevents crash in Google Earth interface when changing clutter profiles.
- Fix: Formatting issue in Google Earth.
- Fix: Helper dialog describing channel noise adjustment when changing bandwidth
- Fix: Tx Height AGL and RF Power textboxes extending over unit drop downs.
- Fix: Crash in Google Earth interface when you change properties that would update the map location when in web UI.

## Version 3.0.1 (2022-07-06)

- Feature: Added support for 180m and 300m resolutions.
- Fix: GPU is populated in dropdown of processing engines when GPU isn't available.

## Version 3.0 (2022-07-04)

- Feature: Radius limit now 400km.
- Feature: GPU processing no longer handled through WebSocket, instead goes through the same API as CPU processing for simplicity. Removed button to close GPU processing engine, instead you now just switch back over to CPU mode from the dropdown in the "Output" menu.
- Feature: Integration of GPU engine with other tools including best server and CSV import.
- Fix: Colour schema list gets populated with some empty values when using GPU mode.
- Fix: Prevents failure on area calculation due to invalid input from logging an error.
- Fix: Interference dialog not always populating networks.
- Fix: Duplicate entry for gpu in the engine drop down.
- Fix: Some antenna patterns were not being saved in templates.

API changes, v3.0 June to 3.6 November

With 30 improvements and features and 28 bugfixes, the API has largely remained the same from a developer’s perspective but it now has more consistency between the CPU and GPU engines, more choice with defining antenna azimuths and more useful error messages to help developer’s make better choices and save time.

Under the hood, the ATAK interface has had a major refactor to support customer’s official TAK Server’s with our infamous chatbot. A new chatbot management interface has been especially created which can validate certificate chains using OpenSSL to warn of issue before you get to them eg. “This certificate is invalid” or “This key is incorrect” or “Cannot ping your TAK server”.

Offline JSON file templates
Multi-Azimuth requests
Best Site Analysis for ATAK

GPU integration

The new GPU engine has received several updates throughout this period to add models, gains, antenna patterns, multi-azimuth patterns and output styling inline with the CPU engine, SLEIPNIR.

In September, the powerful Best Site Analysis capability was integrated with ATAK. When the SOOTHSAYER bot is connected to the TAK server, any polygon sent to it will generate a BSA layer.

Due to demand, the maximum radius for the GPU engine was increased to 150km to support broadcasting. During GPU testing we were generating 100 mega pixel (100,000,000 points) heatmaps in a few seconds which is too large for a client. We are working on a scalable solution to visualise images of this resolution as an image pyramid.

SLEIPNIR mods

The SLEIPNIR CPU engine is now at version 1.6.1 and has conditional smoothing of coarse DTM, knife edge diffraction from clutter (note we already do this for surface models) and slightly (3dB) more optimistic diffraction following feedback from a mountain rescue team.

Received Signal Received Power (RSRP) is not native to the engine instead of the API and the reliability variable can now be used for non-ITM models to provide a 10dB tuning margin where 50% is 0dB and 99% is 9.9dB path loss.

The maximum radius has been increased to 400km for airborne platforms. To support the larger radius, we’re also offering a new low resolution of 300m.

API management

The V1 “URL” API will be retired at the end of this year. Users who have integrated with this API should migrate to the new JSON API as soon as possible. The new JSON API has been powering the web interface for almost two years.

Reference: https://cloudrf.com/documentation/developer/

API Changelog

## Version 3.6.0 (2022-11-25)

- Deprecation: Added deprecation message to `v1` API endpoints. The `area` and `path` will be removed on the 31st December 2022. Please migrate any scripts to the new JSON API and see https://cloudrf.com/documentation/developer/
- Fix: Chatbot welcome-storm with other chatbots.
- Fix: Obsolete paths on terrain coverage map and HTML PPA response.
- Fix: Improvements to automation testing.
- Fix: Sending malformed JSON to a `v2` API endpoint now returns a useful error message.
- Fix: Deprecated `v1` API endpoints were incorrectly counting API credits.
- Fix: Power value of 1mW for GPU fails.

## Version 3.5.0 (2022-11-10)

- Feature: Moved from database-driven to JSON-driven templates with new API endpoints to manage custom templates.
- Feature: System templates can be retrieved and saved as a favourite using new endpoints.
- Fix: Race condition on GPU processing files as its so damned fast.
- Fix: Public shares for GPU layers needed reprojecting to EPSG 3857.

## Version 3.4.0 (2022-10-18)

- Feature: Gain support for GPU engine.
- Feature: Antenna support with azimuth and tilt for GPU engine.
- Feature: Multi-azimuth support for antennas by sending a comma-separated list of values in the `antenna.azi` value, for example, `0,120,240`.
- Fix: Path tool on TAK successfully returning only about 50% of the time.
- Fix: Area requests to `v1` API returning colour key data in improperly formed string values.
- Fix: `clutter` command on TAK chatbot working again.
- Fix: Resolved issue with custom colour schemas and low resolution screens.
- Fix: Sanity check ADF files.
- Mod: Default clutter templates adjusted. Urban height reduced to no more than 2m.
- Mod: GPU engine max radius capped at 10km (CPU max = 400km).

## Version 3.3.0 (2022-09-22)

- Feature: Support for TAK server 4.7 through Chatbot and ATAK.
- Feature: Extended response information when executing `id` to Chatbot through ATAK.
- Feature: Support for GPU BSA on Chatbot with the multipoint tool through ATAK.
- Feature: Added new chatbot manager to spin up chatbots on demand which interact between TAK servers and API.
- Feature: Extended preferences functionality to allow for custom imagery.
- Feature: Delete all clutter button added to clutter form.
- Fix: Processing engine saved in templates.
- Fix: Commercial restrictions messages now return HTTP 403 (Forbidden) rather than HTTP 400 (Bad Request).

## Version 3.2.0 (2022-08-15)

- Feature: System colour schemas no longer able to be deleted by a user so that users will always have a list of schemas to choose from.
- Feature: Height ceiling increased to 120km over 60km.
- Feature: When using Greyscale GIS colour schema no longer returns validation error for mixing outputs and schema.
- Feature: Improvements to the PPA KMZ export with fresnel support and new icons for transmitter, receiver and obstructions.
- Fix: Improvements to the functionality of the colour schema generator.
- Fix: Creating a BER colour schema fails with a HTTP 500.
- Fix: Update to `preferences` stripping out underscores from colour schemas.
- Fix: Google Earth layer no longer allows to manage colour schemas. Added notice with workaround instructions.

## Version 3.1.0 (2022-07-18)

- Feature: Improvements to `preferences` to extend to additional values and give information about a users plan.
- Feature: Improvements to precendence of preferences, where in some cases a previous request values will be taken over a users preferences.
- Feature: Colour palette bucket limit increased to 75.
- Feature: Return request to `preferences` as a correct JSON response.
- Feature: Users with a private server will now be indicated when they make a request to `preferences`.
- Feature: Colour pallette creator extended to support Bit-Error-Rate measured units.
- Feature: Bit-Error-Rate modulation extended from 10e-6 to 10e-8.
- Feature: Allowed support for Received Signal Received Power (RSRP) measured units with an `out` value of `6`.
- Feature: Added default RSRP colour key.
- Feature: SLEIPNIR 1.6.1 calculates and returns (LTE) RSRP if selected
- Feature: SLEIPNIR 1.6.1 uses reliability for non-ITM models with a 10dB range. 50% = +0dB, 99% = +9.9dB
- Fix: Successful deletion of networks in the archive are returned with a success `message` rather than `error`.
- Fix: Correct names of plans when making a request to `preferences`.
- Fix: BER colour palette in JSON response was missing unit.
- Fix: Improvements to the default `PATHLOSS.dB` colour key.
- Fix: Hata model high-plateau issue resolved in SLEIPNIR 1.6.1 

## Version 3.0.1 (2022-07-06)

- Feature: Resolution supported up to 300m.

## Version 3.0 (2022-07-04)

- Feature: Improvements to GPU processing with the use of remote processing nodes.
- Feature: GPU produced images are now styled in the API based on the specified colour schema.
- Feature: Extended `data` API to retrieve a tile based only on tile name.
- Fix: Logic issue with users mixing credits and plan was incorrectly refusing service to users with expired plans, but spare credits.
- Fix: Custom clutter profiles are created with the list of 101 to 199 clutter codes.
- Fix: Obtaining height data from OpenStreetMaps would sometimes fail.
- Fix: KMZ download failing on equator

That’s all folks.

Posted on by

Bandwidth and noise

As communications systems have grown in capacity, they have expanded in physical bandwidth in an increasingly congested RF spectrum.

Effective digital communications planning requires more than knowledge of antennas and propagation fundamentals. It now needs an intimate understanding of bandwidth and noise to co-exist and communicate efficiently.

Unfortunately, this key aspect of modern RF systems is often taught badly, or in some cases not at all, leading to often unwelcome surprises for equipment operators in the field. As a technology and market agnostic platform we’ve observed poor bandwidth knowledge in many markets, but notably MANET and 5G, both of which are accelerating the deployment of wideband systems, often with little to no planning beyond a topographic map study.

Both radio markets are evolving from narrower channel technologies, which in the case of MANET and the VHF Combat-Net-Radio (CNR) it replaces was measured in KHz so they need to update their theory training content and associated software to convey these potentially complex topics to novice students in a digestible manner.

Increasing bandwidth increases noise, which reduces coverage

Teaching noise

As bandwidth increases, so does channel noise. This simple concept might seem easy to remember for a student but without visual aides, and since the demise of analog systems; audible aides, it is hard to demonstrate in practice.

A good teacher may show visual aides like noise charts, FFTs, spectograms and a bad one may show some Johnson-Nyquist formulas buried within an all-day powerpoint which is not helpful except for getting paid.

FFT showing a narrow signal and wideband interference

A student can tick the right box on their exam(s) but spend their career wasting bandwidth and struggling to establish communications because they believe that big is better – it isn’t, or worse still, that bandwidth has no effect on the coverage since that’s a function of transmitted power and/or height. Having an intimate understanding of the interplay between bandwidth, receiver sensitivity and thermal noise will make spectrum users more efficient, effective and considerate.

Bandwidth MHzThermal noise (dBm)
0.1-124
1-114
2-111
4-108
8-105
16-102
32-99
64-96
Bandwidth thermal noise table based on a temperature of 21C

Which waveform is best?

Comparing digital radios is complicated due to the myriad of features, waveforms and software.

Given a particular waveform it will have characteristics such as a minimum Signal-to-Noise Ratio (SNR) value which it requires to achieve a symbol rate necessary to deliver a fast data link for example. This dB value must sit proud of the noise floor so if the noise floor is high at -90dBm, coverage will be reduced and conversely, by taking it to somewhere quiet eg. -110dBm, the coverage will improve by 20 decibels – a huge difference.

To compare waveforms precisely, the same noise floor should be simulated, initially, with fixed values to eliminate random error in field testing. The sensitivity values will be somewhere between 3 and 20dB depending on what the waveform and target Bit-Error-Rate (BER) is.

Bit Error Rate (BER) describes the ratio of errors in a data stream. An ok value is 10-3 or 1 bad bit in a 1000 or 0.1%. This increases with noise until a signal is unrecoverable. For more on BER see an older blog here.

For ground radios designed for noisy environments, a BER of 10-2 (1 error in 100 or 1%) is used here for extracting our planning thresholds from a chart of SNR curves. For an airborne system without obstacles this could be higher, for example 10-5.

Signal to Noise Ratio for different modulation schemas against error rates.

A narrow waveform eg. QPSK gives better coverage and works better in noisy conditions. This is the fallback telemetry mode used in many data radios.

A wide waveform eg. QAM64 is capable of better throughput and delivering high bandwidth streams such as HD video.

The best radio is one which can use different waveforms to satisfy both coverage and capacity.

Modelling bandwidth: A tutorial

Modelling RF Bandwidth and noise

Quick reference guide

A quick reference guide for using bandwidth and noise is available here. For other guides see here.

221025_SNR_quick-guideDownload

Conclusion

Bandwidth and noise is essential knowledge for anyone deploying wideband systems or comparing waveforms.

RF theory training can be enhanced (and needs to be) with visual tooling to let students quickly observe the impact of different inputs in a controlled environment with templates to minimise user error.

For information on how SOOTHSAYER can help with signals training see here.

Posted on by

Dynamic network planning with hardware APIs

Dynamic MANET network planning

Location aware radios

Modern digital radio systems often have Application Programming Interfaces (APIs) for remote management. They also commonly have Global Navigation Satellite System (GNSS) modules for location awareness which enables “network maps” of where the nodes are. These vendor maps are great at showing where nodes are now (provided they are in coverage!) but what they lack is the ability to plan where nodes could be moved to.

When you combine these key features with our open standards modelling API you get a powerful new capability which allows you to observe the network now, and by moving or adding a simulated node, see the network in the future.

Case study – Trellisware

TW-950 Shadow

TrelliswareTM are a market leader in Mobile Ad-hoc Networks (MANET) whose versatile radios work where others fail due to their spectrum efficient TSM waveform. It’s currently in service with Government, Commercial and First responder markets.

The radios have a comprehensive API and integrated software services which allows for remote operation on a IP based network.

We were loaned some TW-950 Shadow radios to explore API integration. In a few days we were able to create an API client (in Python) to interface between the Trellisware API and our RF modelling API on SOOTHSAYER 1.3 which has a MANET planning tool.

Our Python client would interface with the radio network via the Ethernet connector on the side of a TW-950 radio from where the API would expose node metadata.

We would fetch this metadata (as JSON) and package it into a JSON document compatible with our Points API, which powers our MANET and route analysis tools. In the interface’s MANET tool, a new ‘play’ button pulls in the network document and models it through our fast API. Each link is tested using the real parameters, with minimal user interaction.

The flowchart is depicted below.

Ethernet adapter for connecting the hardware to SOOTHSAYER

Once the data is displayed in the SOOTHSAYER interface, an operator can choose to move a node by dragging it or add an entirely new node, using settings of their choice, into the mix. This new node will be modelled alongside the rest of the nodes (many-to-many) to visualise what the impact of the new node will be.

Trellisware (Blue) and CloudRF (Yellow) APIs and components

A faster OODA loop

The Observe, Orient, Decide, Act (OODA) planning loop is an established concept which determines success in fast moving communications environments such as a fire or Police incident.

By combining real-time situational awareness with live modelling we can exercise scenarios and reach sound decisions much faster than by either guessing or using trial and error as is often the case in tactical communications in complex environments. The benefit is faster more efficient use of limited resources.

A Trellisware network map enhanced with two planned nodes to the right.

Demo: Do we launch the drone?

Deploying a drone equipped with a radio is a guaranteed way to fix MANET network issues, especially in cluttered urban environments. It’s resource intensive and risky as the drone is valuable, the radio, and network it enables, is arguably more valuable plus the battery life is very limited so this asset must be used sparingly, which is where planning comes in.

Placing a repeater drone “overhead” is not needed, unless you’re using it for observation also. If the drone has a vertical dipole, overhead would actually be the worst place for it due to the donut shaped antenna pattern as some early OEMs learnt the hard way.

A better place for a communications relay drone is at altitude but on the edge of the target network where it will be at less risk (and will present less risk to net members below it in the event of failure) but it will still be able to serve as an effective repeater.

As recent MANET demos have proven, this repeater could be effective from several miles away.

Demo of dynamic MANET planning in SOOTHSAYER with Trellisware radios

Look ahead

Now that we have a simple standards based, repeatable, method for adding live radios we’ll roll this into future SOOTHSAYER releases, the next of which is 1.4, scheduled for Q3 2022.

Customers can make their own plugins in any language to get their hardware data packaged as a JSON document for either SOOTHSAYER’s MANET tool or direct to the points API which powers it. You can find example scripts in various languages on our Github site and we are available to assist with bespoke integrations.

References

API examples: https://github.com/Cloud-RF/CloudRF-API-clients/tree/master/APIv2

API specification: https://cloudrf.com/documentation/developer/swagger-ui/

Contact

If you are a hardware manufacturer or integrator looking to add this type of capability into either your own map or ours, get in touch as we know what we’re doing and it will save you years of R&D.

Email support@cloudrf.com for more information on how you can achieve dynamic planning quicker.

Posted on by

Connecting smart cows to moove data

Smart cow

Smart farming

Smart farming is using Internet of Things (IoT) technologies in agriculture to enable efficient use of resources.

For this blog we’re focused on cattle farming on large, fence-less farms in New Zealand. The farms in question are vast and remote so connectivity options are limited. This is why an off-grid sub-GHz LPWAN network is ideal due to its long range and the requirement to only send small, infrequent, packets of data.

For the solution to be cost-effective compared with Satellite, as little infrastructure as possible is needed which in this case is a LPWAN gateway on a pole, some collars for the herd and an app to manage the system via a web service.

Siting the LPWAN gateway(s) properly is critical to achieving not only coverage across the farm(s) but to reduce the number of gateways, which reduces complexity and cost.

Sub GHz LPWAN on the farm

An 868MHz LPWAN signal can go for many miles under the right conditions. We know this well from powering the Helium LPWAN network’s planning tool, Helium Vision, where people can communicate data 50 miles with a fraction of a watt of RF power and an omni directional antenna.

Despite it’s useful diffraction properties which enables it to work non-line-of-sight (NLOS), it’s still sensitive to obstructions so clutter on the farm such as buildings and trees needs modelling accurately. CloudRF has 10m Landcover for New Zealand from the European Space Agency and 10m DSM from the LINZ Geospatial agency.

These data sets are adequate for most outdoor scenarios but are not fine enough to model a farm complex of buildings, such as tall grain silos, metal sheds and seasonal obstacles. For high resolution you could source your own surface model, as our customer Halter did…

Farm buildings and silos

Use case: Halter

Halter are a novel agri-tech startup focused on cattle management with a unique solar powered collar.

They needed accessible RF planning software to help their engineers site LPWAN gateways. Having used and liked Cloud-RF, they needed higher resolution surface models of the farms, and no pesky API restrictions!

They also planned to build their own tools on top of our powerful physics based API which is smart as it allows their R&D team to focus on their primary product, and not waste time reinventing the wheel.

Their options were either buy expensive commercial data or self generate data using a drone and photogrammetry software such as Pix4D. Given the prohibitive cost of high resolution commercial LiDAR, it would only take a few jobs to make a return on the purchase of a decent drone!

https://halterhq.com/

Halter purchased a private Keyhole Radio server from us which included the API they needed. The server runs as virtual machine and crucially, lets them import their own terrain data.

They were quickly able to import high resolution, organic data into their server as GeoTIFF files. This allowed them to work with data which was very current, even hours old, so would be an accurate model of tree heights and man made obstructions.

The terrain format accepted by Keyhole Radio and SOOTHSAYER is GeoTIFF, Int16 resolution and WGS84 (EPSG:4326) projection.

LPWAN coverage on a farm in New Zealand

1m resolution

It wasn’t all plain sailing though, they found that there was a limit to the physical tile sizes our server could use caused by memory. The solution was to reprocess the large tile into smaller tiles to make it digestible.

A 5000 x 5000 GeoTIFF at Int16 resolution will require 50MB of disk space. If this is 5m LiDAR, the physical width is 25km x 25km. Our engine can super-sample, so if you used this tile, but requested 1m resolution, it would create a raster in memory measuring 25,000 x 25,000 pixels which would need 1.25GB of memory.

For 1m resolution however, tiles measuring 1000 x 1000px would only require 2MB of disk and memory. You may need to load in a few, lets say 16, to do your model but that’s still only 32MB.

You could also resolve this by increasing the memory available to the server but it’s recommended to prepare data into smaller parcels. We support 1m resolution in our API but don’t hold a lot of 1m data sets due to their substantial cost and size. If you already have 1m data, a Keyhole Radio or SOOTHSAYER server is the answer.

1m resolution

Summary

Cloud-RF’s powerful API is ideal for efficient smart farming.

Our private servers will let you take it to the next level with terrain data you can source yourself, no API restrictions and as a bonus, they work without an internet connection!

Finally, all our jokes are offal.

Posted on by

SOOTHSAYER 1.1b released

soothsayer-1.1b-atak-route-analysis

Today we published a new release for our SOOTHSAYER self-hosted server focused on data management and stability.

It brings the VM up to date with the public service in terms of new features (Route analyis, Multipoint analysis, Mesh by visible, 3D fresnel) and fixes multiple user/data management issues identified through recent testing.

Thank you testers!!!

Software versions in October 2021 release

You can verify your versions from your administration dashboard.

keyholeradio: 1.1b
documentation: 1.4
chatbot: 2.2
sleipnir: 1.4.4
API: 2.3
ui: 2.2

Get the remote update

Login to the console with the account from your administration guide and type the command keyholeradio-update

Download the image

If you want to download a new VMware image contact support from your work email for the 9GB download link. You will need a SOOTHSAYER or KEYHOLE RADIO license to enable the software.

Evaluation licenses are available on request…

route-analysis
route-analysis
3D fresnel zone

SOOTHSAYER changelog

[1.1b 27/10/2021]
Added route analysis in UI
Added merge-by-visible in UI
Added multipoint analysis in UI
Added 3D fresnel zone in UI
Added database reset warning and confirmation
Added authorisation option for external 3D buildings source
Added Admin can change dashboard password
Added support for 1m LiDAR

Fixed legacy opacity tag removed from KML
Fixed remote tile server with global coverage via us.cloudrf.com
Fixed database backup to host / outside VM
Fixed mesh radiocheck to limit parallelism on smaller machines
Fixed SOOTHSAYER bot location to middle of area of operations 
Fixed local map tiles so they no longer create bogus messages when clicked
Fixed custom WMS urls so they can now contain hyphens
Fixed API keys updated in admin dashboard to APIv2 convention
Fixed logging standardised for all services
Fixed even spacing for points along freehand routes & polylines
Fixed database reset now instant when requested
Fixed database integrity & permissions check with auto-repair 
Fixed TAK route analysis as a KMZ data package instead of CoT

[1.0.4b 14/10/2021]
Added clear warning dialog to database reset
Changed database reset to table truncate instead of full factory reset which is in manage_db
manage_db utility now exports to shared data folder for backups
Fixed bug in factory reset which stopped a clean db install
Fixed issue in users table editor where password asterisks were hidden
Fixed issue in users table editor where updated column was not tracking edit times
Fixed issue in users table editor where ATAK UID could not be reset. Can now use #
Factory database reset no longer requires a restart

[1.0.3b]

Added checks during startup and update to ensure 'data' directory is writable.
Added keyholeradio version to admin dashboard.
Added changelog.txt and relevant permission settings during startup.

Fixed issue where banner displayed incorrect WMS URL.
Fixed issue where a user's 'atakuid' failed to update when done manually.
Fixed issue where bot would fail to auth due to NULL value in 'atakuid'.
Fixed issue where no users existed if additionally seats added. New users created upon loading of admin dashboard.