Data-flo Major Update in April 2024

Data-flo is continuously evolving as we learn from our community and adapt its features to accommodate new use cases.

In April 2024, a new version of Data-flo will be availble for testing on https://next.data-flo.io/, the large-scale upgrade includes significant changes to the user interface, improved ability to process large datasets, additional adaptors, and enhanced sharing permissions.

A diagram of Data-flo version 1 compared to the upgraded version of Data-flo.

Release Plan

Pre-release Phase: from 1 April 2024 to 30 June 2024

New version is made available for testing on https://next.data-flo.io/

The current version remains unchanged and available on https://data-flo.io/. A banner is added to https://data-flo.io/ to invite users to try the new version.

Release Phase: from 1 July 2024 to 31 march 2025

The new version ( https://next.data-flo.io/) replaces the current version and replaces https://data-flo.io/.

Workflows on the old version are migrated to the new version. Workflows created on https://next.data-flo.io/ will continue to be available on https://data-flo.io/.

The old version will be available to user on https://legacy.data-flo.io/ until 31 march 2025.

Sunset Phase: from 1 April 2025 onwards

The old version (https://legacy.data-flo.io/)will be turned off.

Release Notes

Highlighted updates:

1. Enhanced workflow description editor: Click the information icon in the top right () which will open the editor. By default you get two side-by-side panels, on the left type your text, and on the right you’ll see the formatted text. This uses markdown language, allowing you to include links, bullets, and even insert images.
2. Adaptor names and descriptions on adaptors: Once an adaptor is added to the canvas, users have the option of adding a custom description. This is displayed under the adaptor name on the canvas, making it easier to track both the adaptor type, and specific use.

3. Debug mode is now integrated into your workflow. Simply click the run icon () and your adaptors will be outlined in green (no issues!) or red (uh-oh!). If you get a red adaptor, simply select it, and you’ll get more details on why it didn’t run successfully. It’s helpful to click run () often as you’re building your workflow to test each adaptor step by step as you build.
4. Sharing permissions: you can now share your workflows with others selecting one of four different levels of privilege!

   

Interface updates

1. Workflow page organization: Your homepage is also called the workflow page where all your saved workflows are stored and displayed. This update of Data-flo allows users to both sort and group their workflows by Name, Date Created, Date updated, and Access Role

2. Reposition the Add New Step screen: use the icon on the top right of the screen to move the side panel from the bottom to the right side of the screen - your choice!

   

Adaptor updates:

See comprehensive list of all adaptors here.

New adaptors

Adaptor Type	Adaptor Name	Description
Transform	add-jittering	Adds random jittering to specified numeric columns in a datatable.
Transform	aggregate-rows	Aggregates values in specified columns grouped by rows.
Transform	calculate-column	Performs a mathematical calculation on two datatable columns.
Transform	duplicate-column	Duplicates a column in a datatable.
Transform	format-time-column	Formats date/time values in a datatable column.
Import	import-file-from-http-request	Imports a file from an HTTP request.
Import	import-from-excel-file	Imports data from an Excel file in XLSX format.
Import	import-from-json-file	Imports data from a file in JSON format.
Transform	import-list-from-text-file	Imports a list from a text file.
Transform	query-datatable	Runs a RBQL (Rainbow Query Language) on a datatable.
Transform	rename-newick-leaf-labels	Renames leaf labels in a Newick tree.
Transform	run-openai-model	Runs an OpenAI language model.
Transform	run-replicate-model	Runs a Replicate model.
Workflows	run-workflow	Runs another Data-flo workflow.
Transform	sample-datatable	Samples rows in a datatable.
Transform	split-geographical-coordinates	Splits geographical coordinates in a datatable column to a latitude & a longitude columns.
Transform	summarise-datatable	Summarises information about specified columns in a datatable.
Transform	transform-columns	Transforms values in specified columns using an OpenAI language model.
Workflows	workflow-repeater	Runs a data-flo adaptor(s) from another Workflow within the current Workflow. This adaptor applies the reference Workflow adaptors to each row of your current Workflow.