Release notes¶
Compatibility table¶
These sets of versions have been tested for compatibility:
substra |
substrafl |
substra |
substra-tools |
substra-backend |
orchestrator |
substra-frontend |
substra-hlf-k8s |
substra-tests |
substra-chaincode |
---|---|---|---|---|---|---|---|---|---|
0.22.0 |
|||||||||
0.21.0 |
↑ OPEN-SOURCING ↑ |
substra |
substrafl |
substra |
substra-tools |
substra-backend |
orchestrator |
substra-frontend |
substra-hlf-k8s |
substra-tests |
substra-chaincode |
---|---|---|---|---|---|---|---|---|---|
0.20.1 |
|||||||||
0.17.1 |
|||||||||
0.17.0 |
|||||||||
0.16.0 |
|||||||||
0.15.0 |
|||||||||
0.14.0 |
|||||||||
0.13.0 |
|||||||||
0.12.0 |
|||||||||
0.11.0 |
|||||||||
0.10.0 |
|||||||||
0.9.0 |
|||||||||
0.8.1 |
|||||||||
0.8.0 |
|||||||||
0.7.0 |
|||||||||
0.6.0 |
|||||||||
0.5.1 |
|||||||||
0.5.0 |
|||||||||
0.4.0 |
|||||||||
0.3.0 |
|||||||||
0.2.0 |
|||||||||
0.1.0 |
↑ CLOSE-SOURCING ↑ |
substra |
substra-chaincode |
substra-backend |
substra-tests |
hlf-k8s |
substra-frontend |
substra-tools |
---|---|---|---|---|---|---|
Changelog¶
This is an overview of the main changes, please have a look at the changelog of every repos to have a full grasp on what has changed:
Substra 0.22.0 - 2022-10-20¶
Main changes
BREAKING CHANGE: the backend type is now set in the
Client
, the env variableDEBUG_SPAWNER
is not used anymore. Default value is deployed.
before:
export DEBUG_SPAWNER=subprocess
client = substra.Client(debug=True)
after:
client = substra.Client(backend_type=substra.BackendType.LOCAL_SUBPROCESS)
BREAKING CHANGE:
schemas.ComputePlanSpec.clean_models
property is now removed, thetransient
property on tasks outputs should be used instead.BREAKING CHANGE:
Model.category
field has been removed.BREAKING CHANGE:
train
andpredict
methods of all substrafl algos now takes datasamples as argument instead of X and y. This is impacting the user code only if he or she overwrite those methods instead of using the_local_train
and_local_predict
methods.BREAKING CHANGE: The result of the
get_data
method from the opener is automatically provided to the given dataset as__init__
arg instead of x and y within thetrain
andpredict
methods of allTorchAlgo
classes. The user dataset should be adapted accordingly:
from torch.utils.data import Dataset
class MyDataset(Dataset):
def __init__(self, x, y, is_inference=False) -> None:
...
class MyAlgo(TorchFedAvgAlgo):
def __init__(
self,
):
torch.manual_seed(seed)
super().__init__(
model=my_model,
criterion=criterion,
optimizer=optimizer,
index_generator=index_generator,
dataset=MyDataset,
)
should be replaced with
from torch.utils.data import Dataset
class MyDataset(Dataset):
def __init__(self, datasamples, is_inference=False) -> None:
...
class MyAlgo(TorchFedAvgAlgo):
def __init__(
self,
):
torch.manual_seed(seed)
super().__init__(
model=my_model,
criterion=criterion,
optimizer=optimizer,
index_generator=index_generator,
dataset=MyDataset,
)
BREAKING CHANGE:
Algo.category
: do not rely on categories anymore, all algo categories will be returned asUNKNOWN
.BREAKING CHANGE: Replaced
algo
byalgo_key
in ComputeTask.
GUI
Improved user management: the last admin cannot be deleted anymore.
Substra
Algo categories are not checked anymore in local mode. Validations based on inputs and outputs are sufficient.
Pass substra-tools arguments via a file instead of the command line. This fixes an issue where compute plan would not run if there was too many data samples.
Substrafl
NOTABLE CHANGES due to breaking changes in substra-tools:
The opener only exposes
get_data
andfake_data
methods.The results of the above method is passed under the datasamples keys within the inputs dict arg of all tools methods (
train
,predict
,aggregate
,score
).All method (
train
,predict
,aggregate
,score
) now takes a task_properties argument (dict
) in addition to inputs and outputs.The rank of a task previously passed under the rank key within the inputs is now given in the
task_properties
dict under the rank key.
This means that all opener.py file should be changed from:
import substratools as tools
class TestOpener(tools.Opener):
def get_X(self, folders):
...
def get_y(self, folders):
...
def fake_X(self, n_samples=None):
...
def fake_y(self, n_samples=None):
...
to:
import substratools as tools
class TestOpener(tools.Opener):
def get_data(self, folders):
...
def fake_data(self, n_samples=None):
...
This also implies that metrics has now access to the results of get_data
and not only get_y
as previously. The user should adapt all of his metrics file accordingly e.g.:
class AUC(tools.Metrics):
def score(self, inputs, outputs):
"""AUC"""
y_true = inputs["y"]
...
def get_predictions(self, path):
return np.load(path)
if __name__ == "__main__":
tools.metrics.execute(AUC())
could be replace with:
class AUC(tools.Metrics):
def score(self, inputs, outputs, task_properties):
"""AUC"""
datasamples = inputs["datasamples"]
y_true = ... # getting target from the whole datasamples
def get_predictions(self, path):
return np.load(path)
if __name__ == "__main__":
tools.metrics.execute(AUC())
Substra 0.21.0 (first OS release) - 2022-09-12¶
This is our first open source release since 2021! When the product was closed source it used to be named Connect. It is now renamed Substra.
Main changes
Admin and user roles have been introduced. The user role is the same as the previous role. The admin role can, in addition, manage users and define their roles. The admin can create users and reset their password in the GUI.
BREAKING CHANGE: remove the shared local folder of the compute plan
BREAKING CHANGE: pass the algo method to execute under the
--method-name
argument within the within the cli of the task execution. If the interface between substra and the backend is handled via substratools, there are no changes to apply within the substra code but algo and metricDockerfiles
should expose a--method-name
argument in theENTRYPOINT
.BREAKING CHANGE: an extra argument
predictions_path
has been added to bothpredict
and_local_predict
methods from allTorchAlgo
classes. The user now have to use the_save_predictions
method to save its predictions in_local_predict
. The user defined metrics will load those saved prediction withnp.load(inputs['predictions'])
. The_save_predictions
method can be overwritten.
Default _local_predict
method from substrafl algorithms went from:
def _local_predict(self, predict_dataset: torch.utils.data.Dataset):
if self._index_generator is not None:
predict_loader = torch.utils.data.DataLoader(predict_dataset, batch_size=self._index_generator.batch_size)
else:
raise BatchSizeNotFoundError(
"No default batch size has been found to perform local prediction. "
"Please overwrite the _local_predict function of your algorithm."
)
self._model.eval()
predictions = torch.Tensor([])
with torch.inference_mode():
for x in predict_loader:
predictions = torch.cat((predictions, self._model(x)), 0)
return predictions
to
def _local_predict(self, predict_dataset: torch.utils.data.Dataset, predictions_path: Path):
if self._index_generator is not None:
predict_loader = torch.utils.data.DataLoader(predict_dataset, batch_size=self._index_generator.batch_size)
else:
raise BatchSizeNotFoundError(
"No default batch size has been found to perform local prediction. "
"Please overwrite the _local_predict function of your algorithm."
)
self._model.eval()
predictions = torch.Tensor([])
with torch.inference_mode():
for x in predict_loader:
predictions = torch.cat((predictions, self._model(x)), 0)
self._save_predictions(predictions, predictions_path)
return predictions
GUI
GUI: the page size has been increased from 10 to 30 items displayed
GUI: Fixed: keep filtering/ordering setup when refreshing an asset list page
GUI: Fixed: filtering on compute plan duration
GUI: Fixed: the columns
name
,status
anddates
are displayed by default in the compute plans pageGUI: Fixed: broken unselection of compute plans in comparison page
GUI: Fixed: CP columns and favorites disappear on logout
GUI: the CP workflow graph now displays CPs with up to 1000 tasks, instead of 300
The test task rank now have the same behaviour as for other tasks (parent task rank + 1)
Substra
added
list_model
to the SDK clientDownload function of the client return the path of downloaded file
Local mode: add a check, a task output of type performance must have public permissions
Fix the filters on status for compute plans and tasks. This fix also introduces some changes: the value for the filters on status must now be a list (like for other filters, there is a OR condition between elements of the list) and its value must be
substra.models.ComputePlanStatus.{name of the status}.value
for compute plans andsubstra.models.Status.{name of the status}.value
for tasks.Example:
# Return all the composite traintuples with the status "doing"
client.list_composite_traintuple(filters={"status": [substra.models.Status.doing.value]})
changed the
metrics
andalgo
definition relying on substra tools. All the methods of those objects now takeinputs
andoutputs
as arguments; which areTypedDict
.
Substrafl
Throw an error if
pytorch 1.12.0
is used. There is a regression bug intorch 1.12.0
, that impacts optimizers that have been pickled and unpickled. This bug occurs for Adam optimizer for example (but not for SGD). Here is a link to one issue covering it: pytorch/pytorch#80345In the PyTorch algorithms, move the data to the device (GPU or CPU) in the training loop and predict function so that the user does not need to do it.
Substra 0.20.1 - 2022-08-24¶
BREAKING CHANGE: Connectlib is now named Substrafl.
BREAKING CHANGE: Python 3.7 support has been dropped.
BREAKING CHANGE: in the CLI, only the cancel, profile, login and organization commands are now available.
BREAKING CHANGE: in substra, Compute task outputs are not hardcoded anymore. This makes it possible to explicitly specify model permissions, instead of having to follow a rule-based logic. The compute task permission field has been deleted. The outputs field on compute task should be used instead.
BREAKING CHANGE, in substrafl:
torch Dataset has been added as an argument of
TorchAlgo
to preprocess the data._local_train
is no longer mandatory to overwrite any more. Its signature passed from(x, y)
to(train_dataset)
._local_predict
is no longer mandatory to overwrite any more. Its signature passed from(x, y)
to(predict_dataset)
._get_len_from_x
has been deleted
BREAKING CHANGE: rename
schemas.ComputeTaskOutput
toschemas.ComputeTaskOutputSpec
BREAKING CHANGE: in local mode, each client has its own organization_id. Removed the
DEBUG_OWNER
mechanism.
Instead of:
client = substra.Client(debug=True)
clients = [client] * 2
do:
clients = [substra.Client(debug=True) for _ in range(2)]
client1_org_id = clients[0].organization_info().organization_id
Assets’ names can now be edited in the GUI, and in library (thanks to new methods
update_compute_plan
,update_algo
andupdate_dataset
methods that allow editing names)In substrafl:
Default batching has been added to predict.
A seed can be set in torch algorithms.
GPU execution has been fixed (the RNG state is now set to CPU in case the checkpoint has been loaded on the GPU).
In substra:
inputs
field has been added tosubstra.sdk.schemas.tupleSpec
andsubstra.sdk.models.tupleModel
.models and performances have been added as
outputs
tosubstra.sdk.schemas.tupleSpec
andsubstra.sdk.models.tupleModel
.inputs
andoutputs
fields have been added to the Algo model.The
Client.organization_info
function now returns a modelOrganizationInfo
instead of adict
GUI:
log scale can be used to display compute plan performances.
non-metadata columns (i.e. default elements such as status/tasks, creation date, start date / end date / duration) can be selected/removed in custom columns.
The number of tuples uploaded in each batch by default is now 500 (instead of 20). This parameter can be changed using the
batch_size
parameter from theadd_compute_plan_tuples
function.zoom controls have been added in the compute plan workflow view.
the compute plans filtered list can be reset when clicking on a refresh button.
fix issue on compute plan tasks display
Substra 0.17.1 - 2022-07-13¶
fix an orchestrator issue when upgrading existing instances
Substra 0.17.0 - 2022-07-11¶
BREAKING CHANGE: The metric concept does not exist anymore. Instead the metric is simply an algo belonging to the metric category.
BREAKING CHANGE: Convert the test task to two tasks: predict task + test task. This change was necessary on the way to have a generic task.
BREAKING CHANGE: The method to add tasks to a compute plan:
Client.update_compute_plan
is renamedClient.add_compute_plan_tuples
.BREAKING CHANGE: Remove CLI commands: add, get and list.
Library: Added functions to download the model of a strategy:
The function
substrafl.model_loading.download_algo_files
downloads the files needed to load the output model of a strategy according to the given round. These files are downloaded to the given folder.The
substrafl.model_loading.load_algo
function to load the output model of a strategy from the files previously downloaded via the the functionsubstrafl.model_loading.download_algo_files
.Those two functions works together:
download_algo_files(client=substra_client, compute_plan_key=key, round_idx=None, dest_folder=session_dir)
model = load_algo(input_folder=session_dir)
GUI: A compute plan can be canceled from the GUI.
GUI: The compute plan workflow can be viewed in the GUI.
GUI: Filters on duration for compute plans and tasks.
Substra 0.16.0 - 2022-06-27¶
GUI: filter on compute plans metadata using the Filters button in the compute plans listing
- BREAKING CHANGE: new filtering and ordering functionalities for list methods in SDK:
- new syntax for filters:
filters={key:["value1", "value2"]}
new possible filters: name, owner, metadata, permissions, compute_plan_key, algo_key, rank, dataset_key, data_sample_key.
For instance:
client.list_dataset(filters={compute_plan_key="d193a5eb", owner=["org-1"]}, ascending=True)
- new syntax for filters:
- new ordering possibilities:
Order compute plans and tasks on creation date, start date, end date. Default: creation date.
Order all assets by ascending or descending creation date (or another date for compute plans and tasks). Default: descending.
For instance:
list_testtuple(filters={data_sample_key=["d193a5eb",”15256612”], compute_plan_key="18a5dfc6"}, order_by='creation_date', ascending=True)
See documentation for a more detailed view on the filtering and ordering possibilities
New strategy in Substrafl: Newton Raphson
Substra 0.15.0 - 2022-06-13¶
Maintainers also check :ref:`upgrade notes <deployment/upgrade_notes:Substra 0.15.0>`
BREAKING CHANGE: Nodes were renamed into Organizations.
This also impacts functions like
client.list_node()
andclient.node_info()
which becomeclient.list_organization()
andclient.organization_info()
.The OneNode strategy has been renamed SingleOrganization.
GUI: The newsfeed in the GUI is automatically refreshed every minute.
GUI: you can customize the columns of the Compute Plan listing and share this configuration with other users.
GUI: The omnisearch was implemented in the GUI, with a single search bar to search for compute plans, datasets, algorithms and metrics with their name or key.
An initialization round was added to centralized strategies (this has been done for the upcoming download model feature):
Each centralized strategy starts with an initialization round composed of one composite train tuple on each train data organization.
One round of a centralized strategy is now: Aggregation -> Composite training.
Strategy rounds start at 1 and the initialization round is now 0. It used to start at 0 and the initialization round was -1. For each composite train tuple, aggregate tuple and test tuple the metadata
round_idx
has changed accordingly to the rule stated above.