diff --git a/docs/examples/Power Flow Example.ipynb b/docs/examples/Power Flow Example.ipynb index 4372ad850c..10b0d0b0bc 100644 --- a/docs/examples/Power Flow Example.ipynb +++ b/docs/examples/Power Flow Example.ipynb @@ -1146,6 +1146,64 @@ "print(output_data[ComponentType.line][\"i_from\"])" ] }, + { + "cell_type": "markdown", + "id": "344efbe8", + "metadata": {}, + "source": [ + "#### Cartesian product of batch datasets\n", + "\n", + "It is possible to conduct a batch calculation of multiple datasets in form of a cartesian product of their scenarios.\n", + "Assume certain batch datasets with N1, N2, N3, ... scenarios. \n", + "This would give us $N1 * N2 * N3 * ...$ possible combinations via the cartesian product.\n", + "The resultant output data is in flat form and it has dimension of N1 * N2 * N3 with first dataset being the highest\n", + "dimension. \n", + "This can be beneficial in reducing complexity of implementation of such batch calculation \n", + "along with keeping the size of such resultant update_data to a minimum.\n", + "\n", + "```note\n", + "The data validation module mentioned in #validation-(optional) does not support cartesian product of datasets. \n", + "It is advised to combine the datasets them manually via numpy to use the validator.\n", + "```" + ] + }, + { + "cell_type": "markdown", + "id": "5de1a2f7", + "metadata": {}, + "source": [ + "We can combine the time series mutation for all n-1 contingencies together in following way.\n", + "Both the batch datasets are passed together in `update_data` in a list." + ] + }, + { + "cell_type": "code", + "execution_count": 29, + "id": "d690eeb7", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Output data has shape (30, 3)\n", + "[0 1 1]\n", + "[0 1 1]\n", + "[1 0 1]\n", + "[1 0 1]\n" + ] + } + ], + "source": [ + "output_data = model.calculate_power_flow(update_data=[n_min_1_mutation_update_specific, time_series_mutation])\n", + "print(\"Output data has shape\", output_data[ComponentType.line].shape)\n", + "line_output = output_data[ComponentType.line][\"energized\"]\n", + "print(line_output[0, :])\n", + "print(line_output[1, :])\n", + "print(line_output[10, :])\n", + "print(line_output[11, :])" + ] + }, { "attachments": {}, "cell_type": "markdown", @@ -1210,7 +1268,7 @@ }, { "cell_type": "code", - "execution_count": 29, + "execution_count": 30, "id": "b5f10bae", "metadata": {}, "outputs": [ @@ -1270,7 +1328,7 @@ }, { "cell_type": "code", - "execution_count": 30, + "execution_count": 31, "id": "1a221507", "metadata": {}, "outputs": [ @@ -1312,7 +1370,7 @@ }, { "cell_type": "code", - "execution_count": 31, + "execution_count": 32, "id": "541af620", "metadata": {}, "outputs": [ @@ -1320,7 +1378,7 @@ "name": "stdout", "output_type": "stream", "text": [ - "Iteration failed to converge after 20 iterations! Max deviation: 3.54512e-16, error tolerance: 1e-20.\n", + "Iteration failed to converge after 20 iterations! Max deviation: 3.54512293063893e-16, error tolerance: 1e-20.\n", "\n", "Try validate_input_data() or validate_batch_data() to validate your data.\n", "\n" @@ -1360,7 +1418,7 @@ }, { "cell_type": "code", - "execution_count": 32, + "execution_count": 33, "id": "20d8285c", "metadata": {}, "outputs": [], @@ -1371,7 +1429,7 @@ }, { "cell_type": "code", - "execution_count": 33, + "execution_count": 34, "id": "b702eb15", "metadata": {}, "outputs": [ @@ -1384,7 +1442,7 @@ "\n", "Failed scenarios: [3 7]\n", "Succeeded scenarios: [0 1 2 4 5 6 8 9]\n", - "Error messages: ['The id cannot be found: 1000\\n', 'Sparse matrix error, possibly singular matrix!\\nIf you get this error from state estimation, it might mean the system is not fully observable, i.e. not enough measurements.\\nIt might also mean that you are running into a corner case where PGM cannot resolve yet.See https://github.com/PowerGridModel/power-grid-model/issues/864.']\n" + "Error messages: ['The id cannot be found: 1000\\n', 'Sparse matrix error, possibly singular matrix!\\nIf you get this error from state estimation, it might mean the system is not fully observable, i.e. not enough measurements.\\nIt might also mean that you are running into a corner case where PGM cannot resolve yet.\\nSee https://github.com/PowerGridModel/power-grid-model/issues/864.']\n" ] } ], @@ -1406,7 +1464,7 @@ }, { "cell_type": "code", - "execution_count": 34, + "execution_count": 35, "id": "1ba71901", "metadata": {}, "outputs": [ @@ -1415,16 +1473,16 @@ "output_type": "stream", "text": [ "Node data with invalid results\n", - "[[ 9.99401170e-001 9.92685785e-001 9.94521366e-001]\n", - " [ 9.99347687e-001 9.86226389e-001 9.89352855e-001]\n", - " [ 9.99288384e-001 9.79654011e-001 9.84095542e-001]\n", - " [-2.66881060e+116 2.33997016e-302 6.70346672e-198]\n", - " [ 9.99151380e-001 9.66149483e-001 9.73298790e-001]\n", - " [ 9.99073166e-001 9.59205860e-001 9.67750710e-001]\n", - " [ 9.98988099e-001 9.52126208e-001 9.62096474e-001]\n", - " [-2.44756775e+092 5.35663612e-256 1.91838796e-203]\n", - " [ 9.98796126e-001 9.37530046e-001 9.50447962e-001]\n", - " [ 9.98688504e-001 9.29997471e-001 9.44441670e-001]]\n", + "[[9.99401170e-001 9.92685785e-001 9.94521366e-001]\n", + " [9.99347687e-001 9.86226389e-001 9.89352855e-001]\n", + " [9.99288384e-001 9.79654011e-001 9.84095542e-001]\n", + " [2.18565566e-312 4.89761332e-310 2.97079411e-313]\n", + " [9.99151380e-001 9.66149483e-001 9.73298790e-001]\n", + " [9.99073166e-001 9.59205860e-001 9.67750710e-001]\n", + " [9.98988099e-001 9.52126208e-001 9.62096474e-001]\n", + " [4.89761332e-310 4.89761332e-310 4.89761332e-310]\n", + " [9.98796126e-001 9.37530046e-001 9.50447962e-001]\n", + " [9.98688504e-001 9.29997471e-001 9.44441670e-001]]\n", "Node data with only valid results\n", "[[0.99940117 0.99268579 0.99452137]\n", " [0.99934769 0.98622639 0.98935286]\n", @@ -1467,7 +1525,7 @@ ], "metadata": { "kernelspec": { - "display_name": "venv", + "display_name": "power-grid-model", "language": "python", "name": "python3" }, @@ -1481,7 +1539,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.13.3" + "version": "3.14.2" } }, "nbformat": 4, diff --git a/docs/examples/Validation Examples.ipynb b/docs/examples/Validation Examples.ipynb index ed8d2c6fae..bada7c4eb2 100644 --- a/docs/examples/Validation Examples.ipynb +++ b/docs/examples/Validation Examples.ipynb @@ -103,7 +103,11 @@ "cell_type": "code", "execution_count": 2, "id": "d997e738", - "metadata": {}, + "metadata": { + "tags": [ + "raises-exception" + ] + }, "outputs": [ { "ename": "IDWrongType", @@ -113,8 +117,8 @@ "\u001b[31m---------------------------------------------------------------------------\u001b[39m", "\u001b[31mIDWrongType\u001b[39m Traceback (most recent call last)", "\u001b[36mCell\u001b[39m\u001b[36m \u001b[39m\u001b[32mIn[2]\u001b[39m\u001b[32m, line 2\u001b[39m\n\u001b[32m 1\u001b[39m \u001b[38;5;66;03m# Without validation\u001b[39;00m\n\u001b[32m----> \u001b[39m\u001b[32m2\u001b[39m model = \u001b[43mPowerGridModel\u001b[49m\u001b[43m(\u001b[49m\u001b[43merror_data\u001b[49m\u001b[43m)\u001b[49m\n\u001b[32m 3\u001b[39m output_data = model.calculate_state_estimation(symmetric=\u001b[38;5;28;01mTrue\u001b[39;00m)\n", - "\u001b[36mFile \u001b[39m\u001b[32m~\\power-grid-model\\src\\power_grid_model\\_core\\power_grid_model.py:125\u001b[39m, in \u001b[36mPowerGridModel.__init__\u001b[39m\u001b[34m(self, input_data, system_frequency)\u001b[39m\n\u001b[32m 123\u001b[39m prepared_input = prepare_input_view(_map_to_component_types(input_data))\n\u001b[32m 124\u001b[39m \u001b[38;5;28mself\u001b[39m._model_ptr = pgc.create_model(system_frequency, input_data=prepared_input.get_dataset_ptr())\n\u001b[32m--> \u001b[39m\u001b[32m125\u001b[39m \u001b[43massert_no_error\u001b[49m\u001b[43m(\u001b[49m\u001b[43m)\u001b[49m\n\u001b[32m 126\u001b[39m \u001b[38;5;28mself\u001b[39m._all_component_count = {k: v \u001b[38;5;28;01mfor\u001b[39;00m k, v \u001b[38;5;129;01min\u001b[39;00m prepared_input.get_info().total_elements().items() \u001b[38;5;28;01mif\u001b[39;00m v > \u001b[32m0\u001b[39m}\n", - "\u001b[36mFile \u001b[39m\u001b[32m~\\power-grid-model\\src\\power_grid_model\\_core\\error_handling.py:175\u001b[39m, in \u001b[36massert_no_error\u001b[39m\u001b[34m(batch_size, decode_error)\u001b[39m\n\u001b[32m 173\u001b[39m error = find_error(batch_size=batch_size, decode_error=decode_error)\n\u001b[32m 174\u001b[39m \u001b[38;5;28;01mif\u001b[39;00m error \u001b[38;5;129;01mis\u001b[39;00m \u001b[38;5;129;01mnot\u001b[39;00m \u001b[38;5;28;01mNone\u001b[39;00m:\n\u001b[32m--> \u001b[39m\u001b[32m175\u001b[39m \u001b[38;5;28;01mraise\u001b[39;00m error\n", + "\u001b[36mFile \u001b[39m\u001b[32m~/repos/power-grid-model/src/power_grid_model/_core/power_grid_model.py:169\u001b[39m, in \u001b[36mPowerGridModel.__init__\u001b[39m\u001b[34m(self, input_data, system_frequency)\u001b[39m\n\u001b[32m 167\u001b[39m prepared_input = prepare_input_view(_map_to_component_types(input_data))\n\u001b[32m 168\u001b[39m \u001b[38;5;28mself\u001b[39m._model_ptr = pgc.create_model(system_frequency, input_data=prepared_input.get_dataset_ptr())\n\u001b[32m--> \u001b[39m\u001b[32m169\u001b[39m \u001b[43massert_no_error\u001b[49m\u001b[43m(\u001b[49m\u001b[43m)\u001b[49m\n\u001b[32m 170\u001b[39m \u001b[38;5;28mself\u001b[39m._all_component_count = {k: v \u001b[38;5;28;01mfor\u001b[39;00m k, v \u001b[38;5;129;01min\u001b[39;00m prepared_input.get_info().total_elements().items() \u001b[38;5;28;01mif\u001b[39;00m v > \u001b[32m0\u001b[39m}\n", + "\u001b[36mFile \u001b[39m\u001b[32m~/repos/power-grid-model/src/power_grid_model/_core/error_handling.py:176\u001b[39m, in \u001b[36massert_no_error\u001b[39m\u001b[34m(batch_size, decode_error)\u001b[39m\n\u001b[32m 174\u001b[39m error = find_error(batch_size=batch_size, decode_error=decode_error)\n\u001b[32m 175\u001b[39m \u001b[38;5;28;01mif\u001b[39;00m error \u001b[38;5;129;01mis\u001b[39;00m \u001b[38;5;129;01mnot\u001b[39;00m \u001b[38;5;28;01mNone\u001b[39;00m:\n\u001b[32m--> \u001b[39m\u001b[32m176\u001b[39m \u001b[38;5;28;01mraise\u001b[39;00m error\n", "\u001b[31mIDWrongType\u001b[39m: Wrong type for object with id 4\n\nTry validate_input_data() or validate_batch_data() to validate your data.\n" ] } @@ -129,7 +133,11 @@ "cell_type": "code", "execution_count": 3, "id": "fd84be3c", - "metadata": {}, + "metadata": { + "tags": [ + "raises-exception" + ] + }, "outputs": [ { "ename": "ValidationException", @@ -139,7 +147,7 @@ "\u001b[31m---------------------------------------------------------------------------\u001b[39m", "\u001b[31mValidationException\u001b[39m Traceback (most recent call last)", "\u001b[36mCell\u001b[39m\u001b[36m \u001b[39m\u001b[32mIn[3]\u001b[39m\u001b[32m, line 4\u001b[39m\n\u001b[32m 1\u001b[39m \u001b[38;5;28;01mfrom\u001b[39;00m\u001b[38;5;250m \u001b[39m\u001b[34;01mpower_grid_model\u001b[39;00m\u001b[34;01m.\u001b[39;00m\u001b[34;01mvalidation\u001b[39;00m\u001b[38;5;250m \u001b[39m\u001b[38;5;28;01mimport\u001b[39;00m assert_valid_input_data\n\u001b[32m 3\u001b[39m \u001b[38;5;66;03m# Assert valid data\u001b[39;00m\n\u001b[32m----> \u001b[39m\u001b[32m4\u001b[39m \u001b[43massert_valid_input_data\u001b[49m\u001b[43m(\u001b[49m\u001b[43merror_data\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43msymmetric\u001b[49m\u001b[43m=\u001b[49m\u001b[38;5;28;43;01mTrue\u001b[39;49;00m\u001b[43m)\u001b[49m\n\u001b[32m 5\u001b[39m model = PowerGridModel(error_data)\n\u001b[32m 6\u001b[39m output_data = model.calculate_state_estimation(symmetric=\u001b[38;5;28;01mTrue\u001b[39;00m)\n", - "\u001b[36mFile \u001b[39m\u001b[32m~\\power-grid-model\\src\\power_grid_model\\validation\\assertions.py:55\u001b[39m, in \u001b[36massert_valid_input_data\u001b[39m\u001b[34m(input_data, calculation_type, symmetric)\u001b[39m\n\u001b[32m 51\u001b[39m validation_errors = validate_input_data(\n\u001b[32m 52\u001b[39m input_data=input_data, calculation_type=calculation_type, symmetric=symmetric\n\u001b[32m 53\u001b[39m )\n\u001b[32m 54\u001b[39m \u001b[38;5;28;01mif\u001b[39;00m validation_errors:\n\u001b[32m---> \u001b[39m\u001b[32m55\u001b[39m \u001b[38;5;28;01mraise\u001b[39;00m ValidationException(validation_errors, \u001b[33m\"\u001b[39m\u001b[33minput_data\u001b[39m\u001b[33m\"\u001b[39m)\n", + "\u001b[36mFile \u001b[39m\u001b[32m~/repos/power-grid-model/src/power_grid_model/validation/assertions.py:56\u001b[39m, in \u001b[36massert_valid_input_data\u001b[39m\u001b[34m(input_data, calculation_type, symmetric)\u001b[39m\n\u001b[32m 52\u001b[39m validation_errors = validate_input_data(\n\u001b[32m 53\u001b[39m input_data=input_data, calculation_type=calculation_type, symmetric=symmetric\n\u001b[32m 54\u001b[39m )\n\u001b[32m 55\u001b[39m \u001b[38;5;28;01mif\u001b[39;00m validation_errors:\n\u001b[32m---> \u001b[39m\u001b[32m56\u001b[39m \u001b[38;5;28;01mraise\u001b[39;00m ValidationException(validation_errors, \u001b[33m\"\u001b[39m\u001b[33minput_data\u001b[39m\u001b[33m\"\u001b[39m)\n", "\u001b[31mValidationException\u001b[39m: There are 5 validation errors in input_data:\n 1. Fields line.id and sym_power_sensor.id are not unique for 2 lines/sym_power_sensors.\n 2. Field 'to_node' does not contain a valid node id for 1 line.\n 3. Field 'power_sigma' is not greater than zero for 2 sym_power_sensors.\n 4. Field 'measured_object' does not contain a valid line/asym_line/generic_branch/transformer id for 1 sym_power_sensor. (measured_terminal_type=branch_from)\n 5. Field 'measured_object' does not contain a valid source id for 1 sym_power_sensor. (measured_terminal_type=source)" ] } @@ -267,11 +275,839 @@ "errors = validate_input_data(error_data, symmetric=True)\n", "print(errors_to_string(errors, details=True))" ] + }, + { + "cell_type": "markdown", + "id": "7344fbde", + "metadata": {}, + "source": [ + "## Batch datasets\n" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "id": "1b8ba993", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "\"There is a validation error in the data, batch #0:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #1:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #2:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #3:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #4:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #5:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #6:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #7:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #8:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #9:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #10:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #11:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #12:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #13:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #14:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #15:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #16:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #17:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #18:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #19:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #20:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #21:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #22:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #23:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #24:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #25:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #26:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #27:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #28:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #29:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #30:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #31:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #32:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #33:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #34:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #35:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #36:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #37:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #38:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #39:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #40:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #41:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #42:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #43:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #44:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #45:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #46:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #47:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #48:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #49:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #50:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #51:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #52:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #53:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #54:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #55:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #56:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #57:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #58:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #59:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #60:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #61:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #62:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #63:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #64:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #65:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #66:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #67:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #68:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #69:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #70:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #71:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #72:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #73:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #74:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #75:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #76:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #77:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #78:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #79:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #80:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #81:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #82:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #83:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #84:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #85:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #86:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #87:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #88:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #89:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #90:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #91:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #92:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #93:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #94:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #95:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #96:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #97:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #98:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #99:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\"" + ] + }, + "execution_count": 7, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "from power_grid_model.validation import validate_batch_data\n", + "\n", + "node = initialize_array(DatasetType.input, ComponentType.node, 1)\n", + "node[:] = (1, 10e3)\n", + "source = initialize_array(DatasetType.input, ComponentType.source, 1)\n", + "source[:] = (2, 1, 1, 1.0, 0.0, 1e10, 0.1, 1.0)\n", + "load = initialize_array(DatasetType.input, ComponentType.sym_load, 1)\n", + "load[:] = (3, 1, 1, 0, 1000, 1000)\n", + "\n", + "input_data = {\n", + " ComponentType.node: node,\n", + " ComponentType.source: source,\n", + " ComponentType.sym_load: load,\n", + "}\n", + "\n", + "# update data\n", + "load_update = initialize_array(DatasetType.update, ComponentType.sym_load, (100, 1))\n", + "load_update[\"id\"] = 3\n", + "load_update[\"status\"] = -5\n", + "batch_data = {ComponentType.sym_load: load_update}\n", + "\n", + "errors = validate_batch_data(input_data=input_data, update_data=batch_data, symmetric=True)\n", + "errors_to_string(errors, details=True)" + ] + }, + { + "cell_type": "markdown", + "id": "cff7ef65", + "metadata": {}, + "source": [ + "### Tip: Validating Large datasets\n", + "\n", + "The data validator is not designed for performance and is quite slow at validating large datasets.\n", + "Practically most errors can be identified in the first few scenarios.\n", + "Hence a handy tip is to validate a small section of the data using slicing. " + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "id": "7ac337fe", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "\"There is a validation error in the data, batch #0:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #1:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #2:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #3:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #4:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #5:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #6:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #7:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #8:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #9:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #10:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #11:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #12:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #13:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #14:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #15:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #16:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #17:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #18:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #19:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #20:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #21:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #22:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #23:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #24:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #25:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #26:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #27:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #28:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #29:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #30:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #31:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #32:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #33:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #34:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #35:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #36:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #37:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #38:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #39:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #40:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #41:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #42:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #43:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #44:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #45:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #46:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #47:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #48:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #49:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #50:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #51:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #52:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #53:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #54:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #55:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #56:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #57:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #58:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #59:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #60:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #61:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #62:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #63:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #64:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #65:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #66:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #67:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #68:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #69:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #70:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #71:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #72:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #73:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #74:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #75:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #76:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #77:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #78:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #79:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #80:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #81:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #82:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #83:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #84:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #85:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #86:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #87:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #88:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #89:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #90:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #91:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #92:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #93:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #94:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #95:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #96:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #97:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #98:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\\nThere is a validation error in the data, batch #99:\\n\\n\\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\\n\\t\\tcomponent: sym_load\\n\\t\\tfield: 'status'\\n\\t\\tids: [3]\\n\"" + ] + }, + "execution_count": 8, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "sliced_batch_data = {component: array[:100] for component, array in batch_data.items()}\n", + "errors = validate_batch_data(input_data=input_data, update_data=sliced_batch_data, symmetric=True)\n", + "errors_to_string(errors, details=True)" + ] + }, + { + "cell_type": "markdown", + "id": "1d82cfc5", + "metadata": {}, + "source": [ + "### Validating cartesian product of batch datasets\n", + "\n", + "[Cartesian product of batch datasets](../user_manual/calculations.md#cartesian-product-of-batch-datasets) are not supported in the `validate_batch_data`. \n", + "The user should flatten the dataset manually to use the validator.\n", + "For example it can be done in the following way: " + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "id": "04778c11", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "There is a validation error in the data, batch #0:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #1:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There are 2 validation errors in the data, batch #2:\n", + " 1. Field 'u_ref' is not greater than zero for 1 source.\n", + " 2. Field 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #3:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #4:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There are 2 validation errors in the data, batch #5:\n", + " 1. Field 'u_ref' is not greater than zero for 1 source.\n", + " 2. Field 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #6:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #7:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There are 2 validation errors in the data, batch #8:\n", + " 1. Field 'u_ref' is not greater than zero for 1 source.\n", + " 2. Field 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #9:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #10:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There are 2 validation errors in the data, batch #11:\n", + " 1. Field 'u_ref' is not greater than zero for 1 source.\n", + " 2. Field 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #12:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #13:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There are 2 validation errors in the data, batch #14:\n", + " 1. Field 'u_ref' is not greater than zero for 1 source.\n", + " 2. Field 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #15:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #16:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There are 2 validation errors in the data, batch #17:\n", + " 1. Field 'u_ref' is not greater than zero for 1 source.\n", + " 2. Field 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #18:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #19:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There are 2 validation errors in the data, batch #20:\n", + " 1. Field 'u_ref' is not greater than zero for 1 source.\n", + " 2. Field 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #21:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #22:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There are 2 validation errors in the data, batch #23:\n", + " 1. Field 'u_ref' is not greater than zero for 1 source.\n", + " 2. Field 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #24:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #25:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There are 2 validation errors in the data, batch #26:\n", + " 1. Field 'u_ref' is not greater than zero for 1 source.\n", + " 2. Field 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #27:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #28:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There are 2 validation errors in the data, batch #29:\n", + " 1. Field 'u_ref' is not greater than zero for 1 source.\n", + " 2. Field 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #30:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #31:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There are 2 validation errors in the data, batch #32:\n", + " 1. Field 'u_ref' is not greater than zero for 1 source.\n", + " 2. Field 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #33:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #34:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There are 2 validation errors in the data, batch #35:\n", + " 1. Field 'u_ref' is not greater than zero for 1 source.\n", + " 2. Field 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #36:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #37:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There are 2 validation errors in the data, batch #38:\n", + " 1. Field 'u_ref' is not greater than zero for 1 source.\n", + " 2. Field 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #39:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #40:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There are 2 validation errors in the data, batch #41:\n", + " 1. Field 'u_ref' is not greater than zero for 1 source.\n", + " 2. Field 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #42:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #43:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There are 2 validation errors in the data, batch #44:\n", + " 1. Field 'u_ref' is not greater than zero for 1 source.\n", + " 2. Field 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #45:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #46:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There are 2 validation errors in the data, batch #47:\n", + " 1. Field 'u_ref' is not greater than zero for 1 source.\n", + " 2. Field 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #48:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #49:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There are 2 validation errors in the data, batch #50:\n", + " 1. Field 'u_ref' is not greater than zero for 1 source.\n", + " 2. Field 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #51:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #52:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There are 2 validation errors in the data, batch #53:\n", + " 1. Field 'u_ref' is not greater than zero for 1 source.\n", + " 2. Field 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #54:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #55:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There are 2 validation errors in the data, batch #56:\n", + " 1. Field 'u_ref' is not greater than zero for 1 source.\n", + " 2. Field 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #57:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #58:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There are 2 validation errors in the data, batch #59:\n", + " 1. Field 'u_ref' is not greater than zero for 1 source.\n", + " 2. Field 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #60:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #61:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There are 2 validation errors in the data, batch #62:\n", + " 1. Field 'u_ref' is not greater than zero for 1 source.\n", + " 2. Field 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #63:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #64:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There are 2 validation errors in the data, batch #65:\n", + " 1. Field 'u_ref' is not greater than zero for 1 source.\n", + " 2. Field 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #66:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #67:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There are 2 validation errors in the data, batch #68:\n", + " 1. Field 'u_ref' is not greater than zero for 1 source.\n", + " 2. Field 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #69:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #70:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There are 2 validation errors in the data, batch #71:\n", + " 1. Field 'u_ref' is not greater than zero for 1 source.\n", + " 2. Field 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #72:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #73:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There are 2 validation errors in the data, batch #74:\n", + " 1. Field 'u_ref' is not greater than zero for 1 source.\n", + " 2. Field 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #75:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #76:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There are 2 validation errors in the data, batch #77:\n", + " 1. Field 'u_ref' is not greater than zero for 1 source.\n", + " 2. Field 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #78:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #79:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There are 2 validation errors in the data, batch #80:\n", + " 1. Field 'u_ref' is not greater than zero for 1 source.\n", + " 2. Field 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #81:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #82:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There are 2 validation errors in the data, batch #83:\n", + " 1. Field 'u_ref' is not greater than zero for 1 source.\n", + " 2. Field 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #84:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #85:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There are 2 validation errors in the data, batch #86:\n", + " 1. Field 'u_ref' is not greater than zero for 1 source.\n", + " 2. Field 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #87:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #88:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There are 2 validation errors in the data, batch #89:\n", + " 1. Field 'u_ref' is not greater than zero for 1 source.\n", + " 2. Field 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #90:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #91:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There are 2 validation errors in the data, batch #92:\n", + " 1. Field 'u_ref' is not greater than zero for 1 source.\n", + " 2. Field 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #93:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #94:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There are 2 validation errors in the data, batch #95:\n", + " 1. Field 'u_ref' is not greater than zero for 1 source.\n", + " 2. Field 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #96:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #97:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There are 2 validation errors in the data, batch #98:\n", + " 1. Field 'u_ref' is not greater than zero for 1 source.\n", + " 2. Field 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #99:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #100:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There are 2 validation errors in the data, batch #101:\n", + " 1. Field 'u_ref' is not greater than zero for 1 source.\n", + " 2. Field 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #102:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #103:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There are 2 validation errors in the data, batch #104:\n", + " 1. Field 'u_ref' is not greater than zero for 1 source.\n", + " 2. Field 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #105:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #106:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There are 2 validation errors in the data, batch #107:\n", + " 1. Field 'u_ref' is not greater than zero for 1 source.\n", + " 2. Field 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #108:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #109:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There are 2 validation errors in the data, batch #110:\n", + " 1. Field 'u_ref' is not greater than zero for 1 source.\n", + " 2. Field 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #111:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #112:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There are 2 validation errors in the data, batch #113:\n", + " 1. Field 'u_ref' is not greater than zero for 1 source.\n", + " 2. Field 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #114:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #115:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There are 2 validation errors in the data, batch #116:\n", + " 1. Field 'u_ref' is not greater than zero for 1 source.\n", + " 2. Field 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #117:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #118:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There are 2 validation errors in the data, batch #119:\n", + " 1. Field 'u_ref' is not greater than zero for 1 source.\n", + " 2. Field 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #120:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #121:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There are 2 validation errors in the data, batch #122:\n", + " 1. Field 'u_ref' is not greater than zero for 1 source.\n", + " 2. Field 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #123:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #124:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There are 2 validation errors in the data, batch #125:\n", + " 1. Field 'u_ref' is not greater than zero for 1 source.\n", + " 2. Field 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #126:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #127:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There are 2 validation errors in the data, batch #128:\n", + " 1. Field 'u_ref' is not greater than zero for 1 source.\n", + " 2. Field 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #129:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #130:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There are 2 validation errors in the data, batch #131:\n", + " 1. Field 'u_ref' is not greater than zero for 1 source.\n", + " 2. Field 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #132:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #133:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There are 2 validation errors in the data, batch #134:\n", + " 1. Field 'u_ref' is not greater than zero for 1 source.\n", + " 2. Field 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #135:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #136:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There are 2 validation errors in the data, batch #137:\n", + " 1. Field 'u_ref' is not greater than zero for 1 source.\n", + " 2. Field 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #138:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #139:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There are 2 validation errors in the data, batch #140:\n", + " 1. Field 'u_ref' is not greater than zero for 1 source.\n", + " 2. Field 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #141:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #142:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There are 2 validation errors in the data, batch #143:\n", + " 1. Field 'u_ref' is not greater than zero for 1 source.\n", + " 2. Field 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #144:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #145:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There are 2 validation errors in the data, batch #146:\n", + " 1. Field 'u_ref' is not greater than zero for 1 source.\n", + " 2. Field 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #147:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #148:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There are 2 validation errors in the data, batch #149:\n", + " 1. Field 'u_ref' is not greater than zero for 1 source.\n", + " 2. Field 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #150:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #151:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There are 2 validation errors in the data, batch #152:\n", + " 1. Field 'u_ref' is not greater than zero for 1 source.\n", + " 2. Field 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #153:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #154:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There are 2 validation errors in the data, batch #155:\n", + " 1. Field 'u_ref' is not greater than zero for 1 source.\n", + " 2. Field 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #156:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #157:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There are 2 validation errors in the data, batch #158:\n", + " 1. Field 'u_ref' is not greater than zero for 1 source.\n", + " 2. Field 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #159:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #160:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There are 2 validation errors in the data, batch #161:\n", + " 1. Field 'u_ref' is not greater than zero for 1 source.\n", + " 2. Field 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #162:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #163:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There are 2 validation errors in the data, batch #164:\n", + " 1. Field 'u_ref' is not greater than zero for 1 source.\n", + " 2. Field 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #165:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #166:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There are 2 validation errors in the data, batch #167:\n", + " 1. Field 'u_ref' is not greater than zero for 1 source.\n", + " 2. Field 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #168:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #169:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There are 2 validation errors in the data, batch #170:\n", + " 1. Field 'u_ref' is not greater than zero for 1 source.\n", + " 2. Field 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #171:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #172:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There are 2 validation errors in the data, batch #173:\n", + " 1. Field 'u_ref' is not greater than zero for 1 source.\n", + " 2. Field 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #174:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #175:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There are 2 validation errors in the data, batch #176:\n", + " 1. Field 'u_ref' is not greater than zero for 1 source.\n", + " 2. Field 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #177:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #178:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There are 2 validation errors in the data, batch #179:\n", + " 1. Field 'u_ref' is not greater than zero for 1 source.\n", + " 2. Field 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #180:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #181:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There are 2 validation errors in the data, batch #182:\n", + " 1. Field 'u_ref' is not greater than zero for 1 source.\n", + " 2. Field 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #183:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #184:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There are 2 validation errors in the data, batch #185:\n", + " 1. Field 'u_ref' is not greater than zero for 1 source.\n", + " 2. Field 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #186:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #187:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There are 2 validation errors in the data, batch #188:\n", + " 1. Field 'u_ref' is not greater than zero for 1 source.\n", + " 2. Field 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #189:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #190:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There are 2 validation errors in the data, batch #191:\n", + " 1. Field 'u_ref' is not greater than zero for 1 source.\n", + " 2. Field 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #192:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #193:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There are 2 validation errors in the data, batch #194:\n", + " 1. Field 'u_ref' is not greater than zero for 1 source.\n", + " 2. Field 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #195:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #196:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There are 2 validation errors in the data, batch #197:\n", + " 1. Field 'u_ref' is not greater than zero for 1 source.\n", + " 2. Field 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #198:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #199:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There are 2 validation errors in the data, batch #200:\n", + " 1. Field 'u_ref' is not greater than zero for 1 source.\n", + " 2. Field 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #201:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #202:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There are 2 validation errors in the data, batch #203:\n", + " 1. Field 'u_ref' is not greater than zero for 1 source.\n", + " 2. Field 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #204:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #205:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There are 2 validation errors in the data, batch #206:\n", + " 1. Field 'u_ref' is not greater than zero for 1 source.\n", + " 2. Field 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #207:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #208:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There are 2 validation errors in the data, batch #209:\n", + " 1. Field 'u_ref' is not greater than zero for 1 source.\n", + " 2. Field 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #210:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #211:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There are 2 validation errors in the data, batch #212:\n", + " 1. Field 'u_ref' is not greater than zero for 1 source.\n", + " 2. Field 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #213:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #214:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There are 2 validation errors in the data, batch #215:\n", + " 1. Field 'u_ref' is not greater than zero for 1 source.\n", + " 2. Field 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #216:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #217:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There are 2 validation errors in the data, batch #218:\n", + " 1. Field 'u_ref' is not greater than zero for 1 source.\n", + " 2. Field 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #219:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #220:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There are 2 validation errors in the data, batch #221:\n", + " 1. Field 'u_ref' is not greater than zero for 1 source.\n", + " 2. Field 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #222:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #223:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There are 2 validation errors in the data, batch #224:\n", + " 1. Field 'u_ref' is not greater than zero for 1 source.\n", + " 2. Field 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #225:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #226:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There are 2 validation errors in the data, batch #227:\n", + " 1. Field 'u_ref' is not greater than zero for 1 source.\n", + " 2. Field 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #228:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #229:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There are 2 validation errors in the data, batch #230:\n", + " 1. Field 'u_ref' is not greater than zero for 1 source.\n", + " 2. Field 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #231:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #232:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There are 2 validation errors in the data, batch #233:\n", + " 1. Field 'u_ref' is not greater than zero for 1 source.\n", + " 2. Field 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #234:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #235:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There are 2 validation errors in the data, batch #236:\n", + " 1. Field 'u_ref' is not greater than zero for 1 source.\n", + " 2. Field 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #237:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #238:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There are 2 validation errors in the data, batch #239:\n", + " 1. Field 'u_ref' is not greater than zero for 1 source.\n", + " 2. Field 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #240:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #241:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There are 2 validation errors in the data, batch #242:\n", + " 1. Field 'u_ref' is not greater than zero for 1 source.\n", + " 2. Field 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #243:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #244:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There are 2 validation errors in the data, batch #245:\n", + " 1. Field 'u_ref' is not greater than zero for 1 source.\n", + " 2. Field 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #246:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #247:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There are 2 validation errors in the data, batch #248:\n", + " 1. Field 'u_ref' is not greater than zero for 1 source.\n", + " 2. Field 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #249:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #250:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There are 2 validation errors in the data, batch #251:\n", + " 1. Field 'u_ref' is not greater than zero for 1 source.\n", + " 2. Field 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #252:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #253:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There are 2 validation errors in the data, batch #254:\n", + " 1. Field 'u_ref' is not greater than zero for 1 source.\n", + " 2. Field 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #255:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #256:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There are 2 validation errors in the data, batch #257:\n", + " 1. Field 'u_ref' is not greater than zero for 1 source.\n", + " 2. Field 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #258:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #259:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There are 2 validation errors in the data, batch #260:\n", + " 1. Field 'u_ref' is not greater than zero for 1 source.\n", + " 2. Field 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #261:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #262:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There are 2 validation errors in the data, batch #263:\n", + " 1. Field 'u_ref' is not greater than zero for 1 source.\n", + " 2. Field 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #264:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #265:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There are 2 validation errors in the data, batch #266:\n", + " 1. Field 'u_ref' is not greater than zero for 1 source.\n", + " 2. Field 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #267:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #268:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There are 2 validation errors in the data, batch #269:\n", + " 1. Field 'u_ref' is not greater than zero for 1 source.\n", + " 2. Field 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #270:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #271:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There are 2 validation errors in the data, batch #272:\n", + " 1. Field 'u_ref' is not greater than zero for 1 source.\n", + " 2. Field 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #273:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #274:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There are 2 validation errors in the data, batch #275:\n", + " 1. Field 'u_ref' is not greater than zero for 1 source.\n", + " 2. Field 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #276:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #277:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There are 2 validation errors in the data, batch #278:\n", + " 1. Field 'u_ref' is not greater than zero for 1 source.\n", + " 2. Field 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #279:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #280:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There are 2 validation errors in the data, batch #281:\n", + " 1. Field 'u_ref' is not greater than zero for 1 source.\n", + " 2. Field 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #282:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #283:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There are 2 validation errors in the data, batch #284:\n", + " 1. Field 'u_ref' is not greater than zero for 1 source.\n", + " 2. Field 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #285:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #286:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There are 2 validation errors in the data, batch #287:\n", + " 1. Field 'u_ref' is not greater than zero for 1 source.\n", + " 2. Field 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #288:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #289:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There are 2 validation errors in the data, batch #290:\n", + " 1. Field 'u_ref' is not greater than zero for 1 source.\n", + " 2. Field 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #291:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #292:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There are 2 validation errors in the data, batch #293:\n", + " 1. Field 'u_ref' is not greater than zero for 1 source.\n", + " 2. Field 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #294:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #295:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There are 2 validation errors in the data, batch #296:\n", + " 1. Field 'u_ref' is not greater than zero for 1 source.\n", + " 2. Field 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #297:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There is a validation error in the data, batch #298:\n", + "\tField 'status' is not a boolean (0 or 1) for 1 sym_load.\n", + "There are 2 validation errors in the data, batch #299:\n", + " 1. Field 'u_ref' is not greater than zero for 1 source.\n", + " 2. Field 'status' is not a boolean (0 or 1) for 1 sym_load.\n" + ] + } + ], + "source": [ + "import numpy as np\n", + "\n", + "source_update = initialize_array(DatasetType.update, ComponentType.source, (3, 1))\n", + "source_update[\"id\"] = [[2]]\n", + "source_update[\"u_ref\"] = [[0.9], [1.0], [-1.1]]\n", + "\n", + "source_2d, load_2d = np.meshgrid(source_update, load_update, indexing=\"xy\")\n", + "\n", + "flattened_update = {\n", + " ComponentType.source: source_2d.reshape(-1, source_update.shape[1]),\n", + " ComponentType.sym_load: load_2d.reshape(-1, load_update.shape[1]),\n", + "}\n", + "errors = validate_batch_data(input_data=input_data, update_data=flattened_update, symmetric=True)\n", + "print(errors_to_string(errors, details=False))" + ] } ], "metadata": { "kernelspec": { - "display_name": "venv", + "display_name": "power-grid-model", "language": "python", "name": "python3" }, @@ -285,7 +1121,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.13.3" + "version": "3.14.2" } }, "nbformat": 4, diff --git a/docs/user_manual/calculations.md b/docs/user_manual/calculations.md index ef91472e3c..9c5acca088 100644 --- a/docs/user_manual/calculations.md +++ b/docs/user_manual/calculations.md @@ -1092,6 +1092,28 @@ for component_idx, scenario in enumerate(line_update): independent_update_data = {'line': line_update} ``` +### Cartesian product of Batch Datasets + +Consider an example of running a contingency analysis with a timeseries data. +Or maybe probablistic data along with timeseries data. +In such simulations, it is required to perform a loadflow on a cartesian product of situations. +This is possible to do via providing the `update_data` with a list of multiple batch datasets. +ie. a list[{py:class}`BatchDataset `] +The datasets can be of row based or columnar format. +The output of such calculation would be flattened with dimension $scenarios * components$. + +#### Example: Cartesian product of datasets + +```py +# 5 scenarios of timeseries +load_update = initialize_array('update', 'sym_load', (5, 1)) +# (Fill load_update) +line_update = initialize_array('update', 'line', (3, 1)) +# (Fill line_update) + +product_update_data = [{'line': load_update}, {'sym_load': line_udpate }] +``` + ### Parallel Computing The batch calculation supports shared memory multi-threading parallel computing. diff --git a/docs/user_manual/data-validator.md b/docs/user_manual/data-validator.md index ce3c134904..63bba06460 100644 --- a/docs/user_manual/data-validator.md +++ b/docs/user_manual/data-validator.md @@ -13,14 +13,6 @@ An alternative approach would be to validate only when an exception is raised, b will raise exceptions, most of them wil just yield invalid results without warning. The main validation functions and classes can be included from `power_grid_model.validation`. -Two helper type definitions are used throughout the validation functions, `InputData` and `UpdateData`. -They are not special types or classes, but merely type hinting aliases: - -```python -InputData = dict[str, np.ndarray] -UpdateData = dict[str, np.ndarray | dict[str, np.ndarray]] -``` - ```{seealso} Check the [example](examples/Validation%20Examples.ipynb) for an example of function applications. ``` @@ -49,11 +41,13 @@ class ValidationError: ### Manual validation The validation functions below can be used to validate input/batch data manually. -The functions require `input_data: InputData`, which is power-grid-model input data, and `symmetric: bool`, stating if +The functions require `input_data` of {py:class}`SingleDataset ` type, +which is power-grid-model input data, and `symmetric: bool`, stating if the data will be used for symmetric or asymmetric calculations. `calculation_type: CalculationType` is optional and can be supplied to allow missing values for unused fields; see the [API reference](../api_reference/python-api-reference.md#enum) for more information. -To validate update/batch data `update_data: UpdateData`, power-grid-model update data, should also be supplied. +To validate update/batch data `update_data` of {py:class}`BatchDataset ` type, +power-grid-model update data, should also be supplied. - `validate_input_data(input_data, calculation_type, symmetric) -> list[ValidationError]` validates input_data. - `validate_batch_data(input_data, update_data, calculation_type, symmetric) -> dict[int, list[ValidationError]]` @@ -68,6 +62,10 @@ In such cases, the latter is leading when only running batch calculations. Running single calculations on an incomplete input data set is, of course, unsupported. ``` +Validating a cartesian product of datasets used in PGM's `update_data` via providing it with `list[BatchDataset]`is +not straightforward. +User should convert such multiple dataset into a single flat batch dataset. + ### Assertions Instead of manual validation it is possible to use the assertion functions below to assert valid data. diff --git a/docs/user_manual/performance-guide.md b/docs/user_manual/performance-guide.md index 7b0fcc7989..19af8fff4c 100644 --- a/docs/user_manual/performance-guide.md +++ b/docs/user_manual/performance-guide.md @@ -53,6 +53,15 @@ If you are running on a system where memory is the bottle-neck, using a columnar footprint. This may or may not induce a slight computational overhead during calculations. +Some simulations might require a cartesian product of scenarios of two batch datasets. +This can be done by passing them to `update_data` as a list +ie. a list[{py:class}`BatchDataset `] +(Check [Power Flow Example](../examples/Power%20Flow%20Example.ipynb)). +This gets treated as a cartesian product of the provided datasets and the combination of scenarios gets handled +internally. +Hence there is no need to allocate memory for full `N1 * N2 * ...` scenarios for a cartesian product of data sets with +scenario size `N1, N2, ...`. + ### Output data volume For most use cases, only certain output values are relevant.