# Data Source API (/docs/symetryml-rest-client/rest-documentation/data-source-api) ## Using a Data Source [#using-a-data-source] In SymetryML, a data source is an abstraction of a CSV file that resides somewhere and that can be used by: * SymetryML projects to learn new data * Models to make predictions and assessments * Encoder to update their internal encoding table SymetryML supports various types of data sources: * Secure File Transfer Protocol (SFTP) * HTTP/HTTPS URL * Amazon Simple Storage Service (S3) * Microsoft Azure Blob Storage * Google Cloud Storage * Oracle OCI Object Storage * Amazon RedShift * Snowflake * Spark Processing: Amazon S3, Google cloud storage, Oracle OCI object storage and Microsoft Azure blob storage data sources can be processed in parallel leveraging a Spark Cluster. * SymetryML data source plugins * JDBC * Local Data Source, that is allows to browse the local file system of the jetty web server with same privileges as the user running the Jetty web server. **Off by default since 7.0**: the server refuses the `localfile` type with `400` unless the administrator sets `rtlm.option.sml.local_ds.enable=1` (see [SymetryML REST Configuration](/docs/guides/installation-guide#symetryml-rest-configuration)). **Streaming Data Sources:** For real-time streaming data sources such as Kafka and NATS, please refer to the [Stream Data Source API](/docs/symetryml-rest-client/rest-documentation/stream-data-source-api) documentation. To use a data source, create a JSON data structure described in [DSInfo](/docs/symetryml-rest-client/rest-documentation/appendix-a-json-data-structure-schema#dsinfo-json) that contains the fields in table [below](/docs/symetryml-rest-client/rest-documentation/data-source-api#required-dsinfo-fields). ### Required DSInfo Fields [#required-dsinfo-fields] | Field | Description | | -------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | **type** |

Type of Data Source

- Secure FTP (SFTP) data source = sftp

- HTTP/HTTPS data source = http

- Amazon S3 = s3

- Oracle OCI Object Storage with S3 Compatibility = s3oci

- Google Clound Storage = gcs

- Amazon Redshift = redshift

- Spark Data Source = please see the Spark Data Source type section for the matrix of all possible data source names involving Spark Processing.

- Data Source Plug ins. - jdbc

- Snowflake = snow

- Local file = localfile (disabled unless the server enables it)

- Microsoft Azure Blob Storage = abs

- Streaming Data Sources (Kafka, NATS) = kafka or nats - see Stream Data Source API

| | **name** | Name of the data source. | | **info** | Hash Map Containing Additional Information Based on Data Source. Please consult the [next section](/docs/symetryml-rest-client/rest-documentation/data-source-api#additional-information-stored-in-data-source) for details about this field. | ### Field Validation [#field-validation] Data source fields are validated on [create](#data-source-create), [update](#data-source-update) and [sample preview](#fetching-sample-data-1). A missing or empty required field is rejected with `400 BAD REQUEST` and a message naming what is missing: | Message | Cause | | -------------------------------------------------------------- | ------------------------------------------------------------------------------------- | | `DS: type is missing or empty` | No `type` was supplied. | | `DS type '': name is missing or empty` | No `name` was supplied. Not checked by sample preview, which does not require a name. | | `DS type '': path is missing or empty` | No `path` was supplied. Required for every type except `kafka`, `nats` and `snow`. | | `DS type '': required field '' is missing or empty` | A field required by that particular data source type is missing. | In 6.3 these conditions surfaced as generic `500` errors. The **Required** column in the tables below reflects exactly what this validation enforces. A field marked Optional is not rejected at create time, which is not the same as saying the data source will function without it. **Note:** this applies to built-in types only. [Data source plugins](#about-data-source-plugins-dsplugins) declare their own fields through `uiInputFields` in the plugin configuration and are not checked here. ### Additional Information Stored in Data Source [#additional-information-stored-in-data-source] The `info` field of a data source contains specific information based on the type of data source. The following tables describes what they are for each different type of data source #### HTTP/HTTPS Data Source [#httphttps-data-source] \| field | Required Description | \| ------ | -------- ------------------------ | \| `path` | Required http\:// or https\:// URL. | **Note:** `http` and `https` data sources follow redirects. A non-2xx response fails with `HTTP GET failed with status N`. When the server sends no `Content-Length` header the reported size is `-1`, so progress cannot be reported for chunked responses. Since 7.0, HTTPS certificates are verified against the server's trust store with hostname verification; a self-signed or private-CA certificate needs `rtlm.option.sml.tls.truststore` (see [SymetryML REST Configuration](/docs/guides/installation-guide#symetryml-rest-configuration)). Connection and read timeouts come from the `rtlm.option.vfs.http.connection.timeout` and `rtlm.option.vfs.http.so.timeout` server settings — the `vfs.` prefix is historical and the names are unchanged. #### Secure FTP (SFTP) [#secure-ftp-sftp] \| field | Required Description | \| ------------ | -------- ------------------------------------------------- | \| `path` | Required path to the file on the server. | \| `sftpuser` | Required user name used to connect to the SFTP server. | \| `sftppasswd` | Required user password used to connect to the SFTP server. | \| `sftphost` | Required host to which you want to connect. | **Note:** since 7.0 the SFTP host key must be listed in the server's `known_hosts` file (the Jetty user's `~/.ssh/known_hosts`, or the file named by `rtlm.option.sml.sftp.known_hosts`); otherwise the connection fails. Add the host with `ssh-keyscan`. #### Amazon S3 [#amazon-s3] \| field | Required Description | \| ------------- | -------- ---------------------------------------------------------- | \| `path` | Required path to file on the server, excluding the Amazon S3 bucket | \| `s3accessKey` | Required Amazon S3 access key to use to connect to S3. | \| `s3secretKey` | Required Amazon S3 secret key to use to connect to S3. | \| `s3bucket` | Required Amazon S3 bucket to use. | #### Oracle OCI Object Storage with S3 Compatibility [#oracle-oci-object-storage-with-s3-compatibility] Please consult [https://docs.oracle.com/en-us/iaas/Content/Object/Tasks/s3compatibleapi.htm](https://docs.oracle.com/en-us/iaas/Content/Object/Tasks/s3compatibleapi.htm) for details on how to configure your Oracle OCI account so that the Amazon S3 Compatibility API can be used. \| field | Required Description | \| -------------- | -------- ------------------------------------------------------------------------------------------- | \| `path` | Required path to file on the server, excluding the Oracle OCI Object Storage bucket | \| `s3accessKey` | Required Oracle Oracle OCI Object Storage access key to use to connect to Oracle OCI Object Storage. | \| `s3secretKey` | Required Oracle Oracle OCI Object Storage secret key to use to connect to Oracle OCI Object Storage. | \| `s3bucket` | Optional Oracle Oracle OCI Object Storage bucket to use. | \| `ocinamespace` | Required Oracle Oracle OCI Object Storage namespace | \| `ociregion` | Required Oracle Oracle OCI Object Storage region | #### Google Cloud Storage [#google-cloud-storage] \| field | Required Description | \| -------------- | -------- ------------------------------------------------------------------------------------------------------------------------------------------------------------- | \| `path` | Required Path to file | \| `gcsaccessKey` | Required GCS [HMAC](https://cloud.google.com/storage/docs/authentication/hmackeys) Access Key | \| `gcssecretKey` | Required GCS [HMAC](https://cloud.google.com/storage/docs/authentication/hmackeys) Secret Key | \| `gcsbucket` | Required GCS Bucket | \| `gcsproject` | Optional GCS Project | \| `gcsmarker` | Optional Optional marker parameter indicating where in the GCS bucket to begin listing. The list will only include keys that occur lexicographically after the marker. | \| `gcsdelimiter` | Optional GCS File/Folder delimiter. */* is used by default | #### Microsoft Azure Blob Storage [#microsoft-azure-blob-storage] \| field | Required Description | \| --------------------------------------------- | -------- ----------------------------------------------------------------------------------------------------------------------------------------------------------- | \| `path` | Required Path to file | \| `azure.credentials.connection.string` | Optional Connection string that specifies credentials to authorize access to Azure Blob Storage. Use one of this, account key or SAS token. | \| `azure.account.name` | Optional Name of the Azure account to use | \| `azure.credentials.sharedkey.account.key` | Optional Account key that specifies credentials to authorize access to Azure Blob Storage. Use one of this, connection string or SAS token. | \| `azure.credentials.sharedkey.sas.token` | Optional SAS token (account or service) that specifies credentials to authorize access to Azure Blob Storage. Use one of this, connection string or account key. | \| `azure.blob.container.name` | Required Name of the Azure Blob Storage container that contains the blob | \| `azure.blob.inputstream.chunk.size.max.bytes` | Optional Maximum size in bytes of each chunk of data when reading the blob contents chunk by chunk. Default: 4194304 | \| `azure.blob.path.delimiter` | Optional String that separates elements of the path to the blob file. Default: / | \| `azure.blob.list.marker` | Optional

Marker that specifies the beginning of the next page of a list of
Azure Blob Storage items to fetch from Azure. This property is used internally.

| **Note:** only `azure.blob.container.name` is required. Credentials are optional — anonymous public read works when the container's access level is set to Container. #### Amazon Redshift [#amazon-redshift] \| field | Required Description | \| ---------- | -------- ------------------------- | \| `path` | Required name of the table to use. | \| `rsuser` | Required Redshift database user | \| `rspasswd` | Required Redshift user password | \| `rsurl` | Required Redshift connection url | **Note:** since 7.0 `rsurl` must have the form `jdbc:redshift://host[:port][/database]`, optionally followed by URL options whose names are in the server's `rtlm.option.sml.redshift.url_options` list (default: `tcpKeepAlive`). Any other URL or option is refused with `400 BAD REQUEST` when the data source is created. #### Spark Map Reduce [#spark-map-reduce] \| field | Required Description | \| ----------------------------------------------------------------------------------------- | -------- ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | \| `sparkmaster` | Required address of the spark’s cluster master | \| `spark.job.process.jvm.heap.size.min` | Optional Mininum JVM size used for the spark Driver process launched by the Jetty Rest Server. Default : 1024m | \| `spark.job.process.jvm.heap.size.max` | Optional Maximum JVM size used for the spark Driver process launched by the Jetty Rest Server. Default: 2048m | \| Any Spark parameters be used also like: `spark.executor.memory` or `spark.executor.cores` | Optional To pass such parameters, prefix them with ‘sml.sparkenv.’ as in the following examples: - sml.sparkenv.spark.executor.cores - sml.sparkenv.spark.cores.max | \| `spark.automl.sample.random.seed` | Optional If AutoML is used, one can set the randomizer seed that will be used to select a random sample of tuple from the data source to be used to bootstrap the AutoML environement. | **Spark Map Reduce Data Source Type** **SymetryML 6.3.0 and later:** Due to changes in the underlying JDK and Jetty server, only Spark 4.1.0 is supported. | Data Source | Spark 4.1.0 | | ------------------------ | -------------------------------- | | **Oracle OCIs3** | sparkocis3\_map\_reduce\_4\_1\_0 | | **Amazon S3** | sparks3\_map\_reduce\_4\_1\_0 | | **Google Cloud Storage** | sparkgcs\_map\_reduce\_4\_1\_0 | | **Microsoft Azure Blob** | sparkabs\_map\_reduce\_4\_1\_0 | #### JDBC [#jdbc] | field | Required | Description | | ----------------- | -------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | `path` | Required | Location of the data. For MySQL this is `database/table`; for the PostgreSQL family see [PostgreSQL and Amazon Redshift over JDBC](#postgresql-and-amazon-redshift-over-jdbc). | | `jdbcdriverclass` | Required | JDBC driver class, for example `org.postgresql.Driver` or `com.mysql.cj.jdbc.Driver`. Together with `jdbcprefix` it must be one of the pairs the server allows (see below). | | `jdbchost` | Required | Database host: a host name or an IP address, nothing else. | | `jdbcport` | Required | Database port: a number from 1 to 65535. | | `jdbcuser` | Required | User name used to connect. | | `jdbcpassword` | Required | Password used to connect. | | `jdbcprefix` | Optional | JDBC URL scheme. **Default `jdbc:mysql`.** Use `jdbc:postgresql` or `jdbc:redshift` for the PostgreSQL family. | **Allowed databases (7.0).** The server accepts only the `jdbcprefix` and `jdbcdriverclass` pairs listed in its `rtlm.option.sml.jdbc.drivers` setting; any other pair, a host or port with extra characters, or an empty list (JDBC disabled) is refused with `400 BAD REQUEST` when the data source is created. The default list is: | `jdbcprefix` | `jdbcdriverclass` | | ----------------- | -------------------------- | | `jdbc:mysql` | `com.mysql.cj.jdbc.Driver` | | `jdbc:mysql` | `com.mysql.jdbc.Driver` | | `jdbc:postgresql` | `org.postgresql.Driver` | Other databases (for example `jdbc:redshift` over the generic `jdbc` type, or Hive) need the administrator to add their pair to the list, with the driver jar on the server's classpath. The server builds the connection URL itself, sends the user and password as connection properties rather than in the URL, and for MySQL always turns off `allowLoadLocalInfile`, `allowUrlInLocalInfile` and `autoDeserialize`. **Database, schema, table and warehouse names** (JDBC, Amazon Redshift and Snowflake) must consist of letters, digits, `_` or `$`, up to 255 characters per dot-separated part; any other name is refused with `400 BAD REQUEST`. The one exception is the table name of a PostgreSQL-family `path`, which is quoted instead, so it may contain other characters (but no control characters). #### PostgreSQL and Amazon Redshift over JDBC [#postgresql-and-amazon-redshift-over-jdbc] PostgreSQL and Amazon Redshift are reached through the `jdbc` data source type rather than a type of their own. Set `jdbcprefix` to `jdbc:postgresql` or `jdbc:redshift` to select this behaviour. `jdbc:redshift` is not in the server's default list of allowed databases; the administrator must add the pair to `rtlm.option.sml.jdbc.drivers` first (see [JDBC](#jdbc)). | Setting | Value | | ----------------- | ----------------------------------------------------------------------------------------------------------------- | | `jdbcprefix` | `jdbc:postgresql` or `jdbc:redshift` | | `jdbcdriverclass` | `org.postgresql.Driver` | | `path` | `database/table`. The database becomes part of the connection URL; the table name is quoted to preserve its case. | **Browsing with no database selected** lists databases as directories. A PostgreSQL connection URL with no database named defaults to the connecting user's own name, which usually does not exist, so the browser connects through a maintenance database instead — trying `postgres` first, then `template1`. **The connecting user needs CONNECT permission on one of those two, or browsing fails.** **Browsing with a database selected** lists tables as files, and **only the `public` schema is listed**. Views are included, matching the behaviour of the MySQL listing. #### Snowflake [#snowflake] SymetryML supports connecting to Snowflake data warehouses using Snowflake's Snowpark Container Services (CP3O) for distributed processing. | field | Required | Description | | --------------------------------------------- | ------------ | ---------------------------------------------------------------------------------------------------------------------------------- | | `snowflake.account` | **Required** | Snowflake account identifier (e.g., `XYZZZZZ-ZBC00000`) | | `snowflake.user` | **Required** | User to connect to the Snowflake account with (e.g., `zzzuser`) | | `snowflake.auth.private.key.raw` | **Required** | String content of the private key to authorize the connection to the account | | `snowflake.account.role` | CP3O | Role to connect to the Snowflake account with (e.g., `ACCOUNTADMIN`). Required in CP3O mode. | | `snowflake.cp3o.compute.pool.instance.family` | Optional | Compute pool instance family to use (e.g., `CPU_X64_S`). **Setting this selects CP3O mode**; leaving it unset selects Reader mode. | | `snowflake.cp3o.worker.total` | CP3O | Number of workers that share the update task (e.g., `3`). Required in CP3O mode. | | `snowflake.cp3o.data.table.order.by.clause` | CP3O | SQL clause used to order the rows of the table so that the workers can partition it. Required in CP3O mode. | | `snowflake.cp3o.compute.pool.max.nodes` | Optional | Maximum number of nodes in the compute pool (e.g., `3`). Derived from `snowflake.cp3o.worker.total` when absent. | | `snowflake.cp3o.compute.pool.min.nodes` | Optional | Minimum number of nodes in the compute pool (e.g., `1`). Derived from `snowflake.cp3o.worker.total` when absent. | | `snowflake.data.warehouse` | Reader | Warehouse used to read the data. Required in Reader mode. | | `snowflake.data.database` | Conditional | Name of the database containing the data. Required unless `path` is given. | | `snowflake.data.schema` | Conditional | Name of the schema containing the data. Required unless `path` is given. | | `snowflake.data.table` | Conditional | Name of the table containing the data. Required unless `path` is given. | | `snowflake.cp3o.ecr.auth.token` | Optional | Authentication token for the container registry used by CP3O. | | `snowflake.job.warehouse` | Optional | Snowflake warehouse to use for queries. **Default: `WH_NB`** | | `snowflake.job.warehouse.size` | Optional | Size of the Snowflake warehouse to use for queries. **Default: `XSMALL`** | | `snowflake.job.warehouse.auto.suspend` | Optional | Inactivity timeout in seconds of the Snowflake warehouse to use for queries. | | `snowflake.list.roles` | Optional | Boolean flag to show a list of Snowflake roles while browsing. **Default: `false`** | | `snowflake.cp3o.data.frame.max.size` | Optional | Maximum size of a Data Frame to use while updating SML projects in processing the job. | **Modes.** Setting `snowflake.cp3o.compute.pool.instance.family` puts the data source in **CP3O** mode, which distributes the work across a compute pool; leaving it unset selects **Reader** mode, which reads through a warehouse. The Required column above marks which fields each mode adds. **Locating the data.** Supply either `path` in the exact form `DB/SCHEMA/TABLE`, or all three of `snowflake.data.database`, `snowflake.data.schema` and `snowflake.data.table`. A `path` with anything other than three non-empty segments — including a trailing `/` — is rejected with `DS type 'snow': path must be in the format [DB/SCHEMA/TABLE]`. Supplying neither is rejected with `DS type 'snow': either 'path' or all of 'snowflake.data.database', 'snowflake.data.schema', 'snowflake.data.table' must be provided`. **Note:** `snowflake.job.role` is **not** a data source field. It is a server setting — see [SymetryML REST Configuration](/docs/guides/installation-guide#symetryml-rest-configuration). #### Amazon EMR and Google Dataproc [#amazon-emr-and-google-dataproc] **Not available in 7.0.** Learning or forgetting data from an `emr` or `dataproc` data source is refused with `400 BAD REQUEST` (`EMR and Dataproc data sources are not available in this release.`). Use a [Spark data source](#spark-map-reduce) to process large files in parallel. ### Additional CSV Options [#additional-csv-options] You can specify additional parameters that describe the ‘type of csv files’. You can also add the following parameters to a data source to change how SymetryML parses your data: **Note:** these options apply to **file-based** data sources only — the CSV, delimited and archive files read from SFTP, HTTP/HTTPS, S3, GCS, Azure Blob, OCI and local disk. They have no effect on database sources such as JDBC, Redshift and Snowflake, which read typed values directly from the result set and never pass them through CSV parsing. | Key | Description | | ------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------- | | **csv\_entry\_separator** | Specifies which character to use as the delimiter for each record for a given tuple. | | **csv\_quote\_character** | Specifies the quote character. | | **csv\_strict\_quotes** | Setting this option to true discards characters outside the quotes. If there are no quotes between delimiters, an empty string is generated. | | **csv\_header\_missing** | Specifies that this data source does not have any header. SymetryML can then generate a header automatically. | ### Missing Values [#missing-values] When a value in a continuous column cannot be read as a number, SymetryML checks it against a list of tokens that mean "missing". A match is stored as `NaN` rather than failing the row. Matching is **case-insensitive**. | Token | Notes | | ----------------------- | --------------------------------------- | | `null` | Also matches `NULL`, `Null`, and so on. | | `nil` | | | `` | | | `nan` | | | `inf`, `-inf` | | | `infinity`, `-infinity` | | | `-` | A bare dash. | Additional tokens can be declared server-side with the `rtlm.option.sml.nan.values` setting, which takes a comma-separated list and defaults to `-`. Tokens added this way are also matched case-insensitively. A number wrapped in parentheses is read as negative, so `(1.5)` is `-1.5`. ### Additional Information on Spark S3 Data Source [#additional-information-on-spark-s3-data-source] SymetryML can leverage a spark cluster to speed up processing of large amounts of data significantly. Currently, your data must reside on Amazon S3. Depending on the size of your data, it may take more or less time for the job to start, as the Spark Cluster must compute the partitions of your data before starting the job. Consequently, if your data is very large, this may take a few minutes. Please be sure to consult the [Spark Installation Guide](/docs/guides/installation-guide/spark-installation-guide) for additional Spark information. **Best practices for Spark S3 Data Source:** * Performance may vary depending on Amazon resource utilization when you run your job. * Be sure all executor nodes in your cluster reside in the same Amazon EC2 placement group. ### About Data Source Plugins (DSPlugins) [#about-data-source-plugins-dsplugins] The SymetryML data source API allows you to create a new data source by the form of Java library (jar) that can be added the server. Instead of transforming data into CSV files, for example, you can write a DS plugin that reads the data natively. ## Data Source Encryption [#data-source-encryption] Data sources might contain sensitive information that should never be passed in the clear. To avoid having to use HTTPS for these services, the SymetryML REST API forces you to pass such information in encrypted form. This can be done easily, as each SymetryML secret key is also a 128-bit Advanced Encryption Standard (AES) secret key. 1. Create your [DSInfo](/docs/symetryml-rest-client/rest-documentation/appendix-a-json-data-structure-schema#dsinfo-json) data structure and enter the appropriate information so that the SymetryML server can access it. 2. Extract the JSON string from that data structure. 3. Encrypt the JSON string with your SymetryML secret key in the `v2:` AES-GCM format described in [Encrypted Request and Response Bodies](/docs/symetryml-rest-client/rest-documentation/symetryml-rest-api-security#encrypted-request-and-response-bodies). The fixed initialization vector and AES/CBC format used before 7.0 are no longer accepted. 4. Send the encrypted string as part of the body to any REST service that requests a [DSInfo](/docs/symetryml-rest-client/rest-documentation/appendix-a-json-data-structure-schema#dsinfo-json) as the body. 5. The server decrypts the string using the client secret key and reconstructs the [DSInfo](/docs/symetryml-rest-client/rest-documentation/appendix-a-json-data-structure-schema#dsinfo-json). ## Data Source Create [#data-source-create] This API function creates a new data source. ### URL [#url] ``` POST /symetry/rest/{cid}/dss/create [Body=DSInfo (encrypted)] ``` ### HTTP Responses [#http-responses] | HTTP Status Code | HTTP Status Message | Description | | ---------------- | ------------------- | -------------------------------------------------------------------------------------------- | | **202** | CREATED | Success. | | **409** | CONFLICT | A data source with the specified name already exists. | | **400** | BAD REQUEST | A required data source field is missing or empty. See [Field Validation](#field-validation). | ### HTTP Response Entity [#http-response-entity] None. ### Sample Request/Response [#sample-requestresponse] Please note that in the following example the BODY is encrypted. Refer to section [Data Source Encryption](/docs/symetryml-rest-client/rest-documentation/data-source-api#data-source-encryption) for encryption details. ``` Request: POST url="http://charm:8080/symetry/rest/c1/dss/create" Body: emUmJ7LXKaICPww/dKRoMR/Go6+B3ATTn+GwgE1/vcK9pN/mLoqGyKiMtGiTYMct4Gth1ElniKLXtmXfHfs7Rfn+QhJHZ+s00w2PBdbvYZIoF3My04H5XCboY21Fh4SkBhsxo+DhYuardN7R+uGphN/DSbiHRLIXe51HijSpfuq0fJuQYW0ccN4FM/B74LJccuDnbb+IouS9u/9rybKu/wsVbDVRTK/zZpWXyn4qtds= Response: {"statusCode":"CREATED","statusString":"DS created with name [ds1] for customer [c1]","values":{}} ``` ## Data Source Update [#data-source-update] This API function update an existing data source. ### URL [#url-1] ``` PUT /symetry/rest/{cid}/dss/{dsname} [Body=DSInfo (encrypted)] ``` ### HTTP Responses [#http-responses-1] | HTTP Status Code | HTTP Status Message | Description | | ---------------- | ------------------- | -------------------------------------------------------------------------------------------- | | **200** | OK | Success. | | **404** | NOT FOUND | If data source with the specified name does not exist. | | **400** | BAD REQUEST | A required data source field is missing or empty. See [Field Validation](#field-validation). | ### HTTP Response Entity [#http-response-entity-1] None. ### Sample Request/Response [#sample-requestresponse-1] Please note that in the following example the BODY is encrypted. Refer to section [Data Source Encryption](/docs/symetryml-rest-client/rest-documentation/data-source-api#data-source-encryption) for encryption details. ``` Request: PUT url="http://charm:8080/symetry/rest/c1/dss/aDataSourceName" Body: emUmJ7LXKaICPww/dKRoMR/Go6+B3ATTn+GwgE1/vcK9pN/mLoqGyKiMtGiTYMct4Gth1ElniKLXtmXfHfs7Rfn+QhJHZ+s00w2PBdbvYZIoF3My04H5XCboY21Fh4SkBhsxo+DhYuardN7R+uGphN/DSbiHRLIXe51HijSpfuq0fJuQYW0ccN4FM/B74LJccuDnbb+IouS9u/9rybKu/wsVbDVRTK/zZpWXyn4qtds= Response: {"statusCode":"OK","statusString":"DS updated with name [ds1] for customer [c1]","values":{}} ``` ## List Customer Data Sources [#list-customer-data-sources] This API function returns all the data sources that belong to a user. ### URL [#url-2] ``` GET /symetry/rest/{cid}/dss/ ``` ### HTTP Responses [#http-responses-2] | HTTP Status Code | HTTP Status Message | Description | | ---------------- | ------------------- | ----------- | | **200** | OK | Success. | ### HTTP Response Entity [#http-response-entity-2] | HTTP Response Entity | Example | | ----------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | [**StringList**](/docs/symetryml-rest-client/rest-documentation/appendix-a-json-data-structure-schema#stringlist) | `{"statusCode":"OK","statusString":"OK","values":{"stringList":{"values":["Iris_SymetryML.csv-predict.csv:s3","h1:http","BigData11g_Test.csv:http","Iris_SymetryML.csv:s3","Smaato_Bids_20130812_CTR.csv:s3"]}}}` | ### Sample Request/Response [#sample-requestresponse-2] ``` Request: GET url="http://charm:8080/symetry/rest/c1/dss" Response:= {"statusCode":"OK","statusString":"OK","values":{"stringList":{"values":["Iris_rtlm-out-neil.csv:s3","Iris_rtlm.csv:s3","BigData11g_num.csv:s3","Smaato_Bids_20130812_CTR.csv:s3"]}}} ``` ## Delete Data Source [#delete-data-source] This API function deletes a data source. ### URL [#url-3] ``` DELETE /symetry/rest/{cid}/dss/{dsname} ``` ### HTTP Responses [#http-responses-3] | HTTP Status Code | HTTP Status Message | Description | | ---------------- | ------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------- | | **200** | OK | Success. | | **409** | CONFLICT | Data source cannot be deleted. A SymetryML project might be using the data source. The response contains an error string with further details. | ### HTTP Response Entity [#http-response-entity-3] None ## Data Source Information [#data-source-information] This API function returns information about a data source. Because this information can be sensitive, the server encrypts it to ensure that the information will be returned safely, even when using HTTP. Use your SymetryML secret key to decrypt the information. For information about how to decrypt the response, see the section [Data Source Encryption](/docs/symetryml-rest-client/rest-documentation/data-source-api#data-source-encryption). ### URL [#url-4] ``` GET /symetry/rest/{cid}/dss/{dsname} ``` ### HTTP Responses [#http-responses-4] | HTTP Status Code | HTTP Status Message | Description | | ---------------- | ------------------- | ------------------------------------------------------------------------------ | | **200** | OK | Success. | | **404** | NOT FOUND | The data source does not exist. `Customer [c1]: cannot find data source [ds1]` | ### HTTP Response Entity [#http-response-entity-4] | HTTP Response Entity | Example | | ---------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------- | | **DSInfo (encrypted)** | [DSInfo](/docs/symetryml-rest-client/rest-documentation/appendix-a-json-data-structure-schema#dsinfo-json) encrypted using the customer secret key. | ### Sample Request/Response [#sample-requestresponse-3] ``` Request: GET url="http://charm:8080/symetry/rest/c1/dss/BigData11g_num.csv" Response: {"statusCode":"OK","statusString":"OK","values":{"dsinfo":"emUmJ7LXKaICPww/dKRoMRNJEMepL37Eq9CgZfhZPWj93mo3A+C8ucfIOGaPwwn2dip/JEuLFjUT/fjHjy18XKFnzFz5Ujp0WmS0uA4ssvAJwNPL6BvnsY6+a/lKa+c/q9/5tz5lr13N13I7OGAhuYhXYV+xb8oFZqsn+bH5spBXRb5u+oyEMXNKLCaNt3pzc/xCyW47KCwIi9V5iSA+fcJAWfetm9ZsIHNbI6utkxKrqrU5OfLmgriGAP++yQtutlGR7r/bKV1bRc8UDsgsXQg1HgoxHcKXCgsAFDFzqsJmZ/5/uQDc0ytc5Fk85GUx"}} ``` ## Data Source Browsing [#data-source-browsing] This API function lists content about a remote data source directory. ### URL [#url-5] ``` POST /symetry/rest/{cid}/projects/dsbrowse/ [Body=DSListingRequest (encrypted)] ``` ### HTTP Responses [#http-responses-5] | HTTP Status Code | HTTP Status Message | Description | | ---------------- | ------------------- | ----------- | | **200** | OK | Success. | ### HTTP Response Entity [#http-response-entity-5] | HTTP Response Entity | Description | | ------------------------------------------------------------------------------------------------------------------------------------ | --------------------------------------------------------------------- | | [**DSListingResponse**](/docs/symetryml-rest-client/rest-documentation/appendix-a-json-data-structure-schema#dslistingresponse-json) | Contains listing information about the requested directory or folder. | ### Sample Request/Response [#sample-requestresponse-4] ``` Request: POST url="http://charm:8080/symetry/rest/c1/projects/dsbrowse" Body: drhSjndw6G15pgevCsDqaSfjX9x3hMo+dNqd/MV943Dsd2rl2guhvq2qUhjEORcfKAEjHaoRZMKmSbQB6bcca2YT6HmUyRxuOG0wiKgGy0MOEq7+iIncbX4orpGr4rhro1Frw909Uy8qcWskaInQHJT4EGRPcwxvwInFlea39hsMkycFK4pKlTpanOUYgcv7 Response: {"statusCode":"OK","statusString":"OK","values":{"dsdirectoryListing":{"ok":true,"dirs":["data2/","folder-a/","folder-datasets/","folder-demo/","folder-dev/","folder-dev-pub-http/","folder-dev-pub-https/","folder-docs/","folder-perf-all-reports/","folder-source/"]}}} ``` ## Fetching Sample Data 1 [#fetching-sample-data-1] This API function fetches a data source sample by specifying the data source information as part of the request body. The response returns up to 128 lines. This REST call needs a request body that contains an encrypted DSInfo data structure (see the section [Data Source Encryption](/docs/symetryml-rest-client/rest-documentation/data-source-api#data-source-encryption)). ### URL [#url-6] ``` POST /symetry/rest/{cid}/dss/sample/preview [body=DSInfo encrypted] ``` ### HTTP Responses [#http-responses-6] | HTTP Status Code | HTTP Status Message | Description | | ---------------- | ------------------- | -------------------------------------------------------------------------------------------- | | **200** | OK | Success. | | **400** | BAD REQUEST | A required data source field is missing or empty. See [Field Validation](#field-validation). | ### HTTP Response Entity [#http-response-entity-6] | HTTP Response Entity | Description | | -------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------- | | [**DataFrame**](/docs/symetryml-rest-client/rest-documentation/appendix-a-json-data-structure-schema#dataframe-json) | Contains a sample of the data source up to 128 lines. | ### Sample Request/Response [#sample-requestresponse-5] ``` Request: POST url="http://charm:8080/symetry/rest/c1/dss/sample/preview Body: WTqUHBoXHbl+cMMjdc0zgjBP8e44G1os15V+I4GZgDOr1dX9uOfvY5uK9ZgC9yral9XC1ohD1W+UvkPlKR4dQT00EgCdS2UPgZz2NwwooHOM+KY1Ysf5qZlkFKiOkxwoWH/mr3mvvgdTUpZS8zrDJk3gwsavFT5fe0J2lTR33F1OH7FwxP4qs5nzRbVz546l Response: {"statusCode":"OK","statusString":"OK","values":{"dataframe":{"attributeNames":["adexchange","imp_width","imp_height","imp_btype","preference","pub","domain","site_base_url","category","device_ip","device_country","device_dma","device_state","device_city","zip","device_carrier","language","device_os","device_make","device_model","device_osv","lat","lon","restriction_bcat","restriction_badv","position","gender","user_keyword","user_yob","user_age","view_count","campaign","creative","creative_type","winner","bid_price","win_price","clicked","rtlm_ctr_score","rtdm","req_hour","req_day","category_count","bcat_count","badv_count"],"data":[["Smaato","320","50","","APP","New IT Solutions Ltd","","","Technology \u0026 Computing","157.55.32.83","US","","WASHINGTON","REDMOND","98052","","","Unknown","","","","47.67398834","-122.1215134","IAB7-28|IAB19-30|IAB22-1|IAB19-3|IAB17-18|IAB26|IAB25|IAB24|IAB9-9|IAB7- (...) ,"attributeTypes":["S","C","C","L","S","L","S","S","S","S","S","S","S","S","S","S","S","S","S","S","S","S","S","L","S","S","S","S","S","S","B","S","S","S","B","C","C","B","S","B","C","C","B","C","B"]}}} ``` ## Fetching Sample Data 2 [#fetching-sample-data-2] ### URL [#url-7] ``` GET /symetry/rest/{cid}/dss/{dsname}/sample ``` ### Query Parameters [#query-parameters] | Parameter | Required / Optional | Description | | -------------- | ------------------- | ---------------------------------------------------------------------------- | | **numrecords** | Optional | Number of records to return. Default is `128`. In 6.3 this was fixed at 128. | ### HTTP Responses [#http-responses-7] | HTTP Status Code | HTTP Status Message | Description | | ---------------- | ------------------- | ----------- | | **200** | OK | Success. | ### HTTP Response Entity [#http-response-entity-7] | HTTP Response Entity | Description | | -------------------- | --------------------------------------------------------------------------------------------------------------------- | | **DataFrame** | See [DataFrame](/docs/symetryml-rest-client/rest-documentation/appendix-a-json-data-structure-schema#dataframe-json). | ### Sample Request/Response [#sample-requestresponse-6] ``` Request: GET url="http://charm:8080/symetry/rest/c1/dss/Smaato_Bids_20130812_CTR.csv/sample" Response: {"statusCode":"OK","statusString":"OK","values":{"dataframe":{"attributeNames":["adexchange","imp_width","imp_height","imp_btype","preference","pub","domain","site_base_url","category","device_ip","device_country","device_dma","device_state","device_city","zip","device_carrier","language","device_os","device_make","device_model","device_osv","lat","lon","restriction_bcat","restriction_badv","position","gender","user_keyword","user_yob","user_age","view_count","campaign","creative","creative_type","winner","bid_price","win_price","clicked","rtlm_ctr_score","rtdm","req_hour","req_day","category_count","bcat_count","badv_count"],"data":[["Smaato","320","50","","APP","New IT Solutions Ltd","","","Technology \u0026 Computing","157.55.32.83","US","","WASHINGTON","REDMOND","98052","","","Unknown","","","","47.67398834","-122.1215134","IAB7-28|IAB19-30|IAB22-1|IAB19-3|IAB17-18|IAB26|IAB25|IAB24|IAB9-9|IAB7- (...) ,"attributeTypes":["S","C","C","L","S","L","S","S","S","S","S","S","S","S","S","S","S","S","S","S","S","S","S","L","S","S","S","S","S","S","B","S","S","S","B","C","C","B","S","B","C","C","B","C","B"]}}} ``` ## SymetryML Project Data Source API [#symetryml-project-data-source-api] ### Add Data Source to a SymetryML project [#add-data-source-to-a-symetryml-project] This API function lets you add a data source to a project. #### URL [#url-8] ``` GET /symetry/rest/{cid}/projects/{pid}/addds/{dsname} ``` #### HTTP Responses [#http-responses-8] | HTTP Status Code | HTTP Status Message | Description | | ---------------- | ------------------- | ----------- | | **200** | OK | Success. | #### HTTP Response Entity [#http-response-entity-8] None ### Remove Data Source from a SymetryML project [#remove-data-source-from-a-symetryml-project] This API function lets you remove a data source from a project. #### URL [#url-9] ``` GET /symetry/rest/{cid}/projects/{pid}/detachds/{dsname} ``` #### HTTP Responses [#http-responses-9] | HTTP Status Code | HTTP Status Message | Description | | ---------------- | ------------------- | ----------- | | **200** | OK | Success. | #### HTTP Response Entity [#http-response-entity-9] None ### Learning Data from a Data Source [#learning-data-from-a-data-source] This API function lets you learn from a previously created data source. #### URL [#url-10] ``` GET /symetry/rest/{cid}/projects/{pid}/dss/{dsname}/learn ``` #### HTTP Responses [#http-responses-10] | HTTP Status Code | HTTP Status Message | Description | | ---------------- | ------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | **200** | OK | Success. Includes an HTTP Location header specifying the location of the job ID that was created to handle the request. Example: `{"statusCode":"ACCEPTED","statusString":"Job Created","values":{}}` | #### HTTP Response Entity [#http-response-entity-10] None #### Sample Request/Response [#sample-requestresponse-7] ``` Request 1 create ds: POST url=http://charm:8080/symetry/rest/c1/dss/create Body 1: {ENCRYPTED} Request 2 learn ds: POST url=http://charm:8080/symetry/rest/c1/projects/r1/dss/irisds/learn BODY 2 {"errorHandling":1,"attributeNames":["sepal_length","sepal_width","petal_length","petal_width","sepal_lengt_b1","sepal_lengt_b2","sepal_width_b1","sepal_width_b2","petal_length_b1","petal_length_b2","petal_width_b1","petal_width_b2","Iris_setosa","Iris_versicolor","Iris_virginica"],"data":[],"attributeTypes":["C","C","C","C","B","B","B","B","B","B","B","B","B","B","B"]} Response 2: {"statusCode":"ACCEPTED","statusString":"Job Created","values":{}} Response 2 Header: Location: http://charm:8080/symetry/rest/c1/jobs/4 Job 3 Request: GET url="http://charm:8080/symetry/rest/c1/jobs/4 Job 3 Response: {"statusCode":"OK","statusString":"Job is finished","values":{"smlInfo":{"pid":"r1","isDirty":true,"modelsList":[],"modelTypeList":[],"attributeNames":["sepal_length","sepal_width","petal_length","petal_width","sepal_lengt_b1","sepal_lengt_b2","sepal_width_b1","sepal_width_b2","petal_length_b1","petal_length_b2","petal_width_b1","petal_width_b2","Iris_setosa","Iris_versicolor","Iris_virginica"],"attributeIndexes":[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14],"attributeTypes":["C","C","C","C","B","B","B","B","B","B","B","B","B","B","B"],"modelAssessments":{},"modelPredictions":{},"hash":-1,"categorySeparator":"$","type":"cpu","creationDate":1488220762857,"lastModificationDate":1488226030143,"loaded":true,"persisted":true}}} ``` ### Forgetting Data from a Data Source [#forgetting-data-from-a-data-source] This API function lets you forget data from a previously created data source. #### URL [#url-11] ``` GET /symetry/rest/{cid}/projects/{pid}/dss/{dsname}/forget ``` #### HTTP Responses [#http-responses-11] | HTTP Status Code | HTTP Status Message | Description | | ---------------- | ------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | **200** | OK | Success. Includes an HTTP Location header specifying the location of the job ID that was created to handle the request. For example: `{"statusCode":"ACCEPTED","statusString":"Job Created","values":{}}` | #### HTTP Response Entity [#http-response-entity-11] None #### Sample Request/Response [#sample-requestresponse-8] See the section previous [Learn](/docs/symetryml-rest-client/rest-documentation/data-source-api#learning-data-from-a-data-source) endpoint and replace `learn` with `forget` in the url. ### Prediction Based on a Data Source [#prediction-based-on-a-data-source] After a model is built, you can use this API function to make predictions using a data source. This action can be performed on very large files if they reside on Amazon S3. A prediction file is created that contains the original rows, along with additional prediction information based on the type of model used. #### URL [#url-12] ``` Request: POST /symetry/rest/{cid}/projects/{pid}/dss/predict/{modelid}?indsname={indsname}&outdsname={outdsname}&impute=false Request Body: { "attributeNames":[{input attributes names}], "attributeTypes":[{input attribute types}] } ``` #### Query Parameters [#query-parameters-1] | Parameter | Required / Optional | Description | | ------------- | ------------------- | -------------------------------------------------------------- | | **indsname** | Required | Data source to use as input file for prediction. | | **outdsname** | Required | Data source to use as output file for prediction. | | **impute** | Optional | Boolean parameter specifying whether to impute missing values. | **Output format (7.0):** the prediction file is written as RFC 4180 CSV. A value that contains a comma, a quote or a line break is enclosed in double quotes, with embedded quotes doubled. A text value that starts with `=`, `+`, `-` or `@` is prefixed with an apostrophe (`'`) so that a spreadsheet does not run it as a formula; plain numbers, including negative predictions, are written unchanged. A missing value is still written as `null`. #### HTTP Responses [#http-responses-12] | HTTP Status Code | HTTP Status Message | Description | | ---------------- | --------------------- | ------------------------------------------------------------------------------------------------------------------------- | | **202** | ACCEPTED | Success. Includes an HTTP Location header specifying the location of the job ID that was created to handle the request. | | **500** | INTERNAL SERVER ERROR | If the server refuses to accept the new job, it notifies the client with the error "Job execution was refused by server." | #### HTTP Response Entity [#http-response-entity-12] None ### Sample Request/Response [#sample-requestresponse-9] ``` Request: POST url="http://charm:8080/symetry/rest/c1/projects/irisTest/dss/predict/testLDA?indsname=dsin&outdsname=dsout&impute=false Request Body: { "attributeNames":["sepal_length","sepal_width","petal_length","petal_width"], "attributeTypes":["C","C","C","C"] } Response Header: Location: http://charm:8080/symetry/rest/c1/jobs/2 Response: {"statusCode":"ACCEPTED","statusString":"Job Created","values":{}} Job Request: GET url="http://charm:8080/symetry/rest/c1/jobs/2 Job Response: {"statusCode":"OK","statusString":"Job is finished","values":{"dataframe":{"attributeNames":["resZ_Result","res_Result","normZ_Result","sepal_length","sepal_width","petal_length","petal_width","sepal_lengt_b1","sepal_lengt_b2","sepal_width_b1","sepal_width_b2","petal_length_b1","petal_length_b2","petal_width_b1","petal_width_b2","Iris_setosa","Iris_versicolor","Iris_virginica"],"data":[["-0.4559922893809558","1","0.2929126883636288","4.3","3","1.1","0.1","1","0","0","1","1","0","1","0","1","0","0"],["-0.5225734165408404","1","0.2965158290613048"," (…) ["-2.6526076600477295","1","0.4232667751179995","6.9","3.1","5.4","2.1","0","1","0","1","0","1","0","1","0","0","1"]]}}} ``` ## Encoder Data Source API [#encoder-data-source-api] ### Updating an Encoder with a Data Source [#updating-an-encoder-with-a-data-source] This API function updates an Encoder with data from a data source. #### URL [#url-13] ``` GET /symetry/rest/{cid}/encoders/{encodername}/learnds [body = DSInfo (encrypted)] ``` #### HTTP Responses [#http-responses-13] | HTTP Status Code | HTTP Status Message | Description | | ---------------- | --------------------- | ------------------------------------------------------------------------------------------------------------------------- | | **202** | ACCEPTED | Success. Includes an HTTP Location header specifying the location of the job ID that was created to handle the request. | | **500** | INTERNAL SERVER ERROR | If the server refuses to accept the new job, it notifies the client with the error "Job execution was refused by server." | #### HTTP Response Entity [#http-response-entity-13] None ### Sample Request/Response [#sample-requestresponse-10] ``` Request: POST url="http://charm:8080/symetry/rest/c1/encoders/enctest/learnds Request Body: emUmJ7LXKaICPww/dKRoMZo4y++JZksKFU3xx80mZ0E83ED6eCXhZ7PR1jzizIQg/uH19jVkSUWnDxul9+NIqZe7a9Yz5a8gW1kpRHJ7SyOYoo7oV90atrkMcZ73Jj0FRn53P81t8Q+7fCTwMYRfD52hjCbwtvdwFCbPbhPo1c9CAk9QLhtAFLDWVqblBeBXTPR8/0zfKJWtwj30Yr0gwqeTI3+BMOMEvH28WFWXo+wBdgkMBpXJsQH/zbPvVdCp9P+BJWp/E1Ju2hUzPO5c2k3/Dmqv3xWNmsQzEJWNMFTJnBG33hTUyTr/+j87NsM2e1luWvf1KaNzaatjS1ZZ9AQCp7gv48vMVrzEHs4ePiOFq5t0UElAu1kzFerhtNxEFmC9A90Gjt3FSLrEGgx5emZ0uogJa6m4nufbQnKUrUZL1sILmLfReZOqnPKp59HqjiiczpkPQ2vPfhmFpaM2RYaEHbqSpMGCnBO9Axij9ExPjR4X9aqHOzKw2yBba4Em Response Header: Location: http://charm:8080/symetry/rest/c1/jobs/4 Response: {"statusCode":"ACCEPTED","statusString":"Job Created","values":{}} Job Request: GET url="http://charm:8080/symetry/rest/c1/jobs/4 Job Response: {"statusCode":"OK","statusString":"Job is finished","values":{}} ``` ### Listing a Data Source Used by an Encoder [#listing-a-data-source-used-by-an-encoder] This API function lists the data source(s) that were used to update an Encoder. #### URL [#url-14] ``` GET /symetry/rest/{cid}/encoders/{encodername}/dss ``` #### HTTP Responses [#http-responses-14] | HTTP Status Code | HTTP Status Message | Description | | ---------------- | ------------------- | ----------------------------------------------------------------------------------------------------------------------- | | **202** | ACCEPTED | Success. Includes an HTTP Location header specifying the location of the job ID that was created to handle the request. | #### HTTP Response Entity [#http-response-entity-14] | HTTP Response Entity | Example | | -------------------- | ----------------------------------------------------------------------------------------------------------------- | | **StringList** | See [StringList](/docs/symetryml-rest-client/rest-documentation/appendix-a-json-data-structure-schema#stringlist) | #### Sample Request/Response [#sample-requestresponse-11] ``` Request: GET url="http://charm:8080/symetry/rest/c1/encoders/enctest/dss Response: {"statusCode":"OK","statusString":"OK","values":{"stringList":{"values":["s12"]}}} ``` ## Data Source Job Status [#data-source-job-status] When invoking a JobStatus for a job that was initiated for learning, forgetting, or making predictions based on a data source, the response might contain an entity. The following sections describe these cases. ### Learning a Data Source [#learning-a-data-source] #### HTTP Responses [#http-responses-15] | HTTP Status Code | HTTP Status Message | Description | | ---------------- | ------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | **200** | OK | No entity. | | **202** | ACCEPTED | Job is not finished. Includes a [JobInfo](/docs/symetryml-rest-client/rest-documentation/appendix-a-json-data-structure-schema#jobinfo) entity. Using the current and maximum fields, you can compute the percentage of the job that was accomplished so far. "Maximum" refers to the size of the file to process and contains the approximate number of bytes processed by the job. | ### Forgetting a Data Source [#forgetting-a-data-source] #### HTTP Responses [#http-responses-16] | HTTP Status Code | HTTP Status Message | Description | | ---------------- | ------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | **200** | OK | No entity. | | **202** | ACCEPTED | Job is not finished. Includes a JobInfo entity. For more information, see the section [JobInfo](/docs/symetryml-rest-client/rest-documentation/appendix-a-json-data-structure-schema#jobinfo) and also the section on [Symetry Jobs](/docs/symetryml-rest-client/rest-documentation/symetryml-job-information). Using the current and maximum fields, you can compute the percentage of the job that was accomplished so far. "Maximum" refers to the size of the file to process and contains the approximate number of bytes processed by the job. | ### Prediction Based on Data Source [#prediction-based-on-data-source] ### HTTP Responses [#http-responses-17] | HTTP Status Code | HTTP Status Message | Description | | ---------------- | ------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | **200** | OK | DataFrame that contains a sample of the predictions (up to 128 lines). Because Amazon files can be very large, it is not possible to return the prediction result file in its entirety within a REST call. Use your favorite tool to fetch the prediction results from the data source (S3 or SFTP). The prediction result file contains all the original file columns, plus the additional prediction column for each row. Any additional columns depend on the type of model used to make the predictions. | | **202** | ACCEPTED | Job is not finished. Includes a [JobInfo](/docs/symetryml-rest-client/rest-documentation/appendix-a-json-data-structure-schema#jobinfo) entity. Using the current and maximum fields, it is possible to compute the percentage of the job that was accomplished so far. "Maximum" refers to the size of the file to process and contains the approximate number of bytes processed by the job. |