phares3757/pocketbase-import
3
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

add batch request chunking (#1)

Browse Source 
+ multiple batch requests of size configurable via max_batch
This commit is contained in:
michal-kapala
2024-11-27 23:29:47 +01:00
parent ec13e0fb17
commit 78f3dfc7bd
4 changed files with 126 additions and 46 deletions
Download Patch File Download Diff File Expand all files Collapse all files

27
README.md
View File

@ -51,13 +51,14 @@ For versions `>=0.23.2`, enable and configure [batch API](https://pocketbase.io/
You can change the default import options to your needs:
| Name | Files | Required | Description | Example use |
| --------- | -------- | -------- | ------------------------------------------------------------------------------------------ | ------------------- |
| input | CSV/JSON | Yes | The name of the input file (with extension) | --input=example.csv |
| id | CSV/JSON | No | Indicates that `_id` column should be typed as plain text, the type is detected by default | --id |
| lf | CSV | No | LF (`\n`) EOL character will be used instead of default CRLF (`\r\n`) | --lf |
| delimiter | CSV | No | Column value separator, defaults to `,` | --delimiter=";" |
| quote | CSV | No | Value quote character, defaults to `'` | --quote="~" |
| Name | Files | Required | Description | Example use |
| --------- | -------- | -------- | ------------------------------------------------------------------------------------------------------------ | ------------------- |
| input | CSV/JSON | Yes | The name of the input file (with extension) | --input=example.csv |
| id | CSV/JSON | No | Indicates that `_id` column should be typed as plain text, the type is detected by default | --id |
| max_batch | CSV/JSON | No | Max batch request size in rows, should not exceed PocketBase's `Max allowed batch requests`. Defaults to 50. | --max_batch=100 |
| lf | CSV | No | LF (`\n`) EOL character will be used instead of default CRLF (`\r\n`) | --lf |
| delimiter | CSV | No | Column value separator, defaults to `,` | --delimiter=";" |
| quote | CSV | No | Value quote character, defaults to `'` | --quote="~" |
# CSV
@ -74,6 +75,12 @@ Basic import (root directory):
deno run csv.ts --input=example.csv
```
Import without permission prompts and with max batch request size of 1 row:
```
deno run --allow-read --allow-env --allow-net csv.ts --input=example.csv --max_batch=1
```
Import without permission prompts and with `_id` column as text:
```
@ -98,6 +105,12 @@ Basic import (root directory):
deno run json.ts --input=example.json
```
Import without permission prompts and with max batch request size of 1 row:
```
deno run --allow-read --allow-env --allow-net json.ts --input=example.json --max_batch=1
```
Import without permission prompts and with `_id` column as text:
```