Retrieving a list of files with JavaScript
There are a lot of routine tasks that are not always related to data cleaning, machine learning, or writing a script in Python in the analyst's work. The journey usually begins with data import or search. With this script, a user can retrieve a list of objects from the computer's file system or Megaladata File Storage for various purposes, including solving multiple import tasks. Implemented using the JavaScript fileapi library.
Algorithm Description
It is recommended to use this submodel as a derived component.. It includes 2 nodes: Retrieving a list of files, Table into Variables.
Two variables are specified in the input port of variables: Path
along which the component will output a list of files, and Boolean WithFolder
. By default, the path is set: \user
, in order to get a list of files that are in the folder on the demo stand. When running the example locally, you need to specify the path to the folder in the computer's file system.
In the JavaScript node, the output table columns are configured. It will have the following structure:
Name | Capture |
---|---|
Name | Name |
FileExtension | FileExtension |
FilePath | FilePath |
Size | Size |
isValidFilePath | isValidFilePath |
isFolder | isFolder |
mDate | mDate |
Code:
import { InputVariables, OutputTable } from "builtIn/Data";
import * as fs from "builtIn/FS";
const { isValidFilePath, Name, isFolder, FileExtension, Size, FilePath, mDate } = OutputTable.Columns;
let path = InputVariables.Items.Path.Value || ".";
if (!fs.existsSync(path)) { // Condition under which the path does not exist
OutputTable.Append();
isValidFilePath.Set(InputVariables.Items.Error.Value); // Error output (variable value)
} else {
try {
let files = fs.readdirSync(path, { withFileTypes: true }); // Creating a files object that contains information about the files at the specified path
for (let file of files) { // A loop in which file system objects are processed
OutputTable.Append();
let fname = path + "/" + file.name; // Formation of the full file name (along with the path)
isValidFilePath.Set(true); // Adding a value to the correct_path field
Name.Set(file.name);
FilePath.Set(fname);
let stat = fs.statSync(fname);
if (file.isDirectory()) { // Checking if an object is a file
isFolder.Set(true);
} else {
let p = file.name.lastIndexOf("."); // Getting the position of a point
let ext = "";
if (p > -1) {
ext = file.name.substring(p + 1); // Formation of expansion
}
FileExtension.Set(ext);
Size.Set(stat.size);
isFolder.Set(false);
}
FilePath.Set(fname);
mDate.Set(stat.mtime);
}
} catch (e) { // Обработка ошибок
OutputTable.Append();
isValidFilePath.Set(e);
}
}
In the Table into Variables node, errors and script execution exceptions are output to the variables port. If there are no errors, true is output.
The SuperNode has an output port for tables and a port for variables.
Name | Caption |
---|---|
Name | Name |
FileExtension | FileExtension |
FilePath | FilePath |
Size | Size |
isFolder | isFolder |
mDate | mDate |
Variables:
Name | Capture | Description |
---|---|---|
result | result | true/false |
Download and open the file in Megaladata. If necessary, you can install the free Megaladata Community Edition.