Retrieving a list of files with JavaScript

There are a lot of routine tasks that are not always related to data cleaning, machine learning, or writing a script in Python in the analyst's work. The journey usually begins with data import or search. With this script, a user can retrieve a list of objects from the computer's file system or Megaladata File Storage for various purposes, including solving multiple import tasks. Implemented using the JavaScript fileapi library.

Start demo

Download example

Algorithm Description

It is recommended to use this submodel as a derived component.. It includes 2 nodes: Retrieving a list of files, Table into Variables.

Implementation of the algorithm

Two variables are specified in the input port of variables: Path along which the component will output a list of files, and Boolean WithFolder. By default, the path is set: \user, in order to get a list of files that are in the folder on the demo stand. When running the example locally, you need to specify the path to the folder in the computer's file system.

In the JavaScript node, the output table columns are configured. It will have the following structure:

Name Capture
Name Name
FileExtension FileExtension
FilePath FilePath
Size Size
isValidFilePath isValidFilePath
isFolder isFolder
mDate mDate

Code:

import { InputVariables, OutputTable } from "builtIn/Data";
import * as fs from "builtIn/FS"; 

const { isValidFilePath, Name, isFolder, FileExtension, Size, FilePath, mDate } = OutputTable.Columns;

let path = InputVariables.Items.Path.Value || ".";


if (!fs.existsSync(path)) { // Condition under which the path does not exist 
    OutputTable.Append();
    isValidFilePath.Set(InputVariables.Items.Error.Value); // Error output (variable value)
} else {
    try {
        let files = fs.readdirSync(path, { withFileTypes: true }); // Creating a files object that contains information about the files at the specified path
        for (let file of files) { // A loop in which file system objects are processed 
            OutputTable.Append(); 
            let fname = path + "/" + file.name; // Formation of the full file name (along with the path)
            isValidFilePath.Set(true); // Adding a value to the correct_path field
            Name.Set(file.name);
            FilePath.Set(fname);
            let stat = fs.statSync(fname);
            if (file.isDirectory()) { // Checking if an object is a file
                isFolder.Set(true);
            } else {
                let p = file.name.lastIndexOf("."); // Getting the position of a point
                let ext = "";
                if (p > -1) {
                    ext = file.name.substring(p + 1); // Formation of expansion
                }
                FileExtension.Set(ext);
                Size.Set(stat.size);
                isFolder.Set(false);
            }
            FilePath.Set(fname);
            mDate.Set(stat.mtime);
        }
    } catch (e) { // Обработка ошибок
        OutputTable.Append();
        isValidFilePath.Set(e);
    }
}

In the Table into Variables node, errors and script execution exceptions are output to the variables port. If there are no errors, true is output.

Output results

The SuperNode has an output port for tables and a port for variables.

Name Caption
Name Name
FileExtension FileExtension
FilePath FilePath
Size Size
isFolder isFolder
mDate mDate

Variables:

Name Capture Description
result result true/false

Download and open the file in Megaladata. If necessary, you can install the free Megaladata Community Edition.

Download example

results matching ""

    No results matching ""