Note

This page was generated from user/data.ipynb.

Data Handling#

cherab-lhd handles LHD-related data like EMC3-EIRENE grid configuration, machine geometry, measurement geometry, etc. Here is a brief description of the dataset containing the above-mentioned data.

Dataset Registries#

cherab-lhd provides a list of hash values of the files instead of providing the data files themselves. fetch module offers functionalities to manage the download of the data files by checking if the files are not present in the cache directory.

[1]:

from cherab.lhd.tools.fetch import show_registries

show_registries()

                                           Registries                                            
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ File Name                  ┃ SHA256                                                           ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ emc3/grid-360.nc           │ 948c65fbefc78118272e223720930d10846b1b177f513b4cadfc33ec9a177f3d │
├────────────────────────────┼──────────────────────────────────────────────────────────────────┤
│ machine/calcam_cad.ccm     │ a93db1a11d16357ccbfe797bf71d7d78aaf008a9e95e6c0aa15a7bb1207cb528 │
├────────────────────────────┼──────────────────────────────────────────────────────────────────┤
│ machine/divertor.rsm       │ 6ae9b4b50139818b7e600a247bb4fc3d1260bac28a60234a60fbd6c1aebdc1ca │
├────────────────────────────┼──────────────────────────────────────────────────────────────────┤
│ machine/plasma.rsm         │ ae950c8d787fad3ebccbfef12f2282cd927d15a6cc9554db28c9d39a4f7eda38 │
├────────────────────────────┼──────────────────────────────────────────────────────────────────┤
│ machine/plasma_cut.rsm     │ 10fb5fc23ccc490b265a12e8e77b962a3cd4c8b899608bb30de6ce0753d45ea0 │
├────────────────────────────┼──────────────────────────────────────────────────────────────────┤
│ machine/plasma_quarter.rsm │ d517d96e690fd458e90379d0e9e6e06e4da8793986f28d636817d59b1eb9171d │
├────────────────────────────┼──────────────────────────────────────────────────────────────────┤
│ machine/plates.rsm         │ d479746123d071c5070b5942ca3a5d53fbe07dda73d7651e64e951c04fbf33c7 │
├────────────────────────────┼──────────────────────────────────────────────────────────────────┤
│ machine/port_65l.rsm       │ cd0debab846d86d097d3b66007496e13b7b8fc76896c102b758a4f47478e3adb │
├────────────────────────────┼──────────────────────────────────────────────────────────────────┤
│ machine/port_65u.rsm       │ 46ad570bd222b7ceae8f2098c65ad2a45d53ef95845b4ac75be221acf60b057c │
├────────────────────────────┼──────────────────────────────────────────────────────────────────┤
│ machine/vessel.rsm         │ ad90ab5b596c317a329cb7820d46da2f54fcb844476bae1abe0203dc383a89bf │
├────────────────────────────┼──────────────────────────────────────────────────────────────────┤
│ machine/wall_outline.nc    │ 77883c34bb89bcf909bb15d612e750b58194726a87cf94fe310a2e80bba625e3 │
├────────────────────────────┼──────────────────────────────────────────────────────────────────┤
│ material/sus316L.json      │ 0f473be0d3f9efc88d11d3cbacbc5a8596ff2b39dcb2dbd561d0fa116de5f301 │
├────────────────────────────┼──────────────────────────────────────────────────────────────────┤
│ observer/IRVB.json         │ a46010c1d67bbe71c0bf71c203447cb984c5622e63e10d20f5ee8b8d2cf93248 │
├────────────────────────────┼──────────────────────────────────────────────────────────────────┤
│ observer/RB.json           │ bffd237b98bc6a158fb4f4875d1d79d6571fe4770561f8e18fd1da72a63bcae2 │
├────────────────────────────┼──────────────────────────────────────────────────────────────────┤
│ tetra/zone0.rsm            │ 2859a0618badd7d764691750bbc617479a00ee4f0a31dcf9d616d81174f3402b │
├────────────────────────────┼──────────────────────────────────────────────────────────────────┤
│ tetra/zone1.rsm            │ a097e6f5dcf0319e3109fc0d7c010442297e6078778cbc795bc37729ff6fc2be │
├────────────────────────────┼──────────────────────────────────────────────────────────────────┤
│ tetra/zone11.rsm           │ dc45ebe22b858db13f1cd73e43422629dde6adc2f8ca2a4c14cb8e4a2b83dd14 │
├────────────────────────────┼──────────────────────────────────────────────────────────────────┤
│ tetra/zone12.rsm           │ a16ef6d09ad1f7551280d405d91aa63021b5eac6e0e397e563ee3130ed52c98e │
├────────────────────────────┼──────────────────────────────────────────────────────────────────┤
│ tetra/zone13.rsm           │ 8ded151d42952c3b83b9293d9c3e4be28ebe847608d7e8707d8daf518063949f │
├────────────────────────────┼──────────────────────────────────────────────────────────────────┤
│ tetra/zone14.rsm           │ 3ab9bfd2e8c2b6631c269a02a1597deebde1fce973ab61f3db9065054ced5538 │
├────────────────────────────┼──────────────────────────────────────────────────────────────────┤
│ tetra/zone15.rsm           │ 25381a62eda6c217f280e0bf8d93fe077bb1c517a1e3dfa1fc60043db5be5fa2 │
├────────────────────────────┼──────────────────────────────────────────────────────────────────┤
│ tetra/zone2.rsm            │ 33f22ac916633281e7c246f1092b24e41e2b9ca9d4497793c2c87a349bad526a │
├────────────────────────────┼──────────────────────────────────────────────────────────────────┤
│ tetra/zone3.rsm            │ 0e9178decb85664309a5de2b6d0d1ea3df0866ecface573407c276f05547bad3 │
├────────────────────────────┼──────────────────────────────────────────────────────────────────┤
│ tetra/zone4.rsm            │ 42c5c1469d9fd4acb14e5dbb581dda570804ce84f232c70178feb42cab0681cc │
└────────────────────────────┴──────────────────────────────────────────────────────────────────┘

Retrieving the Dataset#

The dataset is downloaded through sftp protocol from the remote server if it is not already available in the local cache directory or its hash differs from the one in the registry.

Before downloading the dataset, users need to have access to the remote server. The dataset has not been opened to the public yet. Those who want to access the dataset must contact the package maintainers.

After getting permission to deal with the dataset, users need to set the following environment variables:

  • SSH_RAYTRACE_HOSTNAME: The hostname of the remote server (e.g. sftp://hostname/directory/).

  • SSH_RAYTRACE_USERNAME: The username to log in to the remote server.

  • SSH_RAYTRACE_PASSWORD: The password to log in to the remote server.

Then, use the fetch_file function:

>>> from cherab.lhd.tools.fetch import fetch_file
>>> path = fetch_file("emc3/grid-360.nc")

Or, manually specify them as arguments to the fetch_file function.

>>> path = fetch_file("emc3/grid-360.nc", host="sftp://hostname/directory/", username="username", password="password")

Downloaded files are stored in the user’s cache directory:

  • Linux: ~/.cache/cherab/lhd/

  • macOS: ~/Library/Caches/cherab/lhd/

If the dataset is no longer needed, please delete the downloaded files manually.

Locate the Dataset file manually#

Users can put dataset files in the cache directory manually. Please confirm the cache directory depending on the operating system as specified above and place the files under the cherab/lhd directory with corresponding subdirectories (e.g. emc3, machine, observer, etc.)

Note that even if the files are placed locally, they will be downloaded from the remote server if the hash value is different from the one in the registry.