Python Package for scientific analysis of nanoscience data
Pycroscopy is undergoing a major reorganization and change to the scope and nature of the package.
We run weekly hackathons to develop the pycroscopy ecosystem of python packages. Hackathons are held every Friday 3-5PM USA Eastern time. The requirements for participation are: knowledge of python, git, and the basic structure and philosophy of the pycroscopy ecosystem (available through documentation). If you would like to participate, please email us at vasudevanrk at ornl.gov
This version of the pycroscopy python package is under development and is not yet available to download or use.
Code in this package is meant to be useful for multiple scientific domains or applications.
The upcoming version of pycroscopy is being thoroughly restructured and will be substantially different to current or prior versions. Differences between the upcoming and existing legacy versions are largely centered in how data is handled:
The reimagined pycroscopy package will not deal with data files.
Readersto extract data and metadata from scientific data files into python objects in memory. This is unlike
Translatorsthat were part of pycroscopy that wrote the extracted data into USID – Universal Spectroscopy and Imaging and Data formatted HDF5 files.
Input and output data would be exchanged in the form of
sidpy.Datasetobjects rather than HDF5 Datasets in a file
pycroscopy will not force the use of specific computational backends like
The upcoming version of pycroscopy will be organized as follows:
learn- machine and deep learning tools
stats- statistics tools
image- image analysis and processing tools
signal- signal processing and analysis tools
corr- tools to correlate datasets from multiple sources (images with spectra, simulation with experiment, experiments with machine learning, etc.)
viz- visualization tools and dashboards
The source code is available under the phoenix branch.
V 0.60.7 is the last version of the legacy iteration of pycroscopy available through pip and conda.
For those interested in the older version, please visit the legacy branch, which will not be amended from hereon. The new version is under development in the ‘phoenix’ branch, which will be shifted to master in due course.
The pycroscopy package has so far focused on providing standardized solutions for processing, analyzing, and visualizing multidimensional imaging and spectroscopy data.
The legacy iteration of pycroscopy used a data and file-centric approach based on the USID – Universal Spectroscopy and Imaging and Data model wherein the raw data collected from the microscope, results from analysis and processing routines are all written to standardized hierarchical data format (HDF5) files for traceability, reproducibility, and provenance.
pycroscopy therefore used pyUSID which provides tools to read, write, visualize, and process USID data stored in HDF5 files.
The following provides an overview of the existing organization of the pycroscopy package and how these capabilities have been / will be made available in the reimagined pycroscopy:
Atom finding functions - will be made available under the
imagesubpackage of the reimagined pycroscopy
Band Excitation and General-mode specific functional fitting that have been moved to BGlib
Unsupervised machine learning wrappers -
SVD- these will be available via the
learnsubpackage of the reimagined pycroscopy.
SignalFilterwill be available via the
signalsubpackage of the reimagined pycroscopy
image_processingwill be made available as well.
Band Excitation visualizers - have been moved to
Clustering visualization - will be moved to
vizin the new pycroscopy
Image cleaning visualization - will be moved to
vizin the new pycroscopy
V 0.60.7 is the last version of the legacy iteration of pycroscopy available through pip and conda. All future versions will be on the reimagined package described above.
The source code for the legacy iteration always be available on the legacy branch.