What’s New

Attention

This page is not up to date. Please visit the github project for latest information

Jun 28 2018:

Moved pycroscopy.core into separate package - pyUSID pyUSID will be the engineering package that supports science-focused packages such as pycroscopy similar to how scipy depends on numpy. All references to pycroscopy.core within the pycroscopy package are now referencing pyusid instead. The current release of pycroscopy imports pyUSID and makes it available as pycroscopy.core so that existing imports in user-code do not break. In the next release of pycroscopy, this implicit import will be removed and the following modules would have to be imported directly from pyUSID:

  • hdf_utils

  • write_utils

  • dtype_utils

  • io_utils

  • PycroDataset - renamed to USIDataset

  • Translator

  • ImageTranslator

  • NumpyTranslator

  • Process

  • parallel_compute()

  • plot_utils

  • jupyter_utils

Thus, imports and usages of such modules as:

import pycroscopy as px
px.plot_utils.plot_map(...)
px.hdf_utils.print_tree(h5_file)
px.PycroDataset(h5_dset)
# Other non-core classes:
px.processing.SignalFilter(h5_main, ...)

would need to be changed to:

# Now import pyUSID along with pycroscopy
import pyUSID as usid
import pycroscopy as px
# functions and classes still work the same way
# just use usid instead of px for anything that was in core (see list above).
usid.plot_utils.plot_map(...)
usid.hdf_utils.print_tree(h5_file)
# The only difference is the renaming of the PycroDataset to USIDataset:
usid.USIDataset(h5_dset)
# Other classes and functions outside .core are addressed just as before:
px.processing.SignalFilter(h5_main, ...)

Jun 19 2018:

Jun 18 2018:

Jun 15 2018:

  • Thanks to @ramav87 for bug-fixes in BEPS related translators and notebooks

Jun 14 2018:

  • Thanks to @ealopez for adding AFM simulations

  • Thanks to @nmosto for guides for python novices

Jun 13 2018:

  • Thanks to @str-eat for implementing a PycroDataset to csv exporter

Jun 04 2018:

  • Thanks to @donpatrice for fixing a UTF8 isue with the NanonisTranslator

Jun 01 2018:

  • First skeleton GwyddionTranslator being worked on by @str-eat

  • Added guidelines for contributing code

May 31 2018:

  • All Translators now use absolute paths

  • Improved examples and documentation

May 30 2018:

  • Thanks to @carlodri for donating his utility to read Gwyddion Simple File (gsf) reader

  • Added gwyfile to the requirements of pycroscopy

  • NumpyTranslator now accepts extra datasets and keyword arguments that will be passed on to hdf_utils.write_main_dataset()

May 26 2018:

  • Implemented a general function for reading sections of binary files

  • First version of the BrukerTranslator for translating Bruker Icon and other AFM files

May 03 2018:

  • plot_utils.plot_map() now accepts the extent or the tick values

  • Fixed bug in hdf_utils.write_reduced_spec_dsets() and analysis.BESHOFitter

  • General improvements to the analysis.Fitter class

  • Documentation updates

May 02 2018:

  • Fixed bug in svd_rebuild()

May 01 2018:

  • Minor corrections to documentation formatting

  • pycroscopy.hdf_utils.get_auxillary_datasets() renamed to pycroscopy.hdf_utils.get_auxiliary_datasets()

  • Example on parallel computing rewritten to focus on pycroscopy.parallel_compute()

  • Added setUp() and tearDown() to unit testing classes for hdf_utils and PycroDataset

  • Fixed bug in the sorting capability of pycroscopy.hdf_utils.reshape_to_n_dims()

  • Added logo to website

Apr 29 2018 2:

  • Centralized verification of slice dictionary in pycroscopy.PycroDataset

  • The slice_dict kwarg in pycroscopy.PycroDataset.slice() now the first required argument

  • Lots of minor formatting changes to examples.

  • Removed jupyter notebooks from which the examples were generated.

Apr 29 2018 1:

  • Fixed errors in broken examples

  • Replaced example BE datasets with ones where the central datasets now have quantity and units attributes to make them Main datasets

  • Replaced example STS dataset with a zip file which will download a lot faster for examples. Corrected the example on NumpyTranslator

Apr 28 2018 2:

  • Fixed unit tests for python 2. assertWarns() only applied to python 3 now

  • Added from future import statement to all modules in pycroscopy.core

Apr 28 2018 1:

(Massive) merge of (skunkworks) branch unity_dev into master``

  • Added unit tests for all (feasible) modules in pycroscopy.core

  • Added examples for every single function or class in pycroscopy.core (10 cookbooks in total!)

  • Added a primer to h5py and HDF5

  • Added document with instructions on converting unit tests to examples of functions.

  • Added web page with links to external tutorials

  • Added web page describing contents of package, organization,

  • Added web page with FAQs

  • Moved a simplified (non ptychography version of) ImageTranslator to pycroscopy.core

  • Package reconfigured to use pytests instead of Nose

  • Converted last few assert statements into descriptive Errors

  • Legacy HDF writing classes and functions deprecated now and will be removed in a future release:

    • hdf_writer and virtual_data modules moved out of pycroscopy.core.io and back into pycroscopy.io.

    • Moved functions in pycroscopy.write_utils using above deprecated classes into pycroscopy.io.write_utils. These functions are also deprecated

    • pycroscopy.translators.BEODFTranslator, pycroscopy.analysis.BESHOFitter, and pycroscopy.BELoopFitter, pycroscopy.processing.SignalFilter, pycroscopy.translators.GIVTranslator, pycroscopy.analysis.GIVBayesian, pycroscopy.processing.gmode_utils, etc. now do not use deprecated classes as proof that even the most complex classes can easily be transitioned to using functions in pycroscopy.core.io.hdf_utils and pycroscopy.core.io.write_utils

    • Unit tests for modules in pycroscopy.core.io rewritten to not use deprecated functions or classes.

    • Deprecated classes only being used in translators, two analyses modules and two process modules

    • Removed old examples and tutorials, especially on deprecated classes

  • Upgrades to pycroscopy.Process:

    • pycroscopy.Process now has a new function - test() that allows much easier in-place testing of processes before applying to the entire dataset

    • pycroscopy.processing.Cluster`, pycroscopy.processing.Decomposition, pycroscopy.processing.SVD, pycroscopy.processing.SignalFilter, pycroscopy.processing.GIVBayesian all implement the new test() functionality - return results as correct N-dimensional datasets in expected datatypes

    • pycroscopy.processing.Cluster, pycroscopy.processing.Decomposition now use a user-configured sklearn objects as inputs instead of creating an estimator object

    • SVD, Cluster, Decomposition now correctly write results as Main datasets

  • More robust pycroscopy.gmode_utils functions

  • Updates to pycroscopy.plot_utils:

    • plot_complex_loop_stack merged into plot_complex_spectra()

    • new function that provides best row / column configuration for (identical) subplots: get_plot_grid_size()

    • moved clustering related utilities into pycroscopy.viz.cluster_utils <– significantly revised many functions in there

    • plot_map_stack() accepts x, y labels. plot_map() accepts X and Y vectors instead of sizes for more granular control

    • All compound functions now pass kwargs to underlying functions wherever possible

  • Updates to pycroscopy.write_utils:

    • pycroscopy.write_utils.AncillaryDescriptor and pycroscopy.jupyter_utils.VizDimension merged and significantly simplified to pycroscopy.write_utils.Dimension

    • Swapped all usages of AncillaryDescriptor with Dimension in the entire package

    • More robust handling of attributes with numpy strings as values

    • Added new functions to simplify building of matrices for ancillary datasets - build_ind_val_matrices()

  • Updates to pycroscopy.hdf_utils:

    • Functions updated to using the new Dimension objects

    • Added a few new functions such as write_book_keeping_attrs(), create_indexed_group(), create_results_group()

    • write_main_dataset() can now write empty datasets, use different prefixes for ancillary dataset names, etc.

    • Centralized the writing of book-keeping attributes to write_book_keeping_attrs()

    • generalized certain functions such as copy_attributes, write_simple_attributes() so that they can be applied to any HDF5 object

    • write_main_dataset() and create_empty_dataset() now validate the dtype correctly

    • print_tree() now prints cleaner versions of the tree, only Main datasets if requested.

    • write_book_keeping_attrs() now writes the operating system version and pycroscopy version in addition to the timestamp and machine ID

    • Region references functions such as copy_region_refs() now more robust

  • bug fixes to BE translation, visualization, plotting

Mar 27 2018:

  • Small changes to make pycroscopy available on Conda forge. Thanks to @carlodri !

  • pycroscopy.translators.NanonisTranslator added to translate Nanonis data files

Mar 2 2018:

  • Fixed decode error in pycroscopy.translators.IgorTranslator relevant for new versions of Asylum Research microscope software versions

Mar 3 2018: (on unity_dev and not on master)

  • pycroscopy.plot_utils.plot_map now accepts X and Y vectors

  • Lots of small bug fixes

  • More checks for more robust code in pycroscopy.core

  • New handy function - pycroscopy.hdf_utils.get_region() - directly returns the referenced data as a numpy array

  • Added two new examples on pycroscopy.io_utils and pycroscopy.dtype_utils

Feb 18 2018: (on unity_dev and not on master)

Massive restructuring and overhaul of code:

  • Renamed pycroscopy.ioHDF to pycroscopy.HDFWriter

  • Renamed pycroscopy.MicroDataset and pycroscopy.MicroDataGroup` to pycroscopy.VirtualDataset and pycroscopy.VirtualGroup

  • Data type manipulation functions moved out of pycroscopy.io_utils into pycroscopy.dtype_utils

  • Moved core foundational / science agnostic / engineering elements of pycroscopy into a new subpackage - pycroscopy.core. Rule for move - nothing in .core should import anything out of .core. This may be spun off as its own package at a later stage if deemed appropriate. Contents of pycroscopy.core:

    • core.io - HDFWriter, VirtualData, hdf_utils, write_utils, io_utils, dtype_utils, Translator, NumpyTranslator

    • core.processing - Process, parallel_compute()

    • core.viz - plot_utils, jupyter_utils

  • Started adding a lot of type and value checks for safer and more robust file reading/writing. Expect a lot of descriptive Exceptions that will help in identifying problems easier and sooner.

  • Implemented modular and standalone functions in pycroscopy.hdf_utils that form a (much simpler and more robust) feature-equivalent alternative to pycroscopy.HDFWriter + pyroscopy.VirtualData. pycroscopy.HDFWriter + pyroscopy.VirtualData will be phased out in the near future.

    • First implementation of what may be one of the most popular and important functions - pycroscopy.hdf_utils.write_main_dataset() -

      • Thoroughly checks and validates all inputs. Only if these pass,

      • Writes the a dataset containing the central data

      • Creates / reuses ancillary datasets

      • links Ancillary datasets to create a Main dataset

      • writes quantity and units attributes - now mandatory

      • Also writes any other attributes

    • Other notable functions include write_simple_attrs(), write_region_references, write_ind_val_dsets()

  • pycroscopy.NumpyTranslator now simply calls pycroscopy.hdf_utils.write_main_dataset()

    • pycroscopy.Translator.simple_write() removed. Translators can extend NumpyTranslator instead.

  • Added first batch of unit tests for modules in pycroscopy.core.

  • More robust pycroscopy.parallel_compute() via type checking

  • Added a new class called pycroscopy.AuxillaryDescriptor to describe Position and spectroscopic dimensions. All major functions like write_main_dataset() and write_ind_val_dsets() to use this descriptor

Jan 16 2018: (on unity_dev and not on master)

  • pycroscopy.processing.Cluster and pycroscopy.processing.Decomposition now extend pycroscopy.Process

  • More robust HDF functions for checking the existence of prior results groups.

  • Fixed important bugs for better python2 compatibility (HDF I/O, plotting, etc.)

  • More FFT signal filtering functions

  • Several bug fixes to pycroscopy.viz.plot_utils

  • Simplifications to the image cleaning and GIV notebooks to use the new capabilities of pycroscopy.processing.SVD, pycroscopy.processing.Cluster

Dec 7 2017:

  • New function (visualize()) added to pycroscopy.PycroDataset to facilitate interactive visualization of data in for any dataset (< 4 dimensions)

  • Significantly more customizable plotting functions in pycroscopy.plot_utils

  • Improved pycroscopy.Process that provides the framework for:

    • checking for prior instances of a process run on the same dataset with the same parameters

    • resuming an aborted process / computation

  • Reorganized doSVD() into a new Process called pycroscopy.processing.SVD to take advantage of above advancements.

    • The same changes will be rolled out to pycroscopy.processing.Cluster and pycroscopy.processing.Decomposition soon

Nov 17 2017:

  • Significant improvements and bug fixes to Bayesian Inference for G-mode IV.

Nov 11 2017:

  • New robust class for Bayesian Inference on G-mode IV data - pycroscopy.processing.GIVBayesian

  • Utilities for reading files from Nanois controllers

  • New robust class for FFT Signal Filtering on any data including G-mode - pycroscopy.processing.SignalFilter

  • FFT filtering rewritten and simplified to use objects

Oct 9 2017:

  • Added pycroscopy.PycroDataset - a class that simplifies handling, reshaping, and interpretation of Main datasets.

Sep 6 2017:

  • Added pycroscopy.Process - New class that provides a framework for data processing in Pycroscopy.

Sep 5 2017:

  • Improved the example on parallel computing

Aug 31 2017:

  • New plot function - single_img_cbar_plot() (now merged into plot_map()) for nicer 2D image plots with color-bars.

Aug 29 2017:

  • Improvements to Bayesian Inference on G-mode IV data including resistance compensation.