gnome.persist

Default behavior: Apply colander monkey patch by default

Put all the common Schema nodes in one namespace

Submodules

• gnome.persist.base_schema
• gnome.persist.extend_colander
• gnome.persist.monkey_patch_colander
• gnome.persist.save_load
• gnome.persist.schema_decorator
• gnome.persist.validators

Classes

Savable	Create a mixin for save/load options for saving and loading serialized
References	PyGnome objects like the PointWindMover contain other objects, eg Wind object.
TimeDelta	Add a type to serialize/deserialize timedelta objects
LocalDateTime	A type representing a Python datetime.datetime object.
ObjType	Base class for all schema types
LongLatBounds	Used to define bounds on a map
WorldPoint	Used to define reference points. 3D positions (long,lat,z)
ImageSize	Only contains 2D (long, lat) positions

Functions

load(saveloc[, fname, references])	read json from file and load the appropriate object
is_savezip_valid(savezip)	some basic checks on validity of zipfile. Primarily for checking save
convertible_to_seconds(node, value)	validate only datetime objects
ascending_datetime(node, values)	Check the datetime values in numpy structured array
no_duplicate_datetime(node, values)	Check for duplicate datetime values in numpy structured array like
positive(node, value)
now(node, kw)	Used by TimeseriesValueSchema - assume it defers the calculation of

Package Contents

class gnome.persist.Savable

Bases: object

Create a mixin for save/load options for saving and loading serialized gnome objects. Mix this in with the Serializable class so all gnome objects can save/load themselves

classmethod loads(json_data, saveloc=None, references=None)

loads object from json_data

• load json for references from files

• update paths of datafiles if needed

• deserialize json_data

• and create object with new_from_dict()

json_data: dict containing json data. It has been parsed through the

json.loads() command. The json will be valided here when it gets deserialized. Its references and datafile paths will be recreated here prior to calling new_from_dict()

Optional parameter

Parameters:

• saveloc – location of data files or .json files for objects stored as references. If object requires no datafiles and does not need to read references from a .json file in saveloc, then this can be None.

• references – references object - if this is called by the Model, it will pass a references object. It is not required.

Returns:

object constructed from json_data.

class gnome.persist.References

Bases: object

PyGnome objects like the PointWindMover contain other objects, eg Wind object. When persisting a Model, the Wind object is not created by the PointWindMover, it is merely referenced by the PointWindMover. When persisting a Model, the referenced objects are saved in their own file and a reference is stored for it. This class manages these references.

property files

get_reference(obj)

return key if obj already exists in references list else return None

reference(obj, name=None)

Get a unique reference to the object. By default this string is the filename in which the json for the object is stored If a reference to obj already exists, then it is returned

Parameters:

• obj – object for which a reference must be added

• name=None – add an object reference specified by ‘name’ for filename

retrieve(ref)

retrieve the object associated with the reference

gnome.persist.load(saveloc, fname='Model.json', references=None)

read json from file and load the appropriate object This is a general purpose load method that looks at the json[‘obj_type’] and invokes json[‘obj_type’].loads(json) method

Parameters:

• saveloc – path to zipfile that contains data files and json files. It could also be a directory containing files - keep original support for location files.

• fname – .json file to load. Default is ‘Model.json’. zipfile/dir must contain ‘fname’

• references=None – References object that keeps track of objects in a dict as they are constructed, using the filename as the key

Returns:

object constructed from the json

Note

Function first assumes saveloc is a directory and looks for saveloc/fname. If this fails, it checks if saveloc is a zipfile. If this fails, it checks if saveloc is a file and loads this assuming its json for a gnome object. If none of these work, it just returns None.

gnome.persist.is_savezip_valid(savezip)

some basic checks on validity of zipfile. Primarily for checking save zipfiles loaded from the Web. Following are the types of errors it checks:

Returns:

True if zip is valid, False otherwise

1. Failed to open zipfile

2. CRC failed for a file in the archive - rejecting zip

3. Found a *.json with size > _max_json_filesize - rejecting

4. Reject - found a file with:

   uncompressed_size/compressed_size > _max_compress_ratio.

5. Found a file in archive that has path outside of saveloc - rejecting

   rejects zipfile if it contains an archive with ‘..’

Note

can change _max_json_filesize, _max_compress_ratio if required.

class gnome.persist.TimeDelta

Bases: colander.Float

Add a type to serialize/deserialize timedelta objects

serialize(node, appstruct)

deserialize(*args, **kwargs)

class gnome.persist.LocalDateTime(*args, **kwargs)

Bases: colander.DateTime

A type representing a Python datetime.datetime object.

This type serializes python datetime.datetime objects to a ISO8601 string format. The format includes the date, the time, and the timezone of the datetime.

The constructor accepts an argument named default_tzinfo which should be a Python tzinfo object. If default_tzinfo is not specified the default tzinfo will be equivalent to UTC (Zulu time). The default_tzinfo tzinfo object is used to convert ‘naive’ datetimes to a timezone-aware representation during serialization. If default_tzinfo is explicitly set to None then no default tzinfo will be applied to naive datetimes.

You can adjust the error message reported by this class by changing its err_template attribute in a subclass on an instance of this class. By default, the err_template attribute is the string Invalid date. This string is used as the interpolation subject of a dictionary composed of val and err. val and err are the unvalidatable value and the exception caused trying to convert the value, respectively. These may be used in an overridden err_template as ${val} and ${err} respectively as necessary, e.g. _('${val} cannot be parsed as an iso8601 date: ${err}').

For convenience, this type is also willing to coerce datetime.date objects to a DateTime ISO string representation during serialization. It does so by using midnight of the day as the time, and uses the default_tzinfo to give the serialization a timezone.

Likewise, for convenience, during deserialization, this type will convert YYYY-MM-DD ISO8601 values to a datetime object. It does so by using midnight of the day as the time, and uses the default_tzinfo to give the serialization a timezone.

If the colander.null value is passed to the serialize method of this class, the colander.null value will be returned.

The subnodes of the colander.SchemaNode that wraps this type are ignored.

strip_timezone(_datetime)

serialize(node, appstruct)

Serialize a DateTime object

returns an iso formatted string

deserialize(node, cstruct)

gnome.persist.convertible_to_seconds(node, value)

validate only datetime objects

gnome.persist.ascending_datetime(node, values)

Check the datetime values in numpy structured array (like datetime_value_2d) are in ascending order

gnome.persist.no_duplicate_datetime(node, values)

Check for duplicate datetime values in numpy structured array like datetime_value_2d Reject values if it contains duplicates.

gnome.persist.positive(node, value)

class gnome.persist.ObjType(unknown='ignore')

Bases: colander.SchemaType

Base class for all schema types

unknown = 'ignore'

cstruct_children(node, cstruct)

serialize(node, appstruct, options=None)

deserialize(node, cstruct, refs)

flatten(node, appstruct, prefix='', listitem=False)

unflatten(node, paths, fstruct)

set_value(node, appstruct, path, value)

get_value(node, appstruct, path)

save(node, appstruct, zipfile_, refs)

load(node, cstruct, saveloc, refs)

hydrate_json(node, cstruct, saveloc, refs)

class gnome.persist.LongLatBounds(*args, **kw)

Bases: colander.SequenceSchema

Used to define bounds on a map

bounds

class gnome.persist.WorldPoint(*arg, **kw)

Bases: LongLat

Used to define reference points. 3D positions (long,lat,z)

z

class gnome.persist.ImageSize(*arg, **kw)

Bases: colander.TupleSchema

Only contains 2D (long, lat) positions

width

height

gnome.persist.now(node, kw)

Used by TimeseriesValueSchema - assume it defers the calculation of

datetime.datetime.now to when it is called in Schema