hio.base.during =============== .. py:module:: hio.base.during .. autoapi-nested-parse:: hio.help.during module Support for Durable storage using LMDB Module Contents --------------- .. py:class:: Duror(readonly=False, **kwa) Bases: :py:obj:`hio.base.filing.Filer` Manages an an LMDB database directory and environment. Inherited Class Attributes: (see Filer) HeadDirPath (str): default abs dir path head such as "/usr/local/var" TailDirPath (str): default rel dir path tail when using head CleanTailDirPath (str): default rel dir path tail when creating clean AltHeadDirPath (str): default alt dir path head such as "~" (fallback). AltTailDirPath (str): default alt rel dir path tail (alt head). AltCleanTailDirPath (str): default alt rel path tail when creating clean TempHeadDir (str): default temp abs dir path head such as "/tmp" TempPrefix (str): default rel dir path prefix when using temp head TempSuffix (str): default rel dir path suffix when using temp head and tail Perm (int): explicit default octal perms such as 0o1700 Mode (str): open mode such as "r+" Fext (str): default file extension such as "text" for "fname.text" Inherited Attributes: (see Filer) name (str): unique path component at end of path for directory or file base (str): another unique path component inserted before name temp (bool): True means use TempHeadDir in /tmp directory headDirPath (str): head directory path path (str | None): full directory or file path once created else None perm (int): octal OS permissions for path directory and/or file filed (bool): True means .path ends in file. False means .path ends in directory extensioned (bool): When not filed, True means ensure .path ends with fext. mode (str): file open mode if filed fext (str): file extension if filed file (File | None): File instance when filed and created. opened (bool): True means directory created and if filed then file is opened. False otherwise .. attribute:: env LMDB main (super) database environment :type: lmdb.env .. attribute:: readonly True means open LMDB env as readonly :type: bool Properties: version. File/Directory Creation Mode Notes:: .Perm provides default restricted access permissions to directory and/or files stat.S_ISVTX | stat.S_IRUSR | stat.S_IWUSR | stat.S_IXUSR 0o1700==960 stat.S_ISVTX is Sticky bit. When this bit is set on a directory it means that a file in that directory can be renamed or deleted only by the owner of the file, by the owner of the directory, or by a privileged process. When this bit is set on a file it means nothing stat.S_IRUSR Owner has read permission. stat.S_IWUSR Owner has write permission. stat.S_IXUSR Owner has execute permission. .. py:method:: reopen(readonly=False, **kwa) Open if closed or close and reopen if opened or create and open if not if not preexistent, directory path for lmdb at .path and then Open lmdb and assign to .env Inherited Parameters: (see Filer) temp (bool): assign to .temp; temporary means clear on close, persistent means do not clear. headDirPath (str): optional head directory pathname for main database. Default .HeadDirPath. perm (int): optional numeric os dir permissions for database directory and files. Default .Perm. clear (bool): True means remove directory upon close; False means keep directory. reuse (bool): True means reuse self.path if already exists; False means remake self.path. clean (bool): True means path uses clean tail variant; False means normal tail variant. mode (str): file open mode when .filed fext (str): File extension when .filed :param readonly: True means open database in readonly mode; False means read/write mode. :type readonly: bool .. py:property:: version Return the version of database stored in __version__ key. This value is read through cached in memory :returns: semver string the version of the database or None if not set in the database :rtype: version (str) .. py:method:: close(clear=False) Close lmdb at .env if clear or .temp then remove lmdb directory at .path :param clear: Otherwise do not clear after close :type clear: bool .. py:method:: getVer() Returns the value of the the semver formatted version in the __version__ key in this database :returns: semver formatted version of the database :rtype: str .. py:method:: setVer(version) Set the version of the database in the __version__ key :param version: The new semver formatted version str of the database :type version: str .. py:method:: putVal(sdb, key, val) Write serialized bytes val to location key in db (subdb) Does not overwrite. :returns: True if val successfully written; False if val at key already exists :rtype: result (bool) :param db: opened named subdb with dupsort=False :type db: lmdb._Database :param key: within subdb's keyspace :type key: bytes :param val: to be written at key :type val: bytes .. py:method:: pinVal(sdb, key, val) Write serialized bytes val to location key in db Overwrites existing val if any :returns: True if val successfully written; False otherwise :rtype: result (bool) :param db: opened named subdb with dupsort=False :type db: lmdb._Database :param key: within subdb's keyspace :type key: bytes :param val: to be written at key :type val: bytes .. py:method:: getVal(sdb, key) Get val at key in db :returns: None if no entry at key :rtype: val (bytes) :param db: opened named subdb with dupsort=False :type db: lmdb._Database :param key: within subdb's keyspace :type key: bytes .. py:method:: remVal(sdb, key) Removes value at key in db. :returns: True If key exists in database False otherwise :rtype: result (bool) :param db: opened named subdb with dupsort=False :type db: lmdb._Database :param key: within subdb's keyspace :type key: bytes .. py:method:: cntVals(sdb) Counts entries in subdb db :returns: number of entries in subdb db, or zero otherwise :rtype: count (int) :param db: opened named subdb with dupsort=False :type db: lmdb._Database .. py:method:: getTopItemIter(sdb, top=b'') Iterates over branch of subdb db rooted at top key :returns: of (full key, val) tuples over a branch of the db given by top key where: full key is full database key for val not truncated top key :rtype: items (Iterator) Raises StopIteration Error when empty. :param db: instance of named sub db with dupsort==False :type db: lmdb._Database :param top: truncated top key, a key space prefix to get all the items from multiple branches of the key space. If top key is empty then gets all items in database. In Python str.startswith('') always returns True so if branch key is empty string it matches all keys in db with startswith. :type top: bytes .. py:method:: remTopVals(sdb, top=b'') Removes all values in branch of db given top key. :returns: True if values were deleted at key. False otherwise if no values at key :rtype: result (bool) :param db: instance of named sub db with dupsort==False :type db: lmdb._Database :param top: truncated top key, a key space prefix to get all the items from multiple branches of the key space. If top key is empty then deletes all items in database :type top: bytes Raises StopIteration Error when empty. .. py:method:: suffix(key: bytes | str | memoryview, ion: int, *, sep: bytes | str = b'.') :staticmethod: Return iokey after concatenating the hex ordinal suffix to `key` using `sep`. :param key: apparent effective database key (unsuffixed) :type key: bytes|str|memoryview :param ion: insertion ordered ordinal numberfor set of vals :type ion: int) :param sep: separator character(s) for concatenating suffix :type sep: bytes .. py:method:: unsuffix(iokey: bytes | str | memoryview, *, sep: bytes | str = b'.') :staticmethod: Return `(key, ion)` from splitting `iokey` at the rightmost `sep`. :param iokey: actual database key suffixed :type iokey: bytes|str|memoryview :param sep: separator character(s) for splitting suffix :type sep: bytes .. py:method:: getIoValFirst(sdb, key, *, ion=0, sep=b'.') Gets first of the insertion ordered set of values at key, None if no entry. Returns the first value at the apparent effective key whose iokey is greater than or equal to the iokey from `ion`, or None if no entry. :param db: instance of named sub db with dupsort==False :type db: lmdb._Database :param key: Apparent effective key :type key: bytes :param ion: starting ordinal value, default 0 :type ion: int .. py:method:: getIoValLast(sdb, key, *, sep=b'.') Gets last value (last in) of insertion ordered set values at key Returns the last value at the apparent effective key, or None if no entry. Uses hidden ordinal key suffix for insertion ordering. The suffix is appended and stripped transparently. :param db: instance of named sub db with dupsort==False :type db: lmdb._Database :param key: Apparent effective key :type key: bytes .. py:method:: getIoVals(sdb, key, *, ion=0, sep=b'.') Gets list of all the insertion ordered set of values at key :returns: insertion-ordered values at same apparent effective key; hidden ordinal key suffix is used for insertion ordering and stripped transparently. :rtype: vals (list[bytes]) :param db: instance of named sub db with dupsort==False :type db: lmdb._Database :param key: Apparent effective key :type key: bytes :param ion: starting ordinal value, default 0 :type ion: int .. py:method:: getIoValsIter(sdb, key, *, ion=0, sep=b'.') Gets Iterator of all the insertion ordered set of values at key :returns: iterator over insertion ordered set of values at same apparent effective key. Uses hidden ordinal key suffix for insertion ordering. The suffix is appended and stripped transparently. :rtype: ioset (Iterator) Raises StopIteration Error when empty. :param db: instance of named sub db with dupsort==False :type db: lmdb._Database :param key: Apparent effective key :type key: bytes :param ion: starting ordinal value, default 0 :type ion: int .. py:method:: popIoVal(sdb, key, *, ion=0, sep=b'.') Pops first of the insertion ordered set of values at key. None if no entry. Pop deletes the returned entry. :returns: first val at same apparent effective key whose iokey >= iokey from ion Uses hidden ordinal key suffix for insertion ordering. The suffix is appended and stripped transparently. None if no entry. :rtype: val (bytes|None) :param db: instance of named sub db with dupsort==False :type db: lmdb._Database :param key: Apparent effective key :type key: bytes :param ion: starting ordinal value, default 0 :type ion: int .. py:method:: remIoVals(sdb, key, *, sep=b'.') Deletes all values at apparent effective key. Uses hidden ordinal key suffix for insertion ordering. The suffix is appended and stripped transparently. :returns: True if at least one value was deleted at key. False otherwise, if no values at key :rtype: result (bool) :param db: instance of named sub db with dupsort==False :type db: lmdb._Database :param key: Apparent effective key :type key: bytes .. py:method:: cntIoVals(sdb, key, *, sep=b'.') Count all values with the same apparent effective key. Uses hidden ordinal key suffix for insertion ordering. The suffix is appended and stripped transparently. :returns: count values in set at apparent effective key :rtype: count (int) :param db: instance of named sub db with dupsort==False :type db: lmdb._Database :param key: Apparent effective key :type key: bytes .. py:method:: getTopIoItemIter(sdb, top=b'', *, sep=b'.') Gets Iterator of all values in branch rooted at top key The suffix is appended and stripped transparently. :returns: iterator of tuples (key, val) where key is apparent key with hidden insertion ordering suffixe removed from effective key. Iterates over top branch of insertion ordered set values where each effective key has trailing hidden suffix of serialization of insertion ordering ordinal. Uses hidden ordinal key suffix for insertion ordering. :rtype: items (Iterator[(key,val)]) Raises StopIteration Error when empty. :param db: instance of named sub db with dupsort==False :type db: lmdb._Database :param top: top key in db. When top is empty then every item in db. :type top: bytes :param sep: sep character for attached io suffix :type sep: bytes .. py:method:: addIoVal(sdb, key, val, *, sep=b'.') Add val at end (append) of insertion ordered set of values all with the same apparent effective key. Does not dedup val. Uses hidden ordinal key suffix for insertion ordering. The suffix is appended and stripped transparently. :returns: True is added to set. False if already in set. :rtype: result (bool) :param db: instance of named sub db with dupsort==False :type db: lmdb._Database :param key: Apparent effective key :type key: bytes :param val: serialized value to add :type val: bytes .. py:method:: putIoVals(sdb, key, vals, *, sep=b'.') Adds at end (append) each val in vals to insertion ordered list of values all with the same apparent effective key for each val. Does not dedup added vals. Uses hidden ordinal key suffix for insertion ordering. The suffix is appended and stripped transparently. :returns: True if added to set. False if already in set. :rtype: result (bool) :param db: instance of named sub db with dupsort==False :type db: lmdb._Database :param key: Apparent effective key :type key: bytes :param vals: serialized values to add to set of vals at key :type vals: Iterable .. py:method:: pinIoVals(sdb, key, vals, *, sep=b'.') Erase all vals at key and then add vals as insertion ordered set of values all with the same apparent effective key. Does not dedup vals. Uses hidden ordinal key suffix for insertion ordering. The suffix is appended and stripped transparently. :returns: True is added to set. :rtype: result (bool) :param db: instance of named sub db with dupsort==False :type db: lmdb._Database :param key: Apparent effective key :type key: bytes :param vals: serialized values to add to set of vals at key :type vals: abc.Iterable .. py:method:: addIoSetVal(sdb, key, val, *, sep=b'.') Add val idempotently to insertion ordered set of values all with the same apparent effective key if val not already in set of vals at key. Dedups the added val. Uses hidden ordinal key suffix for insertion ordering. The suffix is appended and stripped transparently. :returns: True is added to set. False if already in set. :rtype: result (bool) :param db: instance of named sub db with dupsort==False :type db: lmdb._Database :param key: Apparent effective key :type key: bytes :param val: serialized value to add :type val: bytes .. py:method:: putIoSetVals(sdb, key, vals, *, sep=b'.') Adds idempotently each val in vals to insertion ordered set of values all with the same apparent effective key for each val that is not already in set of vals at key. Dedups the put vals. Uses hidden ordinal key suffix for insertion ordering. The suffix is appended and stripped transparently. :returns: True if added to set. False if already in set. :rtype: result (bool) :param db: instance of named sub db with dupsort==False :type db: lmdb._Database :param key: Apparent effective key :type key: bytes :param vals: serialized values to add to set of vals at key :type vals: Iterable .. py:method:: pinIoSetVals(sdb, key, vals, *, sep=b'.') Erase all vals at key and then add unique vals as insertion ordered set of values all with the same apparent effective key. Dedups the set vals. Uses hidden ordinal key suffix for insertion ordering. The suffix is appended and stripped transparently. :returns: True is added to set. :rtype: result (bool) :param db: instance of named sub db with dupsort==False :type db: lmdb._Database :param key: Apparent effective key :type key: bytes :param vals: serialized values to add to set of vals at key :type vals: abc.Iterable .. py:method:: remIoSetVal(sdb, key, val, *, sep=b'.') Removes (delete) matching val at apparent effective key if exists. Uses hidden ordinal key suffix for insertion ordering. The suffix is appended and stripped transparently. Because the insertion order of val is not provided must perform a linear search over set of values. Another problem is that vals may get added and deleted in any order so the max suffix ion may creep up over time. The suffix ordinal max > 2**16 is an impossibly large number, however, so the suffix will not max out practically.But its not the most elegant solution. In some cases a better approach would be to use getIoSetItemsIter which returns the actual iokey not the apparent effetive key so can delete using the iokey without searching for the value. This is most applicable when processing escrows where all the escrowed items are processed linearly and one needs to delete some of them in stride with their processing. :returns: True if val was deleted at key. False otherwise if val not found at key :rtype: result (bool) :param db: instance of named sub db with dupsort==False :type db: lmdb._Database :param key: Apparent effective key :type key: bytes :param val: value to delete :type val: bytes .. py:function:: openDuror(*, cls=None, name='test', temp=True, **kwa) Context manager wrapper for Duror instances. Defaults to temporary databases. Context 'with' statements call .close on exit of 'with' block :param cls: Class (subclass) defaults to Duror when None :type cls: Type[Duror] :param name: of Duror dirPath so can have multiple holders that each use different dirPath name :type name: str :param temp: True means open in temporary directory, clear on close Otherwise open in persistent directory, do not clear on close :type temp: bool Usage:: with openDuror(name="gen1") as duror1: duror1.env with openDuror(name="gen2", cls=Baser) as duror2: duror2.env .. py:class:: DurorDoer(duror, **kwa) Bases: :py:obj:`hio.base.doing.Doer` Duror Doer .. attribute:: done completion state: True means completed Otherwise incomplete. Incompletion maybe due to close or abort. :type: bool .. attribute:: duror instance :type: Duror Inherited Properties: tyme (float): relative cycle time of associated Tymist .tyme obtained via injected .tymth function wrapper closure. tymth (func): closure returned by Tymist .tymeth() method. When .tymth is called it returns associated Tymist .tyme. .tymth provides injected dependency on Tymist tyme base. tock (float)): desired time in seconds between runs or until next run, non negative, zero means run asap .. py:method:: enter(*, temp=None) Do 'enter' context actions. Override in subclass. Not a generator method. Set up resources. Comparable to context manager enter. :param temp: True means use temporary file resources if any None means ignore parameter value. Use self.temp :type temp: bool | None Inject temp or self.temp into file resources here if any .. py:method:: exit() Do 'exit' context actions. Override in subclass. Not a generator method. Clean up resources. Comparable to context manager exit. Called by finally after normal return, close, or abort. After .exit() do returns resulting in StopIteration. .. py:class:: SuberBase(db: Duror, *, subkey: str = 'docs.', dupsort: bool = False, sep: str = None, verify: bool = False, **kwa) Base class for Sub DBs of Duror Provides common methods for subclasses Do not instantiate but use a subclass Class Attribues: Sep (str): default separator to convert keys iterator to key bytes for db key .. attribute:: db base LMDB db :type: dbing.LMDBer .. attribute:: sdb instance of lmdb named sub db for this Suber :type: lmdb._Database .. attribute:: sep separator for combining keys tuple of strs into key bytes :type: str .. attribute:: verify True means reverify when ._des from db when applicable False means do not reverify. Default False :type: bool .. py:method:: trim(keys: str | bytes | memoryview | Iterable = b'', *, topive=False) Removes all entries whose keys startswith keys. Enables removal of whole branches of db key space. To ensure that proper separation of a branch include empty string as last key in keys. For example ("a","") deletes 'a.1'and 'a.2' but not 'ab' :param keys: of key parts that may be a truncation of a full keys tuple in in order to address all the items from multiple branches of the key space. If keys is empty then trims all items in database. Either append "" to end of keys Iterable to ensure get properly separated top branch key or use top=True. :type keys: Iterable[str | bytes | memoryview] :param topive: True means treat as partial key tuple from top branch of key space given by partial keys. Resultant key ends in .sep character. False means treat as full branch in key space. Resultant key does not end in .sep character. When last item in keys is empty str then will treat as partial ending in sep regardless of top value :type topive: bool :returns: True if val at key exists so delete successful. False otherwise :rtype: result (bool) .. py:method:: getFullItemIter(keys: str | bytes | memoryview | Iterable[str | bytes] = '', *, topive=False) Iterator over items in .db that returns full items with subclass specific special hidden parts shown for debugging or testing. :returns: (key, val) tuples of each item over the all the items in subdb whose key startswith key made from keys. Keys may be keyspace prefix to return branches of key space. When keys is empty then returns all items in subdb. This is meant to return full parts of items in both keyspace and valuespace which may be useful in debugging or testing. :rtype: items (Iterator[tuple[key,val]]) :param keys: of key parts that may be a truncation of a full keys tuple in in order to address all the items from multiple branches of the key space. If keys is empty then gets all items in database. Either append "" to end of keys Iterable to ensure get properly separated top branch key or use top=True. In Python str.startswith('') always returns True so if branch key is empty string it matches all keys in db with startswith. :type keys: str|bytes|memoryview|Iterable[str | bytes | memoryview] :param topive: True means treat as partial key tuple from top branch of key space given by partial keys. Resultant key ends in .sep character. False means treat as full branch in key space. Resultant key does not end in .sep character. When last item in keys is empty str then will treat as partial ending in sep regardless of top value :type topive: bool .. py:method:: getItemIter(keys: str | bytes | memoryview | Iterable = '', *, topive=False) Iterator over items in .db subclasses that do special hidden transforms on either the keyspace or valuespace should override this method to hide hidden parts from the returned items. For example, adding either a hidden key space suffix or hidden val space proem to ensure insertion order. Use getFullItemIter instead to return full items with hidden parts shown for debugging or testing. :returns: (key, val) tuples of each item over the all the items in subdb whose key startswith key made from keys. Keys may be keyspace prefix to return branches of key space. When keys is empty then returns all items in subdb :rtype: items (Iterator[tuple[key,val]]) :param keys: of key parts that may be a truncation of a full keys tuple in in order to address all the items from multiple branches of the key space. If keys is empty then gets all items in database. Either append "" to end of keys Iterable to ensure get properly separated top branch key or use top=True. In Python str.startswith('') always returns True so if branch key is empty string it matches all keys in db with startswith. :type keys: str|bytes|memoryview|Iterable[str|bytes|memoryview] :param topive: True means treat as partial key tuple from top branch of key space given by partial keys. Resultant key ends in .sep character. False means treat as full branch in key space. Resultant key does not end in .sep character. When last item in keys is empty str then will treat as partial ending in sep regardless of top value :type topive: bool .. py:method:: cntAll() Counts all entries in subdb self.sdb :returns: all entries in .sdb :rtype: count (int) .. py:class:: Suber(db: Duror, *, subkey: str = 'docs.', dupsort: bool = False, **kwa) Bases: :py:obj:`SuberBase` Subclass of `SuberBase` without LMDB duplicates. .. py:method:: put(keys: str | bytes | memoryview | Iterable[str | bytes | memoryview], val: bytes | str | memoryview) Puts val at key made from keys. Does not overwrite :param keys: of key strs to be combined in order to form key :type keys: str|bytes|memoryview|Iterable[str|bytes|memoryview] :param val: value from ._ser :type val: bytes|str|memoryview :returns: True If successful, False otherwise, such as key already in database. :rtype: result (bool) .. py:method:: pin(keys: str | bytes | memoryview | Iterable[str | bytes | memoryview], val: bytes | str | memoryview) Pins (sets) val at key made from keys. Overwrites. :param keys: of key strs to be combined in order to form key :type keys: str|bytes|memoryview|Iterable[str|bytes|memoryview] :param val: value from ._ser :type val: bytes|str|memoryview :returns: True If successful. False otherwise. :rtype: result (bool) .. py:method:: get(keys: str | bytes | memoryview | Iterable[str | bytes | memoryview]) Gets val at keys :param keys: of key strs to be combined in order to form key :type keys: str|bytes|memoryview|Iterable[str|bytes|memoryview] :returns: decoded as utf-8 or whatever ._des provides None if no entry at keys :rtype: data (str) Usage:: Use walrus operator to catch and raise missing entry if (data := mydb.get(keys)) is None: raise ExceptionHere use data here .. py:method:: rem(keys: str | bytes | memoryview | Iterable[str | bytes | memoryview]) Removes entry at keys :param keys: of key strs to be combined in order to form key :type keys: str|bytes|memoryview|Iterable[str|bytes|memoryview] :returns: True if key exists so delete successful. False otherwise :rtype: result (bool) .. py:class:: IoSuber(db: dbing.LMDBer, *, subkey: str = 'docs.', dupsort: bool = False, ionsep: str = None, **kwa) Bases: :py:obj:`SuberBase` Insertion-ordered Suber that supports a durable queue of values per key. Uses a hidden ordinal suffix to preserve insertion order. .. py:method:: getFirst(keys: str | bytes | memoryview | Iterable) Gets first val inserted at effecive key made from keys and hidden ordinal suffix. :param keys: of key strs to be combined in order to form key :type keys: Iterable :returns: value str, None if no entry at keys :rtype: val (str|None) .. py:method:: getLast(keys: str | bytes | memoryview | Iterable) Gets last val inserted at effecive key made from keys and hidden ordinal suffix. :param keys: of key strs to be combined in order to form key :type keys: Iterable :returns: value str, None if no entry at keys :rtype: val (str|None) .. py:method:: get(keys: str | bytes | memoryview | Iterable) Gets vals set list at key made from effective keys :param keys: of key strs to be combined in order to form key :type keys: Iterable :returns: each item in list is str empty list if no entry at keys :rtype: vals (Iterable) .. py:method:: getIter(keys: str | bytes | memoryview | Iterable) Gets vals iterator at effecive key made from keys and hidden ordinal suffix. All vals in set of vals that share same effecive key are retrieved in insertion order. :param keys: of key strs to be combined in order to form key :type keys: Iterable :returns: str values. Raises StopIteration when done :rtype: vals (Iterator) .. py:method:: pop(keys: str | bytes | memoryview | Iterable) Pops first val if any inserted at effecive key made from keys and hidden ordinal suffix. Pop returns and deletes value if any. :param keys: of key strs to be combined in order to form key :type keys: Iterable :returns: value str, None if no entry at keys :rtype: val (str|None) .. py:method:: add(keys: str | bytes | memoryview | Iterable, val: str | bytes | memoryview) Add val idempotently to vals at effective key made from keys and hidden ordinal suffix. Idempotent means that added value is not already in set of vals at key. Does not overwrite or add same value at same key more than once. :param keys: of key parts to be combined in order to form key :type keys: str | bytes | memoryview | Iterable :param val: serialization :type val: str | bytes | memoryview :returns: True means value added , False otherwise. :rtype: result (bool) .. py:method:: put(keys: str | bytes | memoryview | Iterable, vals: str | bytes | memoryview | Iterable) Puts all vals each with an effective key made from keys and hidden ordinal suffix that are not already in set of vals at key. Does not overwrite existing vals. Does not add val if already same val in set. :param keys: of key parts to be combined in order to form key :type keys: str | bytes | memoryview | Iterable :param vals: of str serializations :type vals: str | bytes | memoryview | Iterable :returns: True If successful, False otherwise. :rtype: result (bool) .. py:method:: pin(keys: str | bytes | memoryview | Iterable, vals: str | bytes | memoryview | Iterable) Pins (sets) vals at effective key made from keys and hidden ordinal suffix. Overwrites. Removes all pre-existing vals that share same effective keys and replaces them with vals :param keys: of key strs to be combined in order to form key :type keys: Iterable :param vals: str serializations :type vals: Iterable :returns: True If successful, False otherwise. :rtype: result (bool) .. py:method:: rem(keys: str | bytes | memoryview | Iterable) Removes entry at effective key made from keys and hidden ordinal suffix that matches val if any. Otherwise deletes all values at effective key. :param keys: of key strs to be combined in order to form key :type keys: Iterable :param val: value at key to delete. Subclass ._ser method may accept different value types if val is empty then remove all values at key :type val: str :returns: True if effective key with val exists so rem successful. False otherwise :rtype: result (bool) .. py:method:: cnt(keys: str | bytes | memoryview | Iterable) Return count of values at effective key made from keys and hidden ordinal suffix. Zero otherwise :param keys: of key strs to be combined in order to form key :type keys: Iterable .. py:method:: getItemIter(keys: str | bytes | memoryview | Iterable = '', *, topive=False) Return iterator over all the items in top branch defined by keys where keys may be truncation of full branch. :returns: of (key, val) tuples over the all the items in subdb whose effective key startswith key made from keys. Keys may be keyspace prefix in order to return branches of key space. When keys is empty then returns all items in subdb. Returned key in each item has ordinal suffix removed. :rtype: items (Iterator) :param keys: tuple of bytes or strs that may be a truncation of a full keys tuple in in order to address all the items from multiple branches of the key space. If keys is empty then gets all items in database. Either append "" to end of keys Iterable to ensure get properly separated top branch key or use top=True. In Python str.startswith('') always returns True so if branch key is empty string it matches all keys in db with startswith. :type keys: Iterable :param topive: True means treat as partial key tuple from top branch of key space given by partial keys. Resultant key ends in .sep character. False means treat as full branch in key space. Resultant key does not end in .sep character. When last item in keys is empty str then will treat as partial ending in sep regardless of top value :type topive: bool .. py:class:: IoSetSuber(db: dbing.LMDBer, *, subkey: str = 'docs.', dupsort: bool = False, ionsep: str = None, **kwa) Bases: :py:obj:`SuberBase` Insertion-ordered Suber that supports a set of distinct values per key. Uses a hidden ordinal suffix to preserve insertion order. .. py:method:: getFirst(keys: str | bytes | memoryview | Iterable) Gets first val inserted at effecive key made from keys and hidden ordinal suffix. :param keys: of key strs to be combined in order to form key :type keys: Iterable :returns: value str, None if no entry at keys :rtype: val (str|None) .. py:method:: getLast(keys: str | bytes | memoryview | Iterable) Gets last val inserted at effecive key made from keys and hidden ordinal suffix. :param keys: of key strs to be combined in order to form key :type keys: Iterable :returns: value str, None if no entry at keys :rtype: val (str|None) .. py:method:: get(keys: str | bytes | memoryview | Iterable) Gets vals set list at key made from effective keys :param keys: of key strs to be combined in order to form key :type keys: Iterable :returns: each item in list is str empty list if no entry at keys :rtype: vals (Iterable) .. py:method:: getIter(keys: str | bytes | memoryview | Iterable) Gets vals iterator at effecive key made from keys and hidden ordinal suffix. All vals in set of vals that share same effecive key are retrieved in insertion order. :param keys: of key strs to be combined in order to form key :type keys: Iterable :returns: str values. Raises StopIteration when done :rtype: vals (Iterator) .. py:method:: pop(keys: str | bytes | memoryview | Iterable) Pops first val if any inserted at effecive key made from keys and hidden ordinal suffix. Pop returns and deletes value if any. :param keys: of key strs to be combined in order to form key :type keys: Iterable :returns: value str, None if no entry at keys :rtype: val (str|None) .. py:method:: add(keys: str | bytes | memoryview | Iterable, val: str | bytes | memoryview) Add val idempotently to vals at effective key made from keys and hidden ordinal suffix. Idempotent means that added value is not already in set of vals at key. Does not overwrite or add same value at same key more than once. :param keys: of key parts to be combined in order to form key :type keys: str | bytes | memoryview | Iterable :param val: serialization :type val: str | bytes | memoryview :returns: True means unique value added among duplications, False means duplicate of same value already exists. :rtype: result (bool) .. py:method:: put(keys: str | bytes | memoryview | Iterable, vals: str | bytes | memoryview | Iterable) Puts all vals each with an effective key made from keys and hidden ordinal suffix that are not already in set of vals at key. Does not overwrite existing vals. Does not add val if already same val in set. :param keys: of key parts to be combined in order to form key :type keys: str | bytes | memoryview | Iterable :param vals: of str serializations :type vals: str | bytes | memoryview | Iterable :returns: True If successful, False otherwise. :rtype: result (bool) .. py:method:: pin(keys: str | bytes | memoryview | Iterable, vals: str | bytes | memoryview | Iterable) Pins (sets) vals at effective key made from keys and hidden ordinal suffix. Overwrites. Removes all pre-existing vals that share same effective keys and replaces them with vals :param keys: of key strs to be combined in order to form key :type keys: Iterable :param vals: str serializations :type vals: Iterable :returns: True If successful, False otherwise. :rtype: result (bool) .. py:method:: rem(keys: str | bytes | memoryview | Iterable, val: str | bytes | memoryview = b'') Removes entry at effective key made from keys and hidden ordinal suffix that matches val if any. Otherwise deletes all values at effective key. :param keys: of key strs to be combined in order to form key :type keys: Iterable :param val: value at key to delete. Subclass ._ser method may accept different value types if val is empty then remove all values at key :type val: str :returns: True if deletion succeeds; False otherwise. If val is provided, remove matching value at effective key. If val is empty, remove all values at effective key. :rtype: result (bool) .. py:method:: cnt(keys: str | bytes | memoryview | Iterable) Return count of values at effective key made from keys and hidden ordinal suffix. Zero otherwise :param keys: of key strs to be combined in order to form key :type keys: Iterable .. py:method:: getItemIter(keys: str | bytes | memoryview | Iterable = '', *, topive=False) Return iterator over all the items in top branch defined by keys where keys may be truncation of full branch. :returns: of (key, val) tuples over the all the items in subdb whose effective key startswith key made from keys. Keys may be keyspace prefix in order to return branches of key space. When keys is empty then returns all items in subdb. Returned key in each item has ordinal suffix removed. :rtype: items (Iterator) :param keys: tuple of bytes or strs that may be a truncation of a full keys tuple in in order to address all the items from multiple branches of the key space. If keys is empty then gets all items in database. Either append "" to end of keys Iterable to ensure get properly separated top branch key or use top=True. In Python str.startswith('') always returns True so if branch key is empty string it matches all keys in db with startswith. :type keys: Iterable :param topive: True means treat as partial key tuple from top branch of key space given by partial keys. Resultant key ends in .sep character. False means treat as full branch in key space. Resultant key does not end in .sep character. When last item in keys is empty str then will treat as partial ending in sep regardless of top value :type topive: bool .. py:class:: DomSuberBase(db: Duror, *, subkey: str = 'bags.', sep='_', prosep=None, **kwa) Bases: :py:obj:`SuberBase` Suber that serializes `RegDom` subclasses. Forces `.sep` to `_` and uses a class-name proem (LF) when serializing. .. py:class:: DomSuber(db: Duror, *, subkey: str = 'cans.', sep='_', prosep='\n', **kwa) Bases: :py:obj:`DomSuberBase`, :py:obj:`Suber` Suber that serializes `CanDom` subclasses with LF proem. .. py:class:: DomIoSuber(db: Duror, *, subkey: str = 'dsqs.', sep='_', prosep='\n', ionsep='.', **kwa) Bases: :py:obj:`DomSuberBase`, :py:obj:`IoSuber` Insertion-ordered Suber that serializes `RegDom` subclasses with LF proem. .. py:class:: DomIoSetSuber(db: Duror, *, subkey: str = 'dsqs.', sep='_', prosep='\n', ionsep='.', **kwa) Bases: :py:obj:`DomSuberBase`, :py:obj:`IoSetSuber` Insertion-ordered set Suber that serializes `RegDom` subclasses with LF proem. .. py:class:: Subery(**kwa) Bases: :py:obj:`Duror` Subery subclass of Duror for managing subdbs of Duror for durable storage of action data .. py:method:: reopen(**kwa) Open sub databases .. attribute:: cans subdb whose values are serialized Can instances Can is a durable Bag :type: Suber .. attribute:: drqs subdb whose values are serialized RegDom instances Interfaced via a Durq which is a durable queue (FIFO) :type: IoSetSub .. attribute:: dsqs subdb whose values are serialized RegDom instances Interfaced via a Dusq which is durable set queue (FIFO deduped) :type: IoSetSub