hio.base.webduring ================== .. py:module:: hio.base.webduring .. autoapi-nested-parse:: Browser-safe durable storage for HIO wrappers. Module Contents --------------- .. py:class:: WebSubDb(*, key, dupsort, items) LMDB-like sub-database handle for existing wrapper classes. .. py:class:: WebDuror(*, name='main', base='', temp=False, reopen=False, clear=False, stores=None, storageOpener=None, **kwa) Bases: :py:obj:`hio.base.filing.Filer` Browser durable storage with Duror-compatible method semantics. Uses an async PyScript style storage opener instead of LMDB and exposes the same low-level db methods used by Duror wrappers. .. py:method:: reopen(*, clear=False, stores=None, storageOpener=None, temp=None, **kwa) :async: Open declared browser storage namespaces and load persisted records. .. py:method:: close(clear=False) Close local handles after persisting or clearing pending state. When called inside a running loop, persistence is scheduled on that loop. Use await aclose() when the caller must wait for completion. .. py:method:: aclose(clear=False) :async: Flush pending writes, optionally clear persisted browser records, and close. .. py:method:: flush() :async: Persist dirty stores through their browser storage handles. .. py:property:: version Return the version of database stored in __version__ key. This value is read through cached in memory :returns: semver string the version of the database or None if not set in the database :rtype: version (str) .. py:method:: getVer() Returns the value of the the semver formatted version in the __version__ key in this database :returns: semver formatted version of the database :rtype: str .. py:method:: setVer(version) Set the version of the database in the __version__ key :param version: The new semver formatted version str of the database :type version: str .. py:method:: putVal(sdb, key, val) Write serialized bytes val to location key in db (subdb) Does not overwrite. :returns: True if val successfully written; False if val at key already exists :rtype: result (bool) :param sdb: opened named subdb with dupsort=False :type sdb: WebSubDb :param key: within subdb's keyspace :type key: bytes :param val: to be written at key :type val: bytes .. py:method:: pinVal(sdb, key, val) Write serialized bytes val to location key in db Overwrites existing val if any :returns: True if val successfully written; False otherwise :rtype: result (bool) :param sdb: opened named subdb with dupsort=False :type sdb: WebSubDb :param key: within subdb's keyspace :type key: bytes :param val: to be written at key :type val: bytes .. py:method:: getVal(sdb, key) Get val at key in db :returns: None if no entry at key :rtype: val (bytes) :param sdb: opened named subdb with dupsort=False :type sdb: WebSubDb :param key: within subdb's keyspace :type key: bytes .. py:method:: remVal(sdb, key) Removes value at key in db. :returns: True If key exists in database False otherwise :rtype: result (bool) :param sdb: opened named subdb with dupsort=False :type sdb: WebSubDb :param key: within subdb's keyspace :type key: bytes .. py:method:: cntVals(sdb) Counts entries in subdb db :returns: number of entries in subdb db, or zero otherwise :rtype: count (int) :param sdb: opened named subdb with dupsort=False :type sdb: WebSubDb .. py:method:: getTopItemIter(sdb, top=b'') Iterates over branch of subdb db rooted at top key :returns: of (full key, val) tuples over a branch of the db given by top key where: full key is full database key for val not truncated top key :rtype: items (Iterator) Raises StopIteration Error when empty. :param sdb: instance of named sub db with dupsort==False :type sdb: WebSubDb :param top: truncated top key, a key space prefix to get all the items from multiple branches of the key space. If top key is empty then gets all items in database. In Python str.startswith('') always returns True so if branch key is empty string it matches all keys in db with startswith. :type top: bytes .. py:method:: remTopVals(sdb, top=b'') Removes all values in branch of db given top key. :returns: True if values were deleted at key. False otherwise if no values at key :rtype: result (bool) :param sdb: instance of named sub db with dupsort==False :type sdb: WebSubDb :param top: truncated top key, a key space prefix to get all the items from multiple branches of the key space. If top key is empty then deletes all items in database :type top: bytes Raises StopIteration Error when empty. .. py:method:: suffix(key: bytes | str | memoryview, ion: int, *, sep: bytes | str = b'.') :staticmethod: Return iokey after concatenating the hex ordinal suffix to `key` using `sep`. :param key: apparent effective database key (unsuffixed) :type key: bytes|str|memoryview :param ion: insertion ordered ordinal numberfor set of vals :type ion: int) :param sep: separator character(s) for concatenating suffix :type sep: bytes .. py:method:: unsuffix(iokey: bytes | str | memoryview, *, sep: bytes | str = b'.') :staticmethod: Return `(key, ion)` from splitting `iokey` at the rightmost `sep`. :param iokey: actual database key suffixed :type iokey: bytes|str|memoryview :param sep: separator character(s) for splitting suffix :type sep: bytes .. py:method:: getIoValFirst(sdb, key, *, ion=0, sep=b'.') Gets first of the insertion ordered set of values at key, None if no entry. Returns the first value at the apparent effective key whose iokey is greater than or equal to the iokey from `ion`, or None if no entry. :param sdb: instance of named sub db with dupsort==False :type sdb: WebSubDb :param key: Apparent effective key :type key: bytes :param ion: starting ordinal value, default 0 :type ion: int .. py:method:: getIoValLast(sdb, key, *, sep=b'.') Gets last value (last in) of insertion ordered set values at key Returns the last value at the apparent effective key, or None if no entry. Uses hidden ordinal key suffix for insertion ordering. The suffix is appended and stripped transparently. :param sdb: instance of named sub db with dupsort==False :type sdb: WebSubDb :param key: Apparent effective key :type key: bytes .. py:method:: getIoVals(sdb, key, *, ion=0, sep=b'.') Gets list of all the insertion ordered set of values at key :returns: insertion-ordered values at same apparent effective key; hidden ordinal key suffix is used for insertion ordering and stripped transparently. :rtype: vals (list[bytes]) :param sdb: instance of named sub db with dupsort==False :type sdb: WebSubDb :param key: Apparent effective key :type key: bytes :param ion: starting ordinal value, default 0 :type ion: int .. py:method:: getIoValsIter(sdb, key, *, ion=0, sep=b'.') Gets Iterator of all the insertion ordered set of values at key :returns: iterator over insertion ordered set of values at same apparent effective key. Uses hidden ordinal key suffix for insertion ordering. The suffix is appended and stripped transparently. :rtype: ioset (Iterator) Raises StopIteration Error when empty. :param sdb: instance of named sub db with dupsort==False :type sdb: WebSubDb :param key: Apparent effective key :type key: bytes :param ion: starting ordinal value, default 0 :type ion: int .. py:method:: popIoVal(sdb, key, *, ion=0, sep=b'.') Pops first of the insertion ordered set of values at key. None if no entry. Pop deletes the returned entry. :returns: first val at same apparent effective key whose iokey >= iokey from ion Uses hidden ordinal key suffix for insertion ordering. The suffix is appended and stripped transparently. None if no entry. :rtype: val (bytes|None) :param sdb: instance of named sub db with dupsort==False :type sdb: WebSubDb :param key: Apparent effective key :type key: bytes :param ion: starting ordinal value, default 0 :type ion: int .. py:method:: remIoVals(sdb, key, *, sep=b'.') Deletes all values at apparent effective key. Uses hidden ordinal key suffix for insertion ordering. The suffix is appended and stripped transparently. :returns: True if at least one value was deleted at key. False otherwise, if no values at key :rtype: result (bool) :param sdb: instance of named sub db with dupsort==False :type sdb: WebSubDb :param key: Apparent effective key :type key: bytes .. py:method:: cntIoVals(sdb, key, *, sep=b'.') Count all values with the same apparent effective key. Uses hidden ordinal key suffix for insertion ordering. The suffix is appended and stripped transparently. :returns: count values in set at apparent effective key :rtype: count (int) :param sdb: instance of named sub db with dupsort==False :type sdb: WebSubDb :param key: Apparent effective key :type key: bytes .. py:method:: getTopIoItemIter(sdb, top=b'', *, sep=b'.') Gets Iterator of all values in branch rooted at top key The suffix is appended and stripped transparently. :returns: iterator of tuples (key, val) where key is apparent key with hidden insertion ordering suffixe removed from effective key. Iterates over top branch of insertion ordered set values where each effective key has trailing hidden suffix of serialization of insertion ordering ordinal. Uses hidden ordinal key suffix for insertion ordering. :rtype: items (Iterator[(key,val)]) Raises StopIteration Error when empty. :param sdb: instance of named sub db with dupsort==False :type sdb: WebSubDb :param top: top key in db. When top is empty then every item in db. :type top: bytes :param sep: sep character for attached io suffix :type sep: bytes .. py:method:: addIoVal(sdb, key, val, *, sep=b'.') Add val at end (append) of insertion ordered set of values all with the same apparent effective key. Does not dedup val. Uses hidden ordinal key suffix for insertion ordering. The suffix is appended and stripped transparently. :returns: True is added to set. False if already in set. :rtype: result (bool) :param sdb: instance of named sub db with dupsort==False :type sdb: WebSubDb :param key: Apparent effective key :type key: bytes :param val: serialized value to add :type val: bytes .. py:method:: putIoVals(sdb, key, vals, *, sep=b'.') Adds at end (append) each val in vals to insertion ordered list of values all with the same apparent effective key for each val. Does not dedup added vals. Uses hidden ordinal key suffix for insertion ordering. The suffix is appended and stripped transparently. :returns: True if added to set. False if already in set. :rtype: result (bool) :param sdb: instance of named sub db with dupsort==False :type sdb: WebSubDb :param key: Apparent effective key :type key: bytes :param vals: serialized values to add to set of vals at key :type vals: Iterable .. py:method:: pinIoVals(sdb, key, vals, *, sep=b'.') Erase all vals at key and then add vals as insertion ordered set of values all with the same apparent effective key. Does not dedup vals. Uses hidden ordinal key suffix for insertion ordering. The suffix is appended and stripped transparently. :returns: True is added to set. :rtype: result (bool) :param sdb: instance of named sub db with dupsort==False :type sdb: WebSubDb :param key: Apparent effective key :type key: bytes :param vals: serialized values to add to set of vals at key :type vals: abc.Iterable .. py:method:: addIoSetVal(sdb, key, val, *, sep=b'.') Add val idempotently to insertion ordered set of values all with the same apparent effective key if val not already in set of vals at key. Dedups the added val. Uses hidden ordinal key suffix for insertion ordering. The suffix is appended and stripped transparently. :returns: True is added to set. False if already in set. :rtype: result (bool) :param sdb: instance of named sub db with dupsort==False :type sdb: WebSubDb :param key: Apparent effective key :type key: bytes :param val: serialized value to add :type val: bytes .. py:method:: putIoSetVals(sdb, key, vals, *, sep=b'.') Adds idempotently each val in vals to insertion ordered set of values all with the same apparent effective key for each val that is not already in set of vals at key. Dedups the put vals. Uses hidden ordinal key suffix for insertion ordering. The suffix is appended and stripped transparently. :returns: True if added to set. False if already in set. :rtype: result (bool) :param sdb: instance of named sub db with dupsort==False :type sdb: WebSubDb :param key: Apparent effective key :type key: bytes :param vals: serialized values to add to set of vals at key :type vals: Iterable .. py:method:: pinIoSetVals(sdb, key, vals, *, sep=b'.') Erase all vals at key and then add unique vals as insertion ordered set of values all with the same apparent effective key. Dedups the set vals. Uses hidden ordinal key suffix for insertion ordering. The suffix is appended and stripped transparently. :returns: True is added to set. :rtype: result (bool) :param sdb: instance of named sub db with dupsort==False :type sdb: WebSubDb :param key: Apparent effective key :type key: bytes :param vals: serialized values to add to set of vals at key :type vals: abc.Iterable .. py:method:: remIoSetVal(sdb, key, val, *, sep=b'.') Removes (delete) matching val at apparent effective key if exists. Uses hidden ordinal key suffix for insertion ordering. The suffix is appended and stripped transparently. Because the insertion order of val is not provided must perform a linear search over set of values. Another problem is that vals may get added and deleted in any order so the max suffix ion may creep up over time. The suffix ordinal max > 2**16 is an impossibly large number, however, so the suffix will not max out practically.But its not the most elegant solution. In some cases a better approach would be to use getIoSetItemsIter which returns the actual iokey not the apparent effetive key so can delete using the iokey without searching for the value. This is most applicable when processing escrows where all the escrowed items are processed linearly and one needs to delete some of them in stride with their processing. :returns: True if val was deleted at key. False otherwise if val not found at key :rtype: result (bool) :param sdb: instance of named sub db with dupsort==False :type sdb: WebSubDb :param key: Apparent effective key :type key: bytes :param val: value to delete :type val: bytes .. py:function:: openWebDuror(*, cls=None, name='test', temp=True, clear=False, **kwa) :async: Async context manager wrapper for WebDuror instances.