hio.base.multidoing

hio.core.multidoing Module

Provides multiprocessing support

Module Contents

class hio.base.multidoing.HandDom

Bases: hio.help.RawDom

Configuration dataclass of Crewer crew hand info managed by its Bosser boss.

proc

crew hand subprocess or None

Type:: Any or None

exiting

True means commanded to exit but may not have exited yet False means not yet commanded to exit

Type:: bool

class hio.base.multidoing.CrewDom

Bases: hio.help.RawDom

Configuration dataclass of crew subprocess Doist parameters for crew doist and its Crewer to be injected by Bosser. Use this when storing configuration in database or file. Use RawDom serialization hidden methods:

name

child doist identifier for resources.

Type:: str

tyme

child doist start tyme

Type:: float

tock

child doist tock, tyme lag between runs

Type:: float or None

real

child doist True means run in real time, tyme is real time

Type:: bool

limit

child doist tyme limit. None means run forever

Type:: float or None

doers

child doist List of Doers

Type:: list[Doers]

temp

True means use temporary file resources

Type:: bool or None

boss

Bosser info for Crewer. May be filled at enter time Crewer uses to contact Bosser.

Type:: Bossage or None

class hio.base.multidoing.MemoDom

Bases: hio.help.RawDom

Inter Boss Crew Hand structured memo dataclass. Used for control memos Between Boss and Crew Doers via their .peer UXD BossMemoer or CrewMemoer.

tag

type of memo

Type:: str

name

unique identifier as source of memo

Type:: str

load

type specific payload of memo

Type:: dict

class hio.base.multidoing.RegDom

Bases: hio.help.RawDom

Inter Boss Crew Hand structured memo dataclass. Used for REG memos Between Boss and Crew Doers via their .peer UXD BossMemoer or CrewMemoer.

tag

type of memo

Type:: str

name

unique identifier as source of memo

Type:: str

load

empty dict

Type:: dict

class hio.base.multidoing.AddrDom

Bases: hio.help.RawDom

Load Field Value of ACK

tag

type of memo being acked

Type:: str

name

unique identifier as source of memo being acked

Type:: str

addr

addr of source of memo being acked

Type:: dict

class hio.base.multidoing.AckDom

Bases: hio.help.RawDom

Inter Boss Crew Hand structured memo dataclass. Used for ACK memos Between Boss and Crew Doers via their .peer UXD BossMemoer or CrewMemoer. The load of the ACK is an AddrDom instance with the fields, tag, name, and addr. The AddrDom tag and name fields come from the memo being acked. The addr field is the src address path of the memo being acked.

tag

type of memo

Type:: str

name

unique identifier as source of memo

Type:: str

load

attribution info of acked memo

Type:: AddrDom

class hio.base.multidoing.EndDom

Bases: hio.help.RawDom

Inter Boss Crew Hand structured memo dataclass. Used for END memos Sent by Boss to its Crew Doers to gracefully terminate. Sent via their .peer UXD BossMemoer or CrewMemoer instead of using an OS SIGTERM signal.

tag

type of memo

Type:: str

name

unique identifier of boss

Type:: str

load

empty dict

Type:: dict

class hio.base.multidoing.BokDom

Bases: hio.help.RawDom

Inter Boss Crew Hand structured memo dataclass. Used for BOK memos Sent by Boss to its Crew Doers with an address book of the crew hands. This enables crew hand to crew hand peer-to-peer messages sent via their .peer UXD CrewMemoer.

The load value is a dict keyed by crew hand names with values that are the crew hand UXD peer addres file paths. The items of the dict are (name, addr) tuples

tag

type of memo

Type:: str

name

unique identifier of boss

Type:: str

load

items are (name, addr) tuples

Type:: dict

class hio.base.multidoing.TagDomCodex

Bases: hio.help.IceMapDom

Codex keyed by memo tag with value of associated MemoDom subclass. Attribute values are classes not instances

iters asdict

REG

RegDom

Type:: type[RegDom]

ACK

AckDom

Type:: type[AckDom]

END

EndDom

Type:: type[EndDom]

BOK

BokDom

Type:: type[BokDom]

class hio.base.multidoing.MultiDoerBase(*, name='base', temp=False, reopen=False, bc=4, **kwa)

Bases: hio.help.Namer, hio.core.uxd.peermemoing.PeerMemoer, hio.base.doing.Doer

MultiDoerBase is base class for Doers in multiprocessing. Each subclass has support for UXD peer communications via PeerMemoer super class. As well as support for local logging given scope of module global ogler at enter time.

See Namer, PeerMemoer, and Doer for inherited attributes, properties, and methods.

Inherited Class Attributes:

See Namer, PeerMemoer and Doer Classes

Class Attributes:

Tagex (TagDomCodex):  codex mapping memo tags to memo doms

Inherited Attributes:

(See Doer and PeerMemoer for all)
done (bool): completion state:
             True means completed fully. Otherwise incomplete.
             Incompletion value may be None or False.
opts (dict): schedulaer injects options from .opts into its .do generator
             function as **opts parameter.
name (str): unique identifier for this instance used to manage local resources
temp (bool): True means logger or other file resources created by
                .start() will use temp
             False otherwise
reopen (bool): True (re)open with this init
               False not (re)open with this init but later (default)
bc (int or None): count of transport buffers of MaxGramSize

Attributes:

logger (Logger or None): from module scope ogler created at enter time
                with local resources.
graceful (bool): indication to gracefully exit on next recur. Set by
                 .force signal handler. Must add check in .self.recur
                 if self.graceful: sys.exit()
                 True means exit on next recur
                 False otherwise.

Inherited Properties:

tyme (float): is float relative cycle time of associated Tymist .tyme obtained
    via injected .tymth function wrapper closure.
tymth (closure): function wrapper closure returned by Tymist.tymen()
                method. When .tymth is called it returns associated
                Tymist.tyme. Provides injected dependency on Tymist
                tyme base.
tock (float): desired time in seconds between runs or until next run,
         non negative, zero means run asap
addrByName (dict): mapping between (name, address) pairs, these
    must be one-to-one so that inverse is also one-to-one
nameByAddr (dict): mapping between (address, name) pairs, these
    must be one-to-one so that inverse is also one-to-one
countNameAddr (int): count of entries in .addrByName

Inherited Methods:

- __call__(): makes instance callable as generator function returning generator
- do(): generator method that returns generator
- enter(): is enter context action method
- recur(): recur context action method or generator method
- clean(): clean context action method
- exit(): exit context method
- close(): close context method
- abort(): abort context method
- wind(): injects ._tymth dependency from associated Tymist to get its .tyme
- clearAllNameAddr()
- getAddr(name)
- getName(addr)
- addNameAddr(name, addr)
- remNameAddr(name=None, addr=None)
- changeAddrAtName(name=None, addr=None)
- changeNameAtAddr(addr=None, name=None)

enter(*, temp=None)

Do ‘enter’ context. Set up resources. Comparable to context manager enter. Start processes with config from .loads Not a generator method.

Parameters:

temp (bool or None): True means use temporary file resources if any.
                    None means ignore parameter value. Use self.temp.

Inject temp or self.temp into file resources here if any Doist or DoDoer winds its doers on enter

recur(tyme): Do ‘recur’ context. Must be overridden in subclass

static tojson(d)

Returns compact JSON serialization of d suitable for .memoit.

Parameters:

d (dict or list): object to be serialized

Returns:

s (str): serialized JSON

class hio.base.multidoing.Bosser(*, name='boss', loads=None, **kwa)

Bases: MultiDoerBase

Bosser spawns multiple crew hand subprocesses and injects each with a Doist and Doers. The boss Doists runs the Bosser in the parent process. Each crew hand Doist runs a Crewer that coordinates with the Bosser.

Analogy Boss runs a Crew of Hans. The parent process has a boss Doist which runs a Bosser. Each crew hand is a child process with its own crew doist that runs its own Crewer

See MultiDoerBase for all inherited attributes, properties, and methods.

Inherited Class Attributes:

See MultiDoerBase Class

Inherited Attributes:

(See MultiDoerBase Class for all)
name (str): unique identifier for this instance
            used to manage local resources
logger (Logger or None): from module scope ogler created at enter time
                with local resources.
graceful (bool): indication to gracefully exit on next recur. Set by
                 .force signal handler. Must add check in .self.recur
                 if self.graceful: sys.exit()
                 True means exit on next recur
                 False otherwise.

Attributes:

loads (list[dict]): Bosser info to be injected into Crewer .start()
            containing both crew doist parameters for Process target
            kwargs and Crewer parameters
                    (see Loadage._asdict() or CrewDom._asdict())
ctx (mp.context.SpawnContext or None): context under which to spawn processes
crew (dict): values HandDom instances keyed by name
crewed (bool): True means all crew members have registered memo interface
                    with this boss.
               False means not yet

Inherited Properties:

See MultiDoerBase Class

Inherited Methods:

See MultiDoerBase Class

enter(*, temp=None)

Do ‘enter’ context. Set up resources. Comparable to context manager enter. Start processes with config from .loads Not a generator method.

Parameters:

temp (bool or None): True means use temporary file resources if any.
                    None means ignore parameter value. Use self.temp.

Inject temp or self.temp into file resources here if any

Doist or DoDoer winds its doers on enter

recur(tyme): Do ‘recur’ context.

exit(): Do ‘exit’ (try finally) context.

cease(): Do ‘cease’ context.

abort(ex): Do ‘abort’ context.

serviceRxMemos()

Service all memos in .rxms (greedy) if any

Override in subclass to handle result(s) and put them somewhere

static start(*, name='crew', tyme=0.0, tock=None, real=True, limit=None, doers=None, temp=None, boss=None)

Process target function to spinup and run doist inside crew subprocess after it has been started.

Parameters:

name (str): unique crew hand name to be used to manage resources
tyme (float): crew doist initial value of cycle time in seconds
tock (float or None): crew doist tock time in seconds. None means run Asap
real (bool): crew doist True means run in real time,
            Otherwise run faster than real
limit (float or None): crew doist seconds for max run time of doist.
                      None means no limit.
doers (iterable[Doer] or None): crew doist Doer class instances
                       First entry must be Crewer
temp (bool or None): True means use temp file resources by injection.
                    Otherwise ignore do not inject.
boss (Bossage or None): boss info. May be filled at enter time
                      Crewer uses to contact Bosser.

Doist must be built after process started so local tymth closure is created inside subprocess so that when doist winds the tyme for its doers the tymth closure is locally sourced.

When run inside subprocess, this static method provides the outside scope for any Doist and doers that reference objects inside the __main__ moduleo that calls this method such as ogler. The subprocess gets a picked copy of these module globals. Any Doer inside then has access to the module globals and can update them as neede. In the case of ogler this means changing ogler.level, ogler.temp and running ogler.reopen(temp=temp) as appropriate.

class hio.base.multidoing.Crewer(*, name='crew', boss=None, **kwa)

Bases: MultiDoerBase

Crewer runs interface between a given crew hand subprocess and its boss process. This must be first doer run by crew hand subprocess doist.

See MultiDoerBase for all inherited attributes, properties, and methods.

Inherited Class Attributes:

See MultiDoerBase Class

Inherited Attributes:

(See MultiDoerBase Class for all)
name (str): unique identifier for this instance
            used to manage local resources
logger (Logger or None): from module scope ogler created at enter time
                with local resources.
graceful (bool): indication to gracefully exit on next recur. Set by
                 .force signal handler. Must add check in .self.recur
                 if self.graceful: sys.exit()
                 True means exit on next recur
                 False otherwise.

Attributes:

boss (Bossage or None): contact info for communicating with boss
registered (bool): True means .path acked registered with boss memoing
                   False not yet registered

Inherited Properties:

See MultiDoerBase Class

Inherited Methods:

See MultiDoerBase Class

enter(*, temp=None)

Do ‘enter’ context. Set up resources. Comparable to context manager enter. Not a generator method.

Parameters:

temp (bool or None): True means use temporary file resources if any
                    None means ignore parameter value. Use self.temp

Inject temp or self.temp into file resources here if any

Doist or DoDoer winds its doers on enter

recur(tyme): Do ‘recur’ context.

exit(): Do ‘exit’ (try finally) context.

cease(): Do ‘cease’ context.

abort(ex): Do ‘abort’ context.

serviceRxMemos()

Service all memos in .rxms (greedy) if any

Override in subclass to handle result(s) and put them somewhere

hio.base.multidoing

Module Contents

hio

Navigation

Related Topics