multithreading Module

Code handling the concurrency of data analysis.

multithreading.request

Functions for concurrent GTEx requests.

Warning

Though every effort has been made to ensure thread safety, the concurrency is, as of yet, untested.

Note

This module will likely undergo significant refactoring to generalise the concurrency pipeline and move all data handling code to a separate model.

Any API query is likely to be an I/O bound process, particularly if there are many to make. As this is a single step, many in, many out process, concurrency can be easily achieved with a thread local requests.session and mapping with concurrent.futures.ThreadPoolExecutor.map. The call to concurrent.futures.ThreadPoolExecutor.map is handled in the analysis script.

gtexquery.multithreading.request._get_session(headers: Optional[dict[str, str]] = None, params: Optional[dict[str, str]] = None) → requests.sessions.Session

Instantiate a thread local session.

The requests session is not thread safe, per this thread. To circumvent this, we create a thread local session. This means each session will still make multiple requests but remain isolated to its calling thread.

Parameters

• headers (Optional[dict[str, str]]) – Headers to be used for the session

• params (Optional[dict[str, str]]) – Parameters to be used for the session

Returns

Return type

requests.Session