Using htmap - Revision history

Admin: /* Tips and tricks */

2022-02-16T10:41:56Z

Tips and tricks

← Older revision		Revision as of 10:41, 16 February 2022
Line 69:		Line 69:
	Using a second shell session with the same virtual environment as before enabled, you can monitor the progress of your htmap script with the following command:		Using a second shell session with the same virtual environment as before enabled, you can monitor the progress of your htmap script with the following command:
	<pre>		<pre>
	htmap status --live		$ htmap status --live
	</pre>		</pre>

Admin at 10:41, 16 February 2022

2022-02-16T10:41:40Z

← Older revision		Revision as of 10:41, 16 February 2022
Line 64:		Line 64:
	<pre>		<pre>
	$ deactivate		$ deactivate
			</pre>

			== Tips and tricks ==
			Using a second shell session with the same virtual environment as before enabled, you can monitor the progress of your htmap script with the following command:
			<pre>
			htmap status --live
	</pre>		</pre>

Admin at 10:38, 16 February 2022

2022-02-16T10:38:43Z

← Older revision		Revision as of 10:38, 16 February 2022
Line 7:		Line 7:
	== How to use htmap with our HTCondor cluster? ==		== How to use htmap with our HTCondor cluster? ==

	There are many ways to use htmap, and we haven't tested all of them. This small tutorial will demonstrate how to use htmap in a shared mode. (By "shared", it's meant that it exploits the fact that your user home directory is also available on the workernodes.)		There are many ways to use htmap, and we haven't tested all of them. This small tutorial will demonstrate how to use htmap in a shared mode. (By "shared", it's meant that it exploits the fact that your user home directory is also available on the workernodes.) Let's start!

			1. Login to a m-machine.

			2. Set up a Python 3 virtual environment:
			<pre>
			$ mkdir python-envs
			$ cd python-envs
			$ python3 -m venv htmap_env
			$ source htmap_env/bin/activate
			</pre>
			3. In this new environment, you can install the htmap library:
			<pre>
			$ python -m pip install htmap
			</pre>
			4. Now we'll write a simple script to test htmap. Here is the content:
			<pre>
			#!/usr/bin/env python

			import sys
			import htmap
			from htmap import names

			def _get_base_descriptors_for_shared(
			tag: str,
			map_dir
			):
			return {
			'universe': 'vanilla',
			'executable': sys.executable,
			'transfer_executable': 'False',
			'arguments': f'{names.RUN_SCRIPT} $(component)',
			'transfer_input_files': [
			(map_dir / names.RUN_SCRIPT).as_posix(),
			],
			}

			htmap.register_delivery_method(
			'shared',
			descriptors_func = _get_base_descriptors_for_shared,
			)

			htmap.settings["DELIVERY_METHOD"] = 'shared'

			m = htmap.map(str, range(5))
			print(list(m))
			</pre>

			5. Now you can run this script and wait...
			<pre>
			$ ./test_htmap.py
			['0', '1', '2', '3', '4']
			</pre>

			6. To get out of the Python virtual environment when you are done:
			<pre>
			$ deactivate
			</pre>

Admin: Created page with "== What is htmap? == It's a Python library that allows you to map some function calls out to an HTCondor cluster. It's really easy to use and it brings the computing power of our cluster into your Python codes. You'll find a more detailed presentation [https://htmap.readthedocs.io/en/latest/ here]. == How to use htmap with our HTCondor cluster? == There are many ways to use htmap, and we haven't tested all of them. This small tutorial will demonstrate how to use htma..."

2022-02-16T10:23:45Z

Created page with "== What is htmap? == It's a Python library that allows you to map some function calls out to an HTCondor cluster. It's really easy to use and it brings the computing power of our cluster into your Python codes. You'll find a more detailed presentation [https://htmap.readthedocs.io/en/latest/ here]. == How to use htmap with our HTCondor cluster? == There are many ways to use htmap, and we haven't tested all of them. This small tutorial will demonstrate how to use htma..."

New page

== What is htmap? ==

It's a Python library that allows you to map some function calls out to an HTCondor cluster. It's really easy to use and it brings the computing power of our cluster into your Python codes.

You'll find a more detailed presentation [https://htmap.readthedocs.io/en/latest/ here].

== How to use htmap with our HTCondor cluster? ==

There are many ways to use htmap, and we haven't tested all of them. This small tutorial will demonstrate how to use htmap in a shared mode. (By "shared", it's meant that it exploits the fact that your user home directory is also available on the workernodes.)