============================ Example Hardware Map Queries ============================ .. contents:: Table of Contents :local: The correct syntax when using HWM query objects is not necessarily intuitive, since it tries to bridge python-like syntax with SQL-like queries. Once you understand the mechanics though, it because pretty trivial to construct arbitrarily complex queries. Below are some examples demonstrating most of what people will generally be trying to do. If none of these covers your circumstance, let me know (Joshua). This page in particular should be a resource that grows if need be. What is a Query? ================ A query is a method of hardware map (``hwm``) objects. It can also refer to the object that method returns: .. code-block:: python In [1]: import pydfmux In [2]: session = pydfmux.load_session(open('mcgill/hardware_maps/mcgill_hwm_004/mcgill_hwm_complete.yaml','r')) In [3]: hwm = session['hardware_map'] In [4]: bolos = hwm.query(pydfmux.Bolometer) In [5]: bolos Out[5]: That query object is iterable, like a list of returns, but also has some special properties: in particular it knows about all of the methods of the objects contained. .. code-block:: python In [6]: bolos.all() Out[6]: [Wafer(u'arg1a').Bolometer(u'1A.1.X'), Wafer(u'arg1a').Bolometer(u'1A.1.Y'), Wafer(u'arg1a').Bolometer(u'1A.2.X'), Wafer(u'arg1a').Bolometer(u'1A.2.Y'), Wafer(u'arg1a').Bolometer(u'1A.3.X'), Wafer(u'arg1a').Bolometer(u'1A.3.Y'), Wafer(u'arg1a').Bolometer(u'1A.4.X'), Wafer(u'arg1a').Bolometer(u'1A.4.Y'), Wafer(u'arg1a').Bolometer(u'1A.5.X'), Wafer(u'arg1a').Bolometer(u'1A.5.Y'), Wafer(u'arg1a').Bolometer(u'1A.6.X'), Wafer(u'arg1a').Bolometer(u'1A.6.Y'), Wafer(u'arg1a').Bolometer(u'1A.7.X'), Wafer(u'arg1a').Bolometer(u'1A.7.Y'), Wafer(u'arg1a').Bolometer(u'1A.8.X'), Wafer(u'arg1a').Bolometer(u'1A.8.Y')] In [7]: bolos[0] Out[7]: Wafer(u'arg1a').Bolometer(u'1A.1.X') In [8]: bolos[0]. bolos[0].channel_map bolos[0].name bolos[0].state bolos[0].hwm bolos[0].overbias bolos[0].tune bolos[0].iceboard bolos[0].readout_channel bolos[0].wafer bolos[0].lc_channel bolos[0].rfrac bolos[0].metadata bolos[0].squid In [9]: bolos.lc_channel Out[9]: [LCBoard(u'LC001_A').LCChannel(1), LCBoard(u'LC001_A').LCChannel(2), LCBoard(u'LC001_A').LCChannel(3), LCBoard(u'LC001_A').LCChannel(4), LCBoard(u'LC001_B').LCChannel(1), LCBoard(u'LC001_B').LCChannel(2), LCBoard(u'LC001_B').LCChannel(3), LCBoard(u'LC001_B').LCChannel(4), LCBoard(u'LC001_C').LCChannel(1), LCBoard(u'LC001_C').LCChannel(2), LCBoard(u'LC001_C').LCChannel(3), LCBoard(u'LC001_C').LCChannel(4), LCBoard(u'LC001_D').LCChannel(1), LCBoard(u'LC001_D').LCChannel(2), LCBoard(u'LC001_D').LCChannel(3)] In [10]: bolos.overbias_and_null() 2015-08-21 15:45:15 | INFO | overbias_and_null.py | overbias_and_null | STARTING: overbiasing on 1 modules . . . All of pydfmux "runs on" query objects, in the sense that the core algorithms act using query objects. Often you will see the algorithms executed by calling methods of the object contained within query on the set of all objects contained in the query (``bolos.overbias_and_null()``). So, a query object is the central object of pydfmux, it is important to know how to get and manipulate these objects. Elements of Querying ==================== When formulating a query, there are three primary ingredients: * ``query(pydfmux.OBJECT)`` specifies the type of object to be contained in the query returned. * ``filter(. . .)`` filters the contents based on attributes of the object, or any other joined object. * ``join(pydfmux.ANOTHEROBJECT)`` walks along the ORM map (:ref:`DfmuxObjects`) to "join" objects together in order to filter by joined objects in addition to returned objects (covered in detail at the bottom of this section). Filters ------- A query for Bolometers, filtered based on whether the "tuned" in the hardware map is "True": .. code-block:: python In [11]: bolos_all = hwm.query(pydfmux.Bolometer) In [12]: all(bolos_all.tune) Out[12]: False In [13]: bolos_tune = hwm.query(pydfmux.Bolometer).filter(pydfmux.Bolometer.tune==True) In [14]: all(bolos_tune.tune) Out[14]: True Note here that ``.filter`` is a method of the query object, so additional filters can be performed on the same query object: .. code-block:: python In [15]: bolos_tune_trimmed = bolos_tune.filter(pydfmux.Bolometer.rfrac<0.9) These could also have been done simultaneously as separate arguments to ``filter``: .. code-block:: python In [16]: bolos_tune_trimmed = hwm.query(pydfmux.Bolometer).filter(pydfmux.Bolometer.tune==True, pydfmux.Bolometer.rfrac<0.9) Instead of justing using comparators in the filter (<, >, ==, etc) it can also be used to check for inclusion. Given a list of Bolometer names that I wish to tune separately, I query for bolometer whose names are in that list: .. code-block:: python In [17]: special_bolos = hwm.query(pydfmux.Bolometer).filter(pydfmux.Bolometer.name.in_(['1A.5.Y', '1A.6.X'])) In [18]: special_bolos.name Out[18]: [u'1A.5.Y', u'1A.6.X'] Note that the "in" is NOT a separate word (as you might expect from pure python). There is no space between "Bolometer.name." and "in_". The underscore is also important. If, alternatively, I have a list of bolometers I wish to *exclude*, I can perform a negation by prepending the filter line with a tidle (``~``). .. code-block:: python In [19]: good_bolos = hwm.query(pydfmux.Bolometer).filter(~pydfmux.Bolometer.name.in_(['1A.5.Y', '1A.6.X'])) In [20]: '1A.5.Y' in good_bolos.name Out[20]: False .. warning:: This is different from python and **not** intuitive, so I want to emphasize it. To negate a filtering parameter, instead of using "not" as you would in python, it should be prepended with a tilde (~). Joins ----- What if you want to query for Bolometers, but filter based on the transimpedance of the SQUIDs they are attached to? This will not work: .. code-block:: python In [21]: bolos = hwm.query(pydfmux.Bolometer).filter(pydfmux.SQUID.transimpedance>300) # Bad Query Because "transimpedance" is an attribute of Bolometers, and this hasn't shown the query how to get from Bolometers to SQUIDs. That is the role of ``join``. In order to get from SQUID to Bolometer we need to step through the ChannelMapping object (see :ref:`DfmuxObjects`). A correct query will look like: .. code-block:: python In [22]: bolos = y['hardware_map'].query(pydfmux.Bolometer).join(pydfmux.ChannelMapping, pydfmux.SQUID)\ .filter(pydfmux.SQUID.transimpedance>200) Another example: To query SQUIDs that are controlled by Mezzanine 014, there are two different paths from the SQUID object to the MGMEZZ04 object: .. code-block:: python In [23]: squids = y['hardware_map'].query(pydfmux.SQUID).join(pydfmux.SQUIDModule, pydfmux.SQUIDController, pydfmux.MGMEZZ04)\ .filter(pydfmux.MGMEZZ04.serial=='008') In [24]: squids = y['hardware_map'].query(pydfmux.SQUID).join(pydfmux.ChannelMapping, pydfmux.ReadoutChannel, pydfmux.ReadoutModule, pydfmux.MGMEZZ04)\ .filter(pydfmux.MGMEZZ04.serial=='008') The second query will only work if there actually exist channel mappings between that SQUID and a readout channel. This happens in the CSV file that is part of the hardware map, and is therefore not guaranteed to be present for every SQUID. The first query, on the other hand, makes no such assumptions, and is thus the safer choice. Note that, once items have been joined, any of them may be filtered on. So, in the example above, I can query for SQUIDs attached to Mezzanine 008, Module 1 like so: .. code-block:: python In [25]: squids = y['hardware_map'].query(pydfmux.SQUID).join(pydfmux.SQUIDModule, pydfmux.SQUIDController, pydfmux.MGMEZZ04)\ .filter(pydfmux.MGMEZZ04.serial=='008', pydfmux.SQUIDModule.module==1) ORM Plot Resource ================= The interconnections between these structures are shown in :ref:`DfmuxObjects`. .. _DfmuxObjects: .. figure:: ../software/images/objects.svg :align: center Paths to walk along the ORM.