IW Meeting 2011-02-24

From Inference Web

Jump to: navigation, search


Meeting info


  • Tim (regrets)

(draft) Agenda

  • Proof combination
  • Tracking down the validators - how many are there? (Three listed at http://inference-web.org/wiki/Tools)
    • Who wrote them, are they still active projects, where are they, how are they used.


  • Scribe
  • Proofs


  • Stephan Zednik
  • Patrick "Mr. OPeNDAP" West
  • Paulo Pinheiro da Silva
  • Deborah McGuinness
  • Cynthia Chang
  • Geoff Sutcliffe
  • Leonardo Salayandia
  • soon - Li Ding
  • I "think" our new visitor is joining too - Yangfan He from Wuhan University in China

Action Items

  • Better management of telecon
    • gotomeeting, webex, titanpad
    • host must be on wire
  • SZ,PW: identify low hanging fruit for OPeNDAP work
  • SZ: GeoData 2011 meeting, chat with Peter
  • SZ, PW, DM, PF: during stephan visit week of March 7th chat about this.
  • Need use cases and requirements
  • Leo to provide feedback about the use of PML in WDOit, WDOs, and SAWs.



The small one is the original EP proof. The big one is combined with part of Metis' proof. See how DIFFERENT (not better or worse) they are. That's what we are trying to achieve - creating new proofs that are different from the originals.

The combined one is also different from the original Metis proof.

The happy news is that even with single step combining we are getting what we want.

The next phase is to go to multiple step combinations, and Cynthia and I will proceed with this next.

I will get the minor update that Cynthia mentioned this week or Monday I guess (Cynthia?). I'll check it, and then make sure we (Cynthia and I) agree on the greedy algorithm. Then it's back to Cynthia (sorry - she really does all the hard work), depending on her time.

The deadline for PxTP (where I will submit the combing work) has been announced - May 2nd


OPeNDAP = Open-source Project for a Network Data Access Protocol

  • is an organization that provides software that implements the DAP standard

DAP = Data Access Protocol Hyrax is an opendap server

  • Composed of two parts, a front-end server (OLFS) and back-end server (BES)
    • OLFS handles DAP URLS, BES loads data and generates DAP objects
    • OLFS - RESTful Web Application, handles DAP URLs, hosted as tomcat servlet
    • BES - Reads local data files, converts into DAP Object, can perform server-side functions (aggregation, etc)
    • OLFS communicates with BES via the PPT protocol

provides support for multiple formats including netcdf, hdf, jgofs, etc.

reads data, converts to a dap object, send over the web

Example DAP URLS:


Before the DAP request is made

  • the data is collected by the sensor
  • the data is stored in a particular format with certain metadata
  • the data is processed, such as what we have in the CHiP pipeline
    • can be multiple processing steps
  • the final data is stored on disk somewhere and is made available to the DAP server

How can we add additional provenance information with the resulting OPeNDAP request?

  • Ancillary attribute file that contains a URI to the provenance model
  • Attribute added to the data file itself (not recommended because this would mean an additional step by the data providers)
  • Ancillary attribute file that contains the provenance information itself (not recommended because the provenance information can be expanded)
  • DAP servers need to add information to the DAS/DDX to include:
    • version information of the DAP server
    • the constraint expression used to create the new DAP object
    • the aggregation information used to create the new DAP object
      • union, join new, join existing
      • what data came from what dataset
  • Currently ancillary data is only DAS object. Need to be able to make this a DDX object that can contain XML.

NcML Examples

<netcdf xmlns="http://www.unidata.ucar.edu/namespaces/netcdf/ncml-2.2"> <variable name="time" type="int" >

    <attribute name="units" value="months since 2000-6-16 6:00"/>
    <attribute name="_CoordinateAxisType" value="Time" />
    <values>0 1 2</values>
  <aggregation dimName="time" type="joinNew">
  <variableAgg name="T"/>    
   <netcdf location="time0.nc" />
   <netcdf location="time1.nc" />
   <netcdf location="time2.nc" />


skype notes

<netcdf xmlns="http://www.unidata.ucar.edu/namespaces/netcdf/ncml-2.2">

 <aggregation dimName="time" type="joinExisting">
   <netcdf location="jan.nc" />
   <netcdf location="feb.nc" />


<netcdf xmlns="http://www.unidata.ucar.edu/namespaces/netcdf/ncml-2.2"> <aggregation dimName="time" type="joinNew">

 <variableAgg name="T"/>
 <netcdf location="time0.nc" coordValue="0"/>
    <netcdf location="time1.nc" coordValue="10"/>
    <netcdf location="time2.nc" coordValue="99"/>



<OPeNDAP-Version><BES prefix="/"><library name="bes">3.9.0</library><module name="dap-server/ascii">4.1.0</module><library name="libdap">3.11.0</library><serviceVersion name="dap"><version>2.0</version><version>3.0</version><version>3.2</version></serviceVersion><module name="freeform_handler">3.8.2</module><module name="fileout_netcdf">1.1.0</module><module name="hdf4_handler">3.9.1</module><module name="hdf5_handler">1.4.3</module><module name="netcdf_handler">3.9.2</module><module name="ncml_module">1.1.0</module><module name="dap-server/usage">4.1.0</module><module name="dap-server/www">4.1.0</module><module name="xml_data_handler">1.0.0</module></BES><OLFS><lib><name>olfs</name><version>1.7.1</version></lib></OLFS><Hyrax><lib><name>Hyrax</name><version>1.6.2</version></lib></Hyrax></OPeNDAP-Version>

<serviceVersion name="dap"><version>2.0</version><version>3.0</version><version>3.2</version></serviceVersion>


Personal tools