Caltech Effort for the CMS Software and Computing WBS

This is a brief overview, and some background, for the information contained in the Microsoft Project file Caltech_WBS.mpp

Tasks Summary

Caltech has played, and will continue to play, a leading US-CMS role in the investigation and support of software and hardware systems to meet the needs of the CMS experiment, particularly in the areas of networked databases, large scale simulations of events for physics studies, and OO software prototypes and installations. These activities are vital for determining the scope of the LHC computing challenge as it relates to CMS. Notable achievements include the meeting of the 100MB/second object database writing CMS milestone in June'99, the production of >1,000,000 fully simulated di-jet events, the development of the MONARC simulation program, and the deployment of the GIOD database at several CMS collaborating institutes. In the future, studies of the modes of sharing data between collaborating institutes and CERN, prototyping components of the computing models, support for the installation, maintenance and development of the latest CMS software, will all be undertaken by the Caltech group.

Support activities

Caltech has made significant and crucial contributions to the development of the CMS "ORCA" reconstruction program, most notably in the areas of object persistency and muon detector reconstruction. Another major activity is the Virtual Rooms videoconferencing infrastructure and service, which is now widely used throughout HEP, and in particular by the LHC experiment collaborations.

Activities planned for the future include installation of, and documentation of the procedure, a complete CMS software suite on a server at Caltech, and the coupling of this to substantial (1 TByte) disk and tape stores, use of this system in continued studies of large scale Objectivity-based object data storage and access, detailed simulations of promising candidate baseline computing models in MONARC or a follow on CMS Computing Models project, federated database tests in the WAN with CERN, FermiLab and SDSC, integration of collaborative working (groupware) features in the CMS User Analysis Framework (UAF) as well as continuing management tasks in the US-CMS software project.

Research activities

Latterly, the GIOD project has focussed on prototyping large scale leading edge systems (using the 256-CPU Caltech Exemplar, the Caltech HPSS, a TeraByte scale Object database containing simulated CMS events, local and wide area ATM networks) as prototypes for the CMS computing model. More recently, Caltech has taken a leading role in the MONARC project, whose goal is to develop feasible baseline computing models for the LHC experiments, with a major contribution being the development of a Java-based models simulation tool. An upcoming activity is research into the interface between the object model and the underlying persistent storage mechanism for the objects, in preparation for the CMS milestone in 2001 which calls for a decision on the underlying database to be made. The GIOD Project will continue in 2000 as "GIOD II", focussing on CMS specific aspects of distributed object data access prototyping and modelling.

In addition, Caltech have embarked on two new projects: the "Particle Physics Data Grid" (PPDG) project, a collaboration of all the US HEP labs, which is investigating grid-based middleware for particle physics experiments with an initial milestone of demonstrating 100 MB/second site-to-site transfers of HEP data, and the "Accessing Large Data Archives in Astronomy and Particle Physics" (ALDAP) project, which, together with collaborators on the Sloan Digital Sky Survey, seeks to develop query optimization techniques, data clustering schemes, and agent-based query brokers to expedite physics analysis of particle physics and astronomy data.

Personnel

Name	Function	Initials	Work areas	Funding source
Harvey Newman	Group Leader	HBN	GIOD,MONARC,ORCA,ALDAP, PPDG	DoE Base
Julian Bunn	Senior Scientist	JJB	GIOD,ORCA,ALDAP,PPDG,UAF	Caltech CACR & ALDAP
Takako Hickey	SW Engineer	TH	MONARC,PPDG	DoE MICS(NGI)
Iosif Legrand	SW Engineer	IL	MONARC	DoE Project
Rick Wilkinson	SW Engineer	RPW	GIOD,ORCA	DoE Base
Vladmir Litvin	SW Engineer	VL	ORCA,PPDG	DoE Project
Philippe Galvez	SW Engineer	PG	UAF,ORCA	DoE Base
Gregory Denis	SW Engineer	GD	UAF	DoE MICS(Videoconferencing)

PPDG

Primary work involves setting up the infrastructure at Caltech for the PPDG first milestone: a 100 MB/sec point-to-point test of transfers of large amounts of data. This involves:

• Installation of 1 Tbyte RAID array [JJB]
• Installation of Sun E250 server to support the RAID array [JJB]
• Installation of PPDG middleware to support the tests [TH]
• Perform site to site tests with candidate codes [HBN,JJB,TH,VL]

ORCA & Reconstrction/Analysis Software

As contributions to the overall ORCA effort and local ORCA work, Caltech includes:

• Development of the object persistency scheme [RPW]
• Installation, support and maintenance of CMS software at Caltech, including re-packaging and documentation of the setup to facilitate similar installations at other US institutes [RPW,VL]. The packages involved are:

· cmsim

· CLHEP

· ObjectSpace

· Objectivity

· ORCA

· OSCAR

· FAMOS

· HepODBMS

· HepInventor

· OpenInventor

· OpenGL

· NAG_C

· MasterSuite

· IGUANA

· Qt

· GEANT

· Jetset

· cernlib

· SCRAM

• Population and maintenance of an Objectivity database containing data created by ORCA, both in production runs at CERN, and in ad-hoc production runs at Caltech (including on the Caltech Exemplar and Sun Enterprise 250) [JJB,RPW,VL]
• Instrumentation of the low-level ORCA I/O with network analysis probes in order to develop ORCA versions that optimise WAN access to remote object data [VL,PG]
• Development of simple Java-based analysis tools for ORCA data in the database [JJB]

ALDAP

The ALDAP project will investigate (principally) database query optimization schemes for large amounts of particle physics and astronomy data held in Objy databases. Some of this work will be of direct benefit in relation to optimization of queries on CMS object data (from ORCA):

• Development of query optimization techniques for particle physics object data [HBN,JJB,RPW]

MONARC

The primary tasks are:

• Development of the Java-based MONARC simulation software [IL]
• Analysis of MONARC models [TH,VL]

UAE

The User Analysis Environment in CMS will require enhancements for group working and remote collaboration. The primary thrust of the Caltech effort:

• Integrate, evaluate and improve collaborative working features in the CMS UAE tool suite [PG,GD]

GIOD, GIOD II

The GIOD project will continue to a second phase (GIOD II) in 2000. GIOD II will focus on CMS-specific aspects of the LHC data analysis problem, with particular emphasis on prototype database implementations and tests in the WAN.

• New tests of Objy client applications across the WAN (with FNAL and SDSC) [JJB]

• Preparing the GIOD project report in early 2000[JJB,HBN]
• CMS focussed work on systems modelling (with heavy reliance on the MONARC results and simulation tool):
• Sufficiently realistic I/O latency and queuing representation: Implement in the simulation a set of data containers for prototype datasets, to get at the issues of caching, paging and data clustering. One would have to abstract (if not represent more directly):reference to a schema, locks, container opening latency, tape data set access latency and queueing, disk I/O request latency and queueing, etc. Implement or represent paging, so that can see the effect of database page faults and some of the associated latencies.
• Get at the data access patterns (e.g. ORCA3; prototype analyses built on ORCA3)   Extract from a running prototype, the information on the data access patterns: what is referenced in what order. This means instrumenting some prototype code. Abstract and feed the information into a Model equipped with the features in (a).
• Implement startup time for transfers over the LAN and WAN. Use work on "enhanced" TCPs to characterize the startup latencies and throughput limits for network transfers [ongoing outside of MONARC; ready early in any Phase3].
• Once set up gauge actual system latencies, represent users as presenting both jobs and simultaneously other network loads.
• Implement multiple job queues, group queues in different "performance classes" mapped (for example) on subsets of the CPUs. Set up parameters for job execution (e.g. how many jobs executing in each performance class).
• Implement parameters for "limits": turnaround time-limits for jobs in each class. Queue-length limits. Limits in the number of "pending" processes, etc. On a limited-reached condition set up an error-handler, which can start simple (E.g. "Stop !") and get more sophisticated-and-useful with time.
• Set up an analysis framework (as it was called) to successively run a series of jobs. Set up a workload corresponding to a desired use-case [by this step in the work plan, the work-load should be a multiuser one.] The first jobs will likely run to a "failure" condition, then another job is launched with reduced workload or with an increase in some resource(s). This continues until a "task completed" situation is reached. Once in the range of "success" (with small workload or unconstrained resources), try to optimize by getting a larger ratio of work done to resource invested, in a fixed time interval. [Consider giving "points" for better turnaround times in in each performance class, if that is relevant].

Julian Bunn, November 1999