Completed Actions

  • 20160629-01, Perform self-assessment on cost and bottleneck for identifying user/payload of given IP/workernode and time:
    • Alice: Self test took ~2h. 10 jobs and 3 users were identified for a single WN over 3h, with the corresponing source of submission, executable and input. The logs are kept forever but represent a big data challenge
    • Atlas: Given an IP address it is possible to query BigPanda to know which payloads have run on the specific WN.
    • CMS: Test not perfomed but development ongoing to keep all jobs records in a Kibana dashboard forever
    • LHCb: Shifter can identify the user and the payload by manually checking jobs on the portal. Only experts can directly query the database
    • All VOs seems to store enough logs and can query them within reasonable delays, but data mining can be diffucult and expensive
  • 20160510-01: Compare produced logs to recommentations:
    • Alice follows the high level recommendations of the first document and keeps all information mentioned in the second document. However, the format used is different and Alice does not use syslog but an internal messaging solution
    • CMS relies on GlideinWMS, which may be open to suggestion, but not a complete rewrite is impossible
    • Agreement that we might find parts of each VO logging infrastructure that might be enhance, but that we should no try to rebuild a new logging system
  • 20161123-03: Vincent: Investigate possibility of HTCondor testing cluster at CERN Site
    • A small test HTCondor cluster with Singularity installed has been deployed at CERN
  • 20161123-04: Brian: Ask CernVM to include Singularity (non-SUID)
    • Singularity has been added to CernVM 4 Beta
  • 20170201-01: Prepare presentation on pilot job/payload data workflows & models
  • 20170201-02: Vincent: Announce access details for the CERN HTCondor test cluster
    • Ben explained to the mailing list how to access that cluster, see the mail archive
  • 20161123-01: Vincent,Maarteen: Investigate security review for Singularity within EGI
    • Several institutes contacted, but no positive answers
    • CERN has started  a security review