The Bedework system supports both personal and group calendaring and scheduling and public events.

Introducing Bedework

Bedework is an open-source enterprise calendar system that supports public, personal, and group calendaring. It is designed to conform to current calendaring standards with a goal of attaining strong interoperability between other calendaring systems and clients. Bedework is built with an emphasis on higher education, though it is used by many commercial enterprises.

You may choose to deploy Bedework for public events calendaring, personal calendaring and scheduling, or both. Bedework is suitable for embedding in other applications or in portals and has been deployed across a wide range of environments.

<iframe width="560" height="315" src="https://www.youtube.com/embed/nYZoufGfV6c" frameborder="0" allow="autoplay; encrypted-media" allowfullscreen></iframe>

1. Overview

1.1. Features of Bedework

  • Java: Written completely in Java, Bedework is system independent. This version compiles and runs in Java 11.

  • Standards based and interoperable: Interoperability with other calendar systems and clients by way of standards compliance is a fundamental design goal of the Bedework system. The following standards are supported:

    • iCalendar support (rfc5545)

    • iTIP (rfc2446)

    • CalDAV (rfc4791)CalDAV scheduling (draft)

    • CardDAV v.4

    • VPoll (draft)

  • CalDAV server : a full CalDAV server is a core feature of Bedework. It can be used with any CalDAV capable client and has been shown to work with Mozilla Lightning, Apple’s iCal, Evolution, and others.

  • CardDAV server : Bedework provides a CardDAV server providing personal and public contact stores for use in the personal client. A stand-alone JavaScript address book application is included with the personal web client suitable for deployment in other web applications such as email web clients.

  • Web clients :The Bedework web clients provide access to public events in guest mode and to public and personal events in authenticated mode. All web clients are easily skinned allowing a high degree of customization.

  • Public calendar suites: Public events are displayed using "calendar suites" allowing multiple organizations to maintain their own public views of events with whatever degree of visibility is appropriate. A Bedework public calendaring installation may have one or many calendar suites. A calendar suite provides a customized view of events, custom theming, and control over how events are tagged by event administrators.

  • Public calendar feeds and embeddable JavaScript widgets, supporting ical, jcal, xcal, json, and XML.

  • Personal calendars: Bedework provides a web client for personal and group calendaring including scheduling. Using CalDAV desktop clients, users can see a fully synchronized view of their personal and subscribed events between their desktop client and the web client.

  • Administrative client for public events: Public event entry and maintenance is carried out through the administrative web client. The system supports three roles: Super Users control global system settings including user and calendar maintenance and the setup of calendar suites. Calendar Suite Owners can modify the settings of their calendar suite, and Event Administrators can add and edit events for the administrative groups to which they belong.

  • Public event community submission: Bedework provides a web client for submitting events to a public queue allowing members of your community who are not event administrators to suggest public events.

  • Public events registration: Bedework supports registration for public events with waitlisting, registrant caps, and notifications. Custom registration fields can be built and reused for any registerable event.

  • Account Self-registration: This client allows users to register for a Bedework calendar account to register for events or other features.

  • Highly customizable look and feel and standards based: The web clients are themed using CSS and a theme settings file, or by deeper manipulation of XSLT. Designers can theme Bedework for multiple clients and uses, without involving your programming staff. Bedework comes with skins for producing the web clients, data feeds, and displays suitable for handheld devices. Bedework provides a widget builder that makes it easy to embed dynamic event listings on static web sites.

  • Database independence - Hibernate : The core of the calendar uses Hibernate for all database transactions giving support of many database systems and enterprise level performance and reliability. A number of caching schemes are implemented for Hibernate including clustered systems giving further options for improving availability.

  • Sharing : Full CalDAV access control is available allowing the sharing of calendars and calendar entities based on authentication status and identity.

  • Scheduling : Bedework supports scheduling of meetings including invitations and their responses. Caldav scheduling (still in draft) is supported. Freebusy is supported and the busy time is displayed as attendee lists are built. Access control allows users to determine who may attempt to schedule meetings with them.

  • Import and export : Events can be imported and exported in iCalendar (RFC2445) format. This provides an option for populating the calendar from external sources. A dump/restore utility provides a means to backup and restore xml data files.

  • Calendar subscriptions : Users may subscribe to calendars to which they have access, including public and personal calendars. iCalendar data feeds are available from the public web client.

  • Multiple calendars : The core system supports multiple calendars for users and for public events.

  • Tagging & Filtering : Events and folders can be tagged by any number of categories and event views, feeds, and widgets can be filtered by these.

  • Internationalization : Bedework supports full internationalization, including multilingual content (though multilingual content creation is not yet exposed in the web UI).

  • Data feeds: Bedework produces RSS, Javascript (e.g. json), iCalendar, and XML feeds. Custom feeds can be developed by writing an XSL skin. Feeds can be filtered by category or creator, and a feed and widget builder is available to help end users and developers design public feeds and embeddable event widgets.

  • Portal support : Bedework has been shown to work as a JSR168 portlet in Jetspeed, uPortal and Liferay using the portal-struts bridge.

  • Timezone support : Full timezone support is implemented. There is a set of system defined timezones based upon externally available sets of timezone definitions. In addition users are able to store their own timezone definitions.

  • Recurring events : Extensive recurring event support is available via CalDAV and the web clients.

  • Event references : Users may add public event references to their personal calendars. Event references can be annotated by the user.

  • Pluggable group support : Bedework uses a pluggable class implementation to determine group membership for authenticated users allowing organizations to implement a class which uses an external directory. The default class uses internal tables to maintain group membership. Different implementations can be used for administrative and personal use allowing the separation of any given users roles.

  • Container authentication : There is no authentication code in Bedework. Rather, Bedework behaves as a standard servlet, and all authentication is carried out through external mechanisms. Standard container authentication (via Tomcat or Jboss) and filter based Yale CAS authentication are used in production. The quickstart comes packaged with the Apache DS server against which the quickstart deployment of Bedework authenticates. This server can be used in production, though many deployers opt to authenticate against their organization’s existing directory.

  • Support for other calendar systems and clients : It is possible to access an entire calendar with a single url. This can be used to subscribe to a Bedework calendar from Google or Outlook. Bedework can also take advantage of the richness of CalDAV capable desktop clients such as Apple’s iCal and Mozilla Lightning.

1.2. Release Notes

1.2.1. Release 4.0.3:

This release has been a long time coming largely because it involved a significant amount of restructuring. We’ve moved away from ElasticSearch because of licence issues, and completely overhauled the deployment process.

Part of the refactoring is to split off the read-only system from the read-write components. This would allow deployment of a much lighter weight service for feeds and web-presence. This work is not yet complete.

The bulk of the rest of the work is to use jboss modules to deploy all code once only and have that available for all services. This reduces memory usage and startup time.

1.2.2. ElasticSearch replaced with OpenSearch

This release switches from ElasticSearch to OpenSearch due to the licensing issues with ElasticSearch after version 7.10. This will require a reindex of the data but that would be required anyway as we have made too big a jump between versions for an automatic index update to work.

This has some links to other articles and a search will reveal many others.

While there are reservations about an Amazon supported version it appears to be in their best interests to truly support open source, so - at least for the time being - we have access to a supported Apache 2 licensed search engine.

1.2.3. New wildfly galleon feature pack install.

See Install bedework as a wildfly feature for the new way of installing a working system. This is significantly easier than previously.

1.2.4. Missing tables in database

This fix is needed for attachments to work correctly. It probably does not affect public events as attachmentsare not (much?) used. A significant - but long-standing - bug was discovered. Override attachments were not being stored as the table and schema entries were missing. Updating will require adding the table to the database (or recreating the data from the XML dump).

Additionally, ensure the OpenSearch schema is up to date (wildfly/standalone/configuration/bedework/opensearch) and reindex after the system is updated.

To fix attachments in postgres.

  • BACK UP THE DATABASE

  • log in to psql

  • select the calendar database and execute the following commands. This may (should) be done ahead of time.

CREATE TABLE bw_eventann_attachments (
    eventid integer NOT NULL,
    attachid integer NOT NULL
);

-- Change bedework to whatever you have as owner for your db
ALTER TABLE bw_eventann_attachments OWNER TO bedework;

ALTER TABLE ONLY bw_eventann_attachments
    ADD CONSTRAINT bw_eventann_attachments_pkey PRIMARY KEY (eventid, attachid);

ALTER TABLE ONLY bw_eventann_attachments
    ADD CONSTRAINT bw_eann_attach_fk FOREIGN KEY (attachid) REFERENCES bw_attachments(bwid);

ALTER TABLE ONLY bw_eventann_attachments
    ADD CONSTRAINT bw_eannattach_eid_fk FOREIGN KEY (eventid) REFERENCES bw_event_annotations(eventid);

Ensure all configurations are up to date, especially the OpenSearch schema then reindex the data.

Changes that might need to be made

If you deploy your own copy of bedework wars and ears theer are changes that might affect you. Most of the properties which were chanegd by the bedework deployment process have been replaced with run-time wildfly properties or by values.

For example, when the xsl war was deployed a property in WEB-INF/jboss-web.xml was replaced.

  <context-root>${app.context}</context-root>

has been replaced with

    <context-root>/approots</context-root>
Other Bug Fixes

A further significant bug discovered soon after release of 3.13.2:

Indexing. Referenced entities - e.g. locations, were not getting restored in some cases - see https://github.com/Bedework/bw-calendar-engine/commit/58df20469660d4fe4f2fcef15992147979e3717c#diff-4fb4cfb2524a3a8ea92fc90a4fc31b60

Further bug with recurrences. In caldav if an override was deleted from the event it was not deleted from the system.

Scheduling bug fix Trying to invite a new bedework user to a meeting could result in an NPE

Category bug fix For personal events, multiple new categories in an event were not getting indexed correctly - only the last one. They were created correctly in the database - a reindex will fix any missing categories.

This does not affect public events.

Core RDATE only recurring events were not being indexed correctly - instances didn’t show up.

WebClient Filter out Inbox from result WebClient Events were being moved incorrectly (from Inbox) causing indexing issues. WebClient Fixed the timeview - events near the end of the day appeared in the next days cell.

Updates

Moved most of the deployment into wildfly modules This is to prepare for splitting the deployments into read-only web applications (public calendar, etc) from read-write (admin etc).

Note that this has led to a number of changes to the bw script. The actual web apps rarely need redeploying during development - individual system modules can be built and deployed on the server.

New quickstart deployment method. The quickstart will only be required for development purposes - or for reference to the source. Maven and git are no longer required to install wildfly but are required for the quickstart download.

Make basic config options constants A number of path elements - e.g. the name of the user root collection - are in basic system properties - then flagged with "do not change".

These are being changed to constant strings. Changing the internal path element name is likely to be a problem and having to locate the current config in some places is also a problem.

This does not prevent sites setting the display name to something else.

The properties in question are all those that were in basicSystem.xml, which used to populate BasicSystemProperties.

Updated to wildfly 26.0.1.FINAL appears to have better memory handling.

Updated ical4j brought it close to the Ben Fortuna version.

JsCalendar largely supported.

Timezone server * switch to h2 from leveldb which had too many undesirable dependencies. * Move some config out of the zoneinfo directory into the specified data directory. Changed that path to be effectively one level up. * Bug fixes for problems encountered when updating the data.

Refactoring as part of using wildfly modules. * Move Args class from util conf package to util package.

Many library version updates

1.2.5. Release 3.13.2:

Making a serious effort to get rid of ide warnings. Removing the trivia lets the important stuff stand out. Starting work on implementing new jscalendar and jscontact representations and the jmap protocol.

Changes to log file processor/analyzer. Can produce summary of addresses per ip-address/domain.

  • Bring libraries (jackson + spring) up to the current up to date

  • New jsforj module to parse and generate jscalendar amd jscontact structures.

  • Major internal refactor ready for embedding jsCalendar support:

    • Moved bw-calendar-engine-ical to bw-calendar-engine-convert

    • Added packages in that module for ical, jcal and xcal.

  • Cleanup:

    • Changed a number of internal api calls to use response objects and return errorcodes rather than throw exceptions. Where an exception is the only way out use RuntimeException.

    • Where methods rarely throw an exception - or the exception is the result of a truly hosed system - throw runtime exception instead. Cleans up code and we can concentrate on the issues that matter.

  • More fixes to bw script.

  • Performance

    • Dropped a wait in indexing mark-transaction which was adding a significant amount of time to calls.

    • Figured out how to handle provisioning a new account when we have a read-only svci. Allowed reinstating read-only for caldav read-only methods.

    • Reconfigured and rewrote some of the JMS code to allow asynch sends. Required update of a library version.

  • bw-util

    • move cli libraries into new bw-cliutil project

    • move bw-util-struts into bw-calendar-client-util

    • Split into a number of util projects

  • bw-util-logging

    • Allow setting of log level.

  • webdav

    • Fixes to report/propfind - allprops and propname were not being handled correctly.

  • Indexing

    • Use scroll search for multiget query

    • Delay indexing to end transaction call. Allows for greater efficiency and also less likelihood of index inconsistencies.

    • Fixed mapping so that queries work better against all_content.

  • Other bugs.

    • Fixed alarm equality checks. Bad comparisons for some fields.

    • A few scheduling and sharing fixes.

    • Add a recurrence instance to db for overrides. Need for link back to master.

    • Scheduling: fixes for attendees only on override.

    • Fix cleanup of description and summary strings. Was inserting escaped newlines.

1.2.6. Release 3.13.1:

There was a long standing bug in category handling for updates. An attempt was made to preserve default categories for calsuites when an event is updated. For example if an event is suggested and accepted the accepting calsuite has its default category added to the event.

This code was being applied to collections which made it impossible to turn off a default category added to, e.g. an alias, by mistake.

This release also introduces a new authenticated public context. This is intended to be used for departmental calendars for example. There were a number of changes needed to make this work but most of the work will come in setting up the calendar collections and aliases. Documentation and examples will follow later as always.

  • Drop the explicit reference to maven profile bedework-3 in the bw script. Fix that script to allow -P <profile> and use that in the install script to use bedework-3
    This allows us to specify a default profile that differs from the bedework-3 profile.

  • Additionally - add support for a .bw file in the user home which allows setting of the profile. See Default Maven Profiles

  • Wildfly galleon installer 4.0.3.Final stopped working soon after the last release. Updating to a later version and hoping this won’t break.

  • Updated google maps url generation to use location combinedValues property

  • If the location map url is "NO-LINK" (without quotes) then no link will be generated.

  • Remove BasicHttpClient. This necessitated some config changes -

    • authCardDav.xml and unauthCardDav.xml in bwengine now have a url rather than host, port and context.

    • notify/notify-config.xml changed - removed host, port, context. Added URI

  • Web client changes

    • Change how we select the mode of working -

    • Config for user and submission clients require new entry - <readWrite>true</readWrite>

    • Add a new authenticated public client. This should allow limited read-only access to views of the data. Users will be added to admin groups to control the access.

    • Removed bwapptype parameter from web.xml files. Value is duplicated in client configs.

  • Bug fixes

    • User TermsFilterBuilder for collections. Was generating partially working query

    • change "|" to " or " in xsl - was not encoded - broke some pages

    • Problem related to timestamp handling was causing ES version errors.

1.2.7. Release 3.13.0:

This release mostly consists of upgrades to almost the latest ElasticSearch (always a moving target), the currently latest wildfly and to Java 11 the current LTS release.

Installing the quickstart requires that you first install docker if you wish to have a quickstart image of OpenSearch installed.

There have additionally been some minor changes in configuration and the addition of a tool feature to help in calendar suite creation.

Beyond that there is very little functional change since the last release. However, note that the move to the latest ES required a complete rewrite of the query and indexing modules.

  • Upgrade to ES 7.2.0

  • Upgrade to wildfly 17.0.1.Final

    • Use galleon to install - allows updates

    • Don’t use wildfly modules for deployed ear dependencies.

  • Require java 11.

    • Many changes to build. Much of the XML support is removed from java core.

    • Updates to maven plugin versions

  • Minor changes

    • Add an error log handler

    • Reduce noise in logs

      • Remove bogus elements from config files

      • Remove ldap group member so we don’t get annoying error messages

    • Add auth user update to cli tools

    • Fix NPE when editing auth user that doesn’t exist

    • Some fixes for travis build

1.2.8. Release 3.12.7:

  • Fixes to install script

  • Library updates

    • Update http version to avoid security issues

    • Add missing dependencies to eventreg

  • Add tzsvr data to quickstart

  • Changes to tz conversion - still broken

  • XSL fixes - missing approots

  • Client

    • Remove empty x-properties on event update

  • Log processing

    • Was missing log prefix in parser

    • Add more checks for same task

  • Deployment

    • Use deployment base

  • Sync

    • Use last-modified if etag not present

  • Watch for null x-properties in event list. Can be caused by deleting them in db.

  • Indexing

    • Don’t index x-properties - can be large

1.2.9. Release 3.12.6:

  • Library updates

    • Update servlet api version

    • Update jackson version to avoid security issues

    • Update http client version to avoid security issues

  • Log analysis

    • Updates to generated figures and some analysis of access logs

  • Sync process

    • Update category prop updater to fix NPE

    • Add callback method to fetch location by combined value. Use it when updating or adding an event.

  • Indexing

    • Fix location mapping - was missing combined field.

  • Install

    • bwcli wasn’t being built by install script

  • Restores

    • Restores were failing because the fake event property calpath code was getting an NPE - no principal. Fixed it so principal isn’t needed. Caused cascading updates up the stack. Dropped the principal object where possible. Generally only need the href.

    • Resource content handling was broken in restore. Should just set the byte value and create the blob when we have a session

  • Client

    • Add action to clear any principals notifications

    • Fix feeder main/listEvents action - now works

  • Others

    • Svci pars wasn’t handling the readonly flag properly. Worked for unauth but wasn’t turning on readonly for authenticated methods.

    • Drop loader-repository elements from (some) jboss-app.xml

    • Better error messages when building index docs and in AccessUtil

    • Watch for null home in CalSuites

    • Response: Add method to set Response status from a response

1.2.10. Release 3.12.5:

  • Logging

    • Add a bunch of jsonIgnore to the Logged interface to stop the fields turning up in json.

    • Fix error methods. Use exception message as first param.

  • Client

    • Cache default filters for ro client. Use calsuite as key

    • Cache user collections in session. Use calsuite group as key

    • NoopAction extended MainAction. Should not as it retrieves a lot of unused data.

    • Make session timeout for /cal and /soedept configurable and default to 5

  • Don’t store collection in BwCollectionFilter. Was never used. Just store path as entity

  • Fix FlushMap in utils. Current fetched value was not discarded.

  • Fix bw script - was missing some of the newer modules

  • BwLastMod:

    • Add JsonIgnore to getDbEntity or we get a loop.

    • Set the db entity when we clone or we get an NPE

1.2.11. Release 3.12.4:

  • Fixed a few bugs.

    • BwResourceContent bug below

    • Suppress a request-out log message unlesss really on way out

    • Index wrapper type for calsuite - not calsuite itself

    • Try to force refresh after adding calsuite

    • HttpUtil POST produced Accept rather than Content-type

    • Bad forward in add calsuite produced bogus error message

  • Updated log analyzer so results are easier to read.

  • Factor deployment modules out of bw-util into new bw-util-deploy

1.2.12. Release 3.12.3:

  • Added new cli command to analyze log data.

  • Add new REQUEST-OUT log message for analyzer

  • A number of bug fixes

    • Touch collection on update of acls - was not getting indexed

    • Calling wrong indexer to update resource content

    • Wasn’t saving entity in response from indexer

    • Add cache to SvcSimpleFilterParser so we don’t repeatedly attempt to fetch children of collections.

    • Should be returning an empty array when the event is not found

    • Was calling wrong method to fetch location for update

Note: A bug was discovered almost immediately. The commit is at https://github.com/Bedework/bw-calendar-engine/commit/c83e77e3f5ceb990029b84ca7440af83fdc4e568 and a patch:

Index: bw-calendar-engine-facade/src/main/java/org/bedework/calfacade/BwResourceContent.java
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
--- bw-calendar-engine-facade/src/main/java/org/bedework/calfacade/BwResourceContent.java	(revision b248db13b030a73828d7b8c9428dda9ebf262a0c)
+++ bw-calendar-engine-facade/src/main/java/org/bedework/calfacade/BwResourceContent.java	(revision c83e77e3f5ceb990029b84ca7440af83fdc4e568)
@@ -187,14 +187,11 @@
       while((len = str.read(buffer)) != -1) {
         b64out.write(buffer, 0, len);
       }
+      b64out.close();

       return new String(baos.toByteArray());
     } catch (final Throwable t) {
       throw new CalFacadeException(t);
-    } finally {
-      try {
-        b64out.close();
-      } catch (Throwable t) {}
     }
   }

1.2.13. Release 3.12.2:

  • Added new cli command to allow refresh of tz data.

  • Widespread changes to remove references to log4j. All localized in one source file (and a few poms for runnable code).

  • Use asciidoctor to generate this document.

1.2.14. Release 3.12.1:

Searching for contacts/locations
  • In the admin and event submissions clients replaced simple drop down with a search interface. Requires back end support for the search )a restful style with json response).

ES only read-only clients.
  • Implement an ES only read-only interface. The public client can be built without any hibernate support as it doesn’t interact with the database. This required at least:

    • Minor API changes

    • Indexing of more entities - principals, calendar suites, preferences, filters.

    • New core interface implementation which only handles the read only methods.

    • Refactored the core to remove a callback. Also to spilt off the read only code.

Split out ES indexes
  • Significant change to indexing to try to resolve the contacts issue and prepare for upgrade.

  • ES v7 will allow only one type per index. To prepare the index was split into many. Requires a doctype parameter to be added to most calls, significant changes to the (re)indexing process and other associated changes.

  • Almost all calendar engine classes were affected in some way - mostly relatively minor.

  • Configuration changes: no longer have a public/user calendar name. The location of the mappings is a directory - not a file and there are multiple mapping files under directories named with the lowercased doctype name.

Use ES only read-only interface for CalDAV read-only methods.
  • The hope is this will provide a significant performance improvement for those methods.

Other changes.
  • Merged pull request from viqueen. Deal with DAV security issue.

1.2.15. Release 3.12.0:

Move to github/maven
  • A number of modules have been replaced with their github/maven equivalents from the 4.x branches. Other than changes for the build process these modules are functionally equivalent. This change was initiated to make some module classes available for externally built plugin modules. The 3.x modules and their 4.x replacements are:

    • rpiutil → bw-util

    • bwaccess → bw-access

    • webdav → bw-webdav

    • caldav → bw-caldav (bwcaldav is the bedework implementation of the interface)

    • bwxml → bw-xml

    • eventreg → bw-event-registration

    • selfreg → bw-self-registration

    • synch → bw-synch

  • Related changes were to build the runnable post-deploy app in bw-util-bw-deploy and run that. Some configuration properties had to be changed to align.

  • Having done the above the master on github for the calendar engineand client is now the current 3.x dev version, there is a 4.x branch for future development and release branches will be created as necessary.

  • The urls for wsdls is changed. e.g. /wsdls/synch/wssvc.wsdl becomes /xmlspecs/wsdls/synchws/wssvc.wsdl. This necessitates changes to configurations:

    • synch/../orgSyncV2.xml

    • synch/../localBedework.xml

    • bwengine/synch.xml

    • bwengine/system.xml

    • eventreg.xml

  • Yet more refactoring was needed. Turns out we had an unbuildable set of modules with bw-xml depending on bw-util for the deployment. Broke out the 2 modules with a dependency on bw-xml as bw-util2

  • Moved all the xsl into it’s own module - bw-calendar-xsl. Thi salso needs changes to configs - all xsl url paths are now prefixed with /approots - the context at which the xsl is deployed. Look for elements appRoots and browserResourceRoots in the configs

Scheduling
  • Fixes to scheduling code to try to ensure pending inbox events get deleted

  • Updates to iSchedule client for later version of httplient. Moved some code out of caldav tester into common utils

Notifications
  • Fix the listeners so they close down without exceptions

Websockets
  • Add code to support websockets for a new experimental streaming protocol (a CalConnect initiative)

  • Many changes to build process - wewbsockets applications cannot be inside an ear file. Now possible to deploy as a standalone war. Websockets endpoint is now a separate module.

  • Websockets moduleacts as a proxy to caldav.

Other
  • Delay getting a change table entry when realiasing. Was intefering with a test in update.

  • Getting deadlocks when deleting tombstoned events. Change the colpath so they disapppear but need a purge process to finally remove them.

  • Tasks collections were not getting created with correct type - nor were they returning a supported component type.

  • Some fixes to the selfreg feature and additions to the cli to drive it.

1.2.16. Release 3.11.2:

Indexing
  • Add a reindex operation which reindexes all the data in place. Used when ES schema changes.

  • Add an indexstats operation to get counts for a named index

  • Add a setProdAlias operation. Rebuild index no longer automatically makes new index prod. This also allows us to back off the index.

  • Extra operations added to cli to reindex and change indexes

  • Fix update of UpdateInfo in ES index. Was doing a string concat rather than an increment.

  • Index individual location fields so they can be searched

  • Add a fetch single event method to the indexer

  • Synch around event cache accesses

Notifications
  • Add a preference to allow suppression of notifications for a user. This shoudl be applied to public-user to avoid a lot of overhead

  • Change logging is now modified. Messages are now logged to audit.org.bedework.chgnote. Requires a change to standalone.xml or the equivalent

Sync and orgSync:
  • Add orgSync connector to sync engine

  • Fully index location sub-fields - add a set of keys for mapping locations

  • New indexer methods to enable searching for particular location keys

  • Allow specification of a mapping key in subscription and in x-property

  • Updates x-calendar xsd for mapping key as param

  • Changes to admin client to allow specification of orgSync

  • Upgrade to httpClient to handle orgSync certs

  • Add further parameters to OrgSync subscription -updated admin client to support

  • Unsubscribe before deleting content to avoid race.

  • Get persisted event on fetch for update

  • Allow for pw without id in subscription - it’s the key in OrgSync

  • Implement setting category on add and update from containing collection.

  • Update was setting datestamps before checking for no changes - was propagated to db entity preventing further updates.

  • Do a better job of setting content-type and encoding for SOAP interactions.

  • Add array of keys to location entity for use by synch process.

  • Fix handling of locations in Synch engine. Add the locKey parameter to the location. It gets propagated to the x-prop for use later.

  • Refresh rate wasn’t getting through. Fixed

Public events admin
  • Try to mitigate errors caused when a validation error occurs on publish. Indexed and db version did not match.

  • Added missing retry action in event submit.

  • Fixed race condition when selecting a group in admin client

  • Fix the eventsPending page. POST was losing the filter

  • Calsuite specific approvers

  • Avoid ConcurrentModificationException in admin client

  • Changes for eventreg

    • Add some commands to cli

    • Use wildfly modules

    • More HttpUtil methods for use in eventreg and sync

    • Fix web.xml and post-deploy for wildfly

  • Use of deleted flag

    • Index the flag

    • Changes to allow DeleteEventAction to just set the flag

    • Searching can filter on deleted flag

    • Add mark deleted button to form

  • Add tool command to set authuser roles

  • Add tool command to add/remove approver for calsuite

Clients
  • Fix errors caused by entry into showEventMore with a new session

  • Switch public client to use href in urls instead of calPath + guid + recurrenceId

  • Last date in header was the same as the first date

Other
  • Removed the principal path elements from the basic config. Changing them is always a bad idea so they may as well be fixed.

  • Use wildfly modules where possible - ensure we get consistent SOAP behavior

  • Further changes for httpclient. Fix to timezones

  • Logging changes to try to reduce output

  • Try to spot ConnectionResetByPeer errors and leave quietly

  • Try to make less noise when a hung session is shut down

  • Avoid tzsvr startup errors - and db should be static

  • Allow setting of session timeout in deploy properties

  • Drop deprecated jboss config

  • Allow setting of soap address in post deploy

  • Try to fix some issues with JMX which surfaced when testing eventreg

  • Add an Events method to calculate instances for recurring event

  • Fix carddav logging

  • Add flag to ifInfo to indicate a dontKill server process. Stops autokill killing off some of the long running system jobs.

  • Fixes to get carddav working again. Most of them backported to 3.11.1

  • Fixes to get vpoll working again. Broke as a result of ical4j upgrade.

  • Add event dumping to the new (incomplete) dump format.

  • Try another approach to stop exceptions when a new user turns up

1.2.17. Release 3.11.1:

  • Change the schema and filter to allow searches on x-properties.

  • Backported carddav changes from 3.11.2

1.3. Issues

  • Modifying a recurring event so that it is not recurring will leave instances indexed.

  • CalintfHelper.getCollection returns null - causes failures higher up.

  • Index waits and refresh could be set to only occur on testing if there is any real performance issue. Generally - in real life - the delay isn’t a problem.

  • Should I be removing the entity from entity caches in the indexer?

1.4. System Overview

Bedework System Architecture

Bedework has a central server architecture and is modular and extensible. It consists of the following components:

  • Bedework core calendar engine

  • Public events web client, called a “Calendar Suite”, for display of public events

  • Public events cached feeds and widget builder, supporting ical, json, and XML and for producing embeddable JavaScript widgets for use on other websites.

  • Public events administration web client for entering public events, moderating pending events from the submissions client, and configuring calendar suites

  • Public events submission web client for authenticated members of your community to suggest public events – these become pending events in the admin client

  • Personal and group calendaring web client with a subscription model to Bedework public calendars, user calendars, and external calendar feeds

  • CalDAV server for integration with CalDAV capable desktop (and web) clients such as Apple’s iCal or Mozilla Lightning

  • CardDAV server that supports contacts for scheduling in the personal client.

  • A JavaScript-only CardDAV address book web client is available for use with the CardDAV server. The address book comes with the Bedework personal web client, and is suitable for use with other web applications (e.g. webmail).

  • TimeZone server for support of timezone information.

  • Dump/Restore command-line utilities for database backup, initialization, and upgrades.

1.4.1. Calendar Architecture

The Bedework system is divided into two main spaces: the public events space, and the personal and group calendaring space which are kept separate by design. Public events are stored below a public calendar root folder and personal calendars are below a user calendar root folder.

Basic Architecture

Personal calendar users (and other clients) can subscribe to public events, but users may only add public events using the Administrative and Community Submissions web clients. For more information about Bedework’s public and personal event calendaring models, see Public Events Calendaring and Personal & Group Calendaring.

1.5. Bedework History and Background

Bedework was established in March 2005 in succession to UW Calendar. In December 2006 Bedework received the Andrew W. Mellon Foundation’s Technology Collaboration (MATC) Award. Since then the project has prospered, and in early 2009, Bedework became an incubator project of Jasig (http://www.jasig.org/) now a part of the Apereo Foundation (http://www.apereo.org).

Bedework is in use by institutions large and small, educational, governmental, commercial, and non-profit.

Bedework is named after the Venerable Bede, a highly influential monk and scholar from the area of Northumbria in Britain who in 725AD wrote the treatise, "On the Reckoning of Time". Bede is pronounced like “bead”, Bedework is pronounced “bead work”.

1.6. Calendaring Standards & Interoperability

The Bedework implementation team participates in the standards process as a member of CalConnect, the Calendaring and Scheduling Consortium, which is "focused on the interoperable exchange of calendaring and scheduling information between dissimilar programs, platforms, and technologies."

Interoperability with other calendar systems and clients by way of standards compliance is an important design goal of the Bedework system.

Bedework is designed to conform to existing calendar standards. For more information about calendaring and calendaring standards, please visit the CalConnect Devguide.

1.7. Underlying technologies

The following lists some of the core technologies used by the Bedework platform:

  • Apache Struts MVC

  • CalDAV

  • Cascading Style Sheets (CSS)

  • iCAL: RFC 5545

  • iCal4j

  • Java Servlet API

  • Java Server Pages (JSP)

  • JavaScript (including jcal, jQuery, jQueryUI, Bootstrap, and others)

  • Portlet API: JSR-168

  • XHTML

  • XML

  • XPath

  • XSLT

2. General Administration

2.1. Reindexing

Opensearch is used for indexing and searching the data. Nearly all complex queries take place against OpenSearch. The public facing web client and feeds use ES exclusively - there are no database interactions. CalDAV read-only methods (e.g. GET, PROPFIND, etc) also interact only with ES.

The database is used only as a transactional store and only the admin clients and CalDAV write methods (PUT, POST etc) use the database. Changes made to the database result in updates to the index which in some cases can take up to a second to appear.

Occasionally the index will get out of step with the database. The database is the authoritative copy of the data. Sometimes - for an individual entity making a trivial change to the data via the admin client can get things back in step. If it’s an event which was being destroyed and it remained in the index a super-user can delete by href from the system tab.

If things are significantly out of step or you are restoring data you’ll need to reindex. Depending on the amount of data present this can take from a few minutes to some hours.

During reindexing the system can continue to run using the previous index. However, any changes to the data may be lost so it is better to suspend any such changes until after reindexing is complete.

A reindex takes 2 steps:

  1. Build a new set of indexes from the data

  2. If no errors make the new indexes production.

These are both manually initiated.

To reindex

cd bw-cli
./target/client/bin/client -id <adminid> -pw <adminpw>
rebuildidx
<some time passes>
makeidxprod all

Before the last step ensure that all phases of the indexing took place. If - for some reason - the client loses touch with the server, you can check the status with

rebuildstatus

The response should include a section which might look like this (for the quickstart data)

Statistics for Public
current status: indexCollection(/public/unbrowsable/submissions/submissions)
Statistics for Public
unreachableEntities: 0
principals: 11
preferences: 1
collections: 71
categories: 62
locations: 5
contacts: 4
filters: 0
events: 3
resources: 10
resourceContents: 10

Note that - other than for filters - there are non-zero counts for everything. If resources for example are 0 check for errors.

3. Public Administrator Features

3.1. Adding a topical area.

3.1.1. First create a category for the topical area.

Or categories if necessary. The normal case is to filter and categorise with the same category.

Go to SystemManage-categories and add any required new category or categories.

3.1.2. Create a system alias.

Go to SystemManage calendars & folders and determine where in the hierarchy you want the topical area to appear. Usually under aliases.

Create a new subscription

  • set the system name

  • display name

  • description

  • set the category or categories for auto-tagging and filtering

  • select "Subscription".

  • Set the url to bwcal:///public/cals/MainCal

  • ensure all other fields are blank and add the alias.

You may need to refresh the display to see the new alias.

Using system aliases in this way ensures consistency in categorisation.

3.1.3. Create a subscription to the new alias.

Switch to the appropriate group then the Calendar Suite tab.

If the new alias is in a folder it may already be available.

Otherwise

  • Select Manage Subscriptions

  • Click + on the top level folder (named the same as the suite)

  • Set the names - probably the same as the alias

  • Add any extra filtering - note a default category will be added for the suite so normally no action is required on categories

  • Ensure the type is Public alias

  • Click Select a public calendar or folder

  • Click on the new alias

  • Add

Now go to Main MenuAdd Event and ensure the topical area is visible.

3.1.4. Add it to the view for the main campus

The above needs to be done for the main campus calendar suite. Then

  • go to the Calendar Suite tab.

  • Select Manage views

  • Select the appropriate view

  • Add the new topical area

Currently, it appears a restart is needed for it to appear (or a long wait). I’ll try to add a flush to fix this.

3.2. Public Events Registration

Bedework supports a public events registration system that allows authenticated users to register for events. Users may view and modify their registrations, such as unregistering or changing the number of tickets they’ve requested. When registration is full, users may choose to be placed on a waiting list. Users on waiting lists will automatically be moved up in the queue if space becomes available.

Registered Event

With the form elements as shown below, administrators can specify how many users may register, how many tickets each registrant may request, and set the opening and closing dates of registration. Administrators can view and modify a registration list and download CSV files of their registrations on-demand.

Register Event Form

3.2.1. Displaying the current status

Clicking on the "View registrations" button will take you to a screen showing the current registrations for the event. This screen also allows the administrator to hold some tickets and to update or remove registrations.

3.2.2. Custom Fields

Administrators may add custom field elements for use during registration. These custom fields are displayed to the user when they register for the event. These fields can be used to obtain extra information from the user as they register, for example dietary restrictions.

Currently the results are only visible in the downloaded registrations.

3.2.3. Data elements

Data about the event is maintained in x-properties attached to the event and provides the following information:

  • Booking window start and end

  • Number of tickets

  • Max number of tickets per person

3.3. Subscribing to a feed for public calendars.

You probably want the incoming events to all be flagged with a specific category or categories. This can be useful for filtering.It makes it easier to include the calendar collections imported by the sync engine. Before carrying out the following steps ensure these categories are created.

For public events there are a few more options available when subscribing.

Process Locations and Contacts: This causes locations and contacts to be moved into x-properties "X-BEDEWORK-LOCATION" and "X-BEDEWORK-CONTACT". The receiving end (bedework) may carry out further actions to validate the location or contact and as a result may set a preexisting location or contact in the event. The x-property is always available for display but this process allows the system to validate the location/contact.

Process categories: This does the same for categories.

3.3.1. Note:

If you don’t select those options then categories and locations will be created in bedework. This is probably not what is wanted as these are uncurated.

By setting the Process…​. flag you ensure that such locations and categories don’t end up in the database.

Suppress deletion of events? This means that if events disappear from the feed they will stay in the database. This effectively turns the feed into a change feed adn can significantly reduce the size of the data. It does mean that events that MUST be deleted will require help from an administrator with sufficient privileges.

3.3.2. Creating a public subscription

As a super user

  • Switch to the System tab and select "Manage calendars and folders".

  • Open up "public" and click the "+" on the cals folder.

  • Set the name - e.g "Athletics"

  • Check off categories for filtering

    • Click on "show/hide categories used for filtering"

    • Scroll down and select a previously created category or categories

  • Check off any categories you want applied on input -

    • Click on "show/hide categories for auto-tagging on input"

    • Scroll down and select a previously created category or categories

  • Mark as a subscription

  • Set the URL in the URl field

  • Set an id/password if necessary

  • Select the "Process Locations and Contacts" and "Process Categories" checkboxes if desired (probably so)

  • Set the location key field name for mapping locations

  • Set the refresh rate in minutes

  • Save

3.3.3. Mapping Locations

Locations created and stored within bedework are broken out into a number of fields. Out of these the following are used to create a combined address value which is used for export and import mapping:

  • Street address

  • Room

  • City

  • State

  • zip/postcode

These values are separated by a comma and a space. Missing values are skipped.

The update process matches the value of the supplied LOCATION property against that combined value.

If there is an EXACT match then the bedework location will be attached to the syncronized event. Otherwise the location will be left in the x-property. Feeds and the front end display will normally display the x-location as the location for the event.

3.3.4. Mapping categories

An equivalent process takes place with categories though the mapping is much simpler. Categories only have a single value. An exact match in the imported event will cause the bedework category to be applied.

The use of bedework categories is important for much of the front-end filtering of events as this is driven primarily by categories.

As an example if we have the alias structure
performances/comedy
and comedy is tagged with the category comedy and performances with the category performances then the imported event needs both categories.

3.3.5. Examples

3.4. Public events authorised users

These are users with rights to create events, perhaps publish them and possibly have access to calendar suite setup.

3.4.1. Creating a user

Go to the "Users" tab and click on "Manage Admin Roles".

If the user does not appear in the list then type the account into the box labeled "Edit admin roles by userid" and hit enter.

All users should have the "Public Events" flag set.

Users who may create but not publish should have the "Content Admin" flag set.

Users who may create and publish will also have the "Approver" flag set.

3.4.2. Updating a user

Go to the same location, "Users" → "Manage Admin Roles", locate the user in the list and click on the "edit" link.

Flags may be set or unset from there.

3.4.3. Updating group membership.

Until the user is added to a group or groups they cannot create events. Each calendar suite will have one or more groups associated with it. Typically there is an admin group associated directly with the suite and a submission group which is a member of the admin group.

Approvers are typically added to the admin group and event creators get added to the submission group.

To add a user or group click on "Users" then "Manage Admin Groups".

Find the group to update and click on the "membership" link.

To add a user enter their account in the "Add member" box, ensure "User" is selected then press the "Add" button.

To add a group ensure "Group" is selected.

To remove a group member, click on the trash icon next to the account.

3.4.4. A problem to be resolved later:

Note that the rights of the users are attached to their auth user entry. So a user is always an approver whatever groups they are in.

There is no provision for a user being an approver in one group and an event creator in another. At some point we may move the rights to the associated groups so that approval rights will come about because of group membership.

4. System Components

5. Bedework Sometime

This is a fork of the Apereo (formerly jasig) scheduling assistant project. It was developed at the University of Wisconsin and provides a way for students to book time with faculty.

See the bw-sometime documentation at …​

6. Deployers

6.1. Install bedework as a wildfly feature

Version 4 only.

Note
This page is still being updated. The Fast-track demo version steps are correct (probably) but the detailed steps beyond still require work.

6.1.1. Note: Security

When installing bedework on wildfly steps must be taken to ensure that services that should be hidden are not exposed on the network.

Do NOT expose the OpenSearch and database ports by - for example - opening up all ports to the network.

The best approach is to disable all ports and only open up those necessary. Bedework only uses the standard web ports. Additionally, even for the web ports (80, 8080 etc) it is best to pass traffic through haproxy or apache web server and only allow known contexts (/cal, /caladmin etc).

6.1.2. Galleon, wildfly and bedework

Wildfly can be installed by using the galleon installer, which allows the installation of tailored instances of wildfly.

Additionally extra features can be installed using the same approach. Bedework now can be installed in this way with a choice of features.

6.1.3. Fast-track demo version

This will install a version of wildfly with bedework with a full set of demonstration clients. A demo version of opensearch and apacheds will also be included.

These steps have been tested on a mac and on the latest LTS linux.

Note
If this fails and you are going to retry I suggest first remove the wildfly directory created when running galleon before retrying.
./galleon-4.2.8.Final/bin/galleon.sh \
   install org.bedework.deploy:bw-wf-feature-pack:4.0.3 \
   --dir=wildfly --verbose --layers=bw-demoall-h2
  • In window 2 cd into the same directory and start the required services:

    export JAVA_HOME=<where the jdk is installed>
    ./wildfly/bin/bwstarth2.sh
    ./wildfly/bin/bwdirstart.sh
    ./wildfly/bin/bwstartoschqs.sh
    tail -f ./wildfly/opensearch/logs/bedework-2019.log
wait till it says something about cluster state going yellow
  • Back in the first window

./wildfly/bin/bwstartwildfly.sh
and wait for the messages listing the indexes
bwcategory20220114t153229<----bwcategory
bwcollection20220114t153228<----bwcollection
bwcontact20220114t153232<----bwcontact
bwevent20220114t153234<----bwevent
bwfilter20220114t153233<----bwfilter
bwlocation20220114t153230<----bwlocation
bwpreferences20220114t153226<----bwpreferences
bwprincipal20220114t153223<----bwprincipal
bwresource20220114t153235<----bwresource

6.1.4. More detailed steps:

These are the steps to install a version closer to a production ready bedework. As noted earlier you should run bedework behind a front-end service such as haproxy or apache. Wildfly can deliver any associated static content so there is no need to move that content elsewhere.

./galleon-4.2.8.Final/bin/galleon.sh \
   install org.bedework.deploy:bw-wf-feature-pack:4.0.3 \
   --dir=wildfly-26.0.1.Final --verbose \
   --layers=bw-cal-eventsubmit-pg,web-console

ln -s wildfly-26.0.1.Final wildfly
  • Install and configure OpenSearch

  • Start wildfly

  • Set a management id/password for the cli

  • restart wildfly (so the new id/pw is used)

  • For postgres initialise the database

  • reindex.

6.1.5. Wildfly and bedework layers

Bedework is installed as a feature pack which has a dependency on wildfly. The galleon tool supports layers which effectively define a particular flavor of the feature.

Many bedework layers have multiple versions for supported databases. For each the appropriate driver will be installed and datasources will be configured. The database is indicated by the suffix "-xx" where xx is:

Table 1. Bedework supported database configurations
Suffix Database Notes

h2

h2

Used for quickstart and demo purposes only

pg

postgresql

Postgresql 9 drivers are installed and datasources are configured.

Following are tables showing all defined layers. Many are flagged as dependencies of others so, for example, bw-calendar-pg will include bw-calendar-ro. In general the only layers to use are the top level layers that specify a database.

The bw-prod* levels provide a more production ready version which will probably still require configuration changes but should be close.

The demo systems will include apacheds as an ldap server with a preconfigured set of accounts - all with the password "bedework". Also the minimal version of OpenSearch will be installed. Startup scripts will be included in the wildfly/bin directory to start and stop each of the services.

Table 2. Top level layers
Layer Function

bw-demoall-xx

Deploy all bedework components for the indicated database as a demo system.

bw-demopublic-xx

Deploy all bedework public events components for the indicated database as a demo system.

bw-democaluser-xx

Deploy only bedework personal and group calendaring components for the indicated database as a demo system.

Thd next set of layers are used to install specific bedework apps. These could be used to install a specific subset of applications.

Table 3. Specific app layers
Layer Function

bw-public-ro

Deploy the readonly public events system (web clients and service and feeder) along with the timezone service. No database drivers or datasources are configured.

bw-public-xx

Deploy the full public events system (readonly, admin and submission tools) for the indicated database along with the timezone service.

bw-caluser-xx

Deploy the personal and group calendaring system for the indicated database along with the timezone service.

bw-carddav-xx

Deploy the carddav gateway server for the indicated database along with the timezone service.

bw-cal-eventsubmit-xx

Deploy the bw-public-xx layer and the event submission client for the indicated database

bw-eventreg-xx

Deploy the event registration service for the indicated database

bw-notify-h2

Deploy the notification service for the indicated database

bw-selfreg-h2

Deploy the self registration service for the indicated database

bw-synch-xx

Deploy the full synch engine for the indicated database along with the timezone service.

bw-tzserver

Deploy the timezone service

The next layers are used to install libraries used by servlet filters for CAS authentication or the keycloak servlet filter for saml V2 (shibboleth).

Table 4. Servlet filter layers
Layer Function

bw-keycloak-saml-filter

Adds the keycloak servlet filter libraries. Further configuration to the affected servlets will still be required.

bw-cas-filter

Adds the CAS filter libraries. Further configuration to the affected servlets will still be required.

The next layers are used to install certain functions and may be useful with some of the application layers.

Table 5. Subsidiary layers
Layer Function

bw-auth-apacheds

Configure wildfly to handle ldap authentication using a deployed apacheds ldap server. This is used for the demo system and testing.

bw-auth-ldap

Configure wildfly to handle ldap authentication. This is the same configuration used for apacheds. It WILL need editing to connect to other ldap servers but should provide a good starting point.

bw-auth-props

Authenticate using the wildfly property files. Not used much.

The remainder are dependencies of the other layers..

Table 6. Lower level layers
Layer Function

bw-calendar-rw

Configuration needed by all calendar app levels.

bw-common

Configuration needed by all levels.

bw-h2

Installs an h2 driver. Used by other layers that use h2 for jdbc.

bw-postgresql

Installs a postgresql driver. Used by other layers that use postgresql for jdbc.

Note that, while different database layers can be mixed, it’s not clear what will result from selecting the same application for different databases, e.g. bw-public-h2 AND bw-public-pg.

In addition to the bedework layers there are wildfly layers that might be useful.

Table 7. Wildfly layers
Layer Function

web-console

A console which gives access to the wildfly application server. See https://hal.github.io/documentation/manual/

6.1.6. Installing examples

These assume galleon has been installed and is runnable. For example it may be installed in the home directory and runnable as:

~/galleon-4.2.8.Final/bin/galleon.sh

In the following examples we will simply write galleon.sh

Example 1. Calendar server with console

galleon.sh install org.bedework:bw-wf-feature-pack:4.0.3 --dir=wildfly --verbose --layers=bw-public-pg,web-console

6.1.7. Installing snapshot version

This may not work as snapshots can be out-of-date or inconsistent but for reference…​

Download and unzip galleon then run the binary and enter teh commands as shown:

./galleon-4.2.8.Final/bin/galleon.sh
maven add-repository --name=ossrh-snapshots --url=https://oss.sonatype.org/content/repositories/snapshots/ --enable-snapshot=true
maven resolve-feature-pack org.bedework:bw-wf-feature-pack:4.0.4-SNAPSHOT
install org.bedework:bw-wf-feature-pack:4.0.4-SNAPSHOT --dir=wildfly-26.0.1.Final --verbose --layers=bw-demo-pg,web-console
exit

6.2. Installing bedework on Ubuntu

  1. Make sure that apt is up to date

    sudo apt-get update
  2. Ensure java 11 jdk installed

    sudo apt-get install openjdk-11-jdk-headless
    sudo update-alternatives --config java

    The last command should state there are no alternatives but if another has been installed pick the appropriate version.

  3. Install wildfly with bedework using galleon

    If you are not wanting to install the quickstart for builds and development then install a wildfly service by following the instructions at Install bedework as a wildfly feature. and disregard the remaining instructiosn below.

  4. Install maven if you want to do builds.

    sudo apt install maven
  5. Install git if you want to install the quickstart

    sudo apt install git
  6. Set up maven

    If you are doing builds of versions before 4.0.0 then set up the maven profile which must be named "bedework-3"

    mkdir ~/.m2
    emacs ~/.m2/settings.xml

    and paste in a modified form of the following (change the paths "/home/mike/bw-3.13.2/" to correspond to the directory you’re about to create)

    <settings xmlns="http://maven.apache.org/SETTINGS/1.0.0"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.0.0
                             http://maven.apache.org/xsd/settings-1.0.0.xsd">
    <pluginGroups>
    <pluginGroup>org.bedework</pluginGroup>
    </pluginGroups>
    
      <profiles>
        <profile>
          <id>bedework-3</id>
          <activation>
            <activeByDefault>true</activeByDefault>
          </activation>
          <properties>
            <maven.compiler.source>11</maven.compiler.source>
            <maven.compiler.target>11</maven.compiler.target>
            <org.bedework.deployment.basedir>/home/mike/bw-3.13.2/quickstart-dev</org.bedework.deployment.basedir>
            <org.bedework.deployment.properties>/home/mike/bw-3.13.2/quickstart-dev/bedework/config/wildfly.deploy.properties</org.bedework.deployment.properties>
          </properties>
        </profile>
      </profiles>
    </settings>
  7. Provide a bw settings file.

    If necessary create a .bw file in your home directory and add properties to it. Currently the only proeprties allow setting the profile for the .bw script used to build and the location of the deployment proeprties used to configure your deployed modules.

    An example file is

    #
    # Defaults for bw script
    #
    echo "Setting defaults from .bw"
    bw_mvnProfile="-P bedework_dev"
    
    bw_deployProps="/home/myhome/bwstuff/myprops.properties"
  8. Install the quickstart

    We create a directory in which to install bedework, download an install script and execute the script.

    mkdir bw-4.0.0
    wget https://github.com/Bedework/bedework/raw/bedework-4.0.0/build/quickstart/linux/installQS.sh
    chmod +x installQS.sh
    export JAVA_HOME=/usr/lib/jvm/java-11-openjdk-amd64
    ./installQS.sh

    and answer the questions presented to you. If you need to restart - probably easier to delete and recreate the directory.

    Some time will pass. You’ll be asked to provide an id and password for wildfly admin. I suggest hawtadmin and a password of your own choosing. More time will pass…​

  9. Install OpenSearch 1.2.3

    The eployed wildfly has scripts to install a docker image and start and stop it. See [opensearch] for instructions on installing and (re)indexing. You’ll need to start the system before reindexing.

  10. Start the system

    cd <into quickstart>
    ./wildfly/bin/bwstarth2.sh
    ./wildfly/bin/bwdirstart.sh
    ./wildfly/bin/bwstartwildfly.sh

    If you need to reindex there will certainly be some errors at startup.

  11. Reindex

    Use the installed script to run the reindexer

cd <into quickstart>
./wildfly/bin/bw-cli.sh -id hawtadmin -pw mypw
help
listidx
rebuildidx
listidx
makeidxprod all
q

For versions 4.0.0 onwards it is not necessary to install a quickstart. You can install a qyuickstart if you wish to do development or have convenient access to the sources.

If you are only interested in having a runnable version for demonstration or production then see the instructions at Install bedework as a wildfly feature

6.3. Installing the quickstart (3.12.x onwards)

The current quickstart is installed by executing a script which will create the quickstart directory and then download, clone and build. In the event of a failure part way through, the script may be restarted with the restart parameter.

Note
This requires git 2.x or greater. The clones use a tag rather than branch and this does not work for older gits.

The latest script may be downloaded from github. Alternatively clone the bedework repo and use the script at bedework/build/quickstart/linux/installQS.sh

Specific versions of the script are at:

6.3.1. Which version

The script will ask you if you want the latest, developmentor pre-release version.

You probably want latest. The dev version is obviously for development adn can be very unstable. The pre-release version is ther only for testing of the install script.

6.3.2. Apacheds

The script will install apacheds. This will install a directory with some initial ids in place for testing. Ultimately you’ll want to use your own directory service.

6.3.3. OpenSearch

You need a running OpenSearch - currently release 1.2.3 The glleon install will provide a script to install a docker image and customize it for the quickstart.

This requires you to have docker and docker-compose installed first.

The instructions related to installing a docker image can be found at https://opensearch.org/docs

After the install is complete use the 2 scripts bwstartosch.sh and bwstoposch.sh to start and stop the docker image.

If you want to install your own copy and configure it the configuration directory is wildfly/standalone/configuration/bedework/opensearch/config. Use this to replace the config in the downloaded OpenSearch.

6.3.4. Maven

Maven is required if you want to install the sources. If not you can skip this section.

This is a maven project and as usual you need to set up your maven profile in ~/.m2/settings.xml. The script will display a possible settings.xml file with the paths filled in.

If you want to merge in the profile to an existing settings.xml ensure you also merge in the pluginGroups section.

The profile does not need to be active by default if you have other profiles. The build process will specify the bedework-qs profile.

Below is the contents of an example settings.xml file. This file must be set up before allowing the script to continue on to the builds otherwise they will fail during deployment.

<settings xmlns="http://maven.apache.org/SETTINGS/1.0.0"
          xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
          xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.0.0
                         http://maven.apache.org/xsd/settings-1.0.0.xsd">
  <pluginGroups>
    <pluginGroup>org.bedework</pluginGroup>
  </pluginGroups>

  <profiles>
    <profile>
      <id>bedework-3</id>
      <activation>
        <activeByDefault>false</activeByDefault>
      </activation>
      <properties>
        <org.bedework.deployment.basedir>$qs</org.bedework.deployment.basedir>
        <org.bedework.deployment.properties>$qs/bedework/config/wildfly.deploy.properties</org.bedework.deployment.properties>
      </properties>
    </profile>
  </profiles>
</settings>

6.3.5. Default Maven Profiles

Note there appears to be a bug in the use of activeByDefault. Explicitly specifying a profile with the maven -P parameter is supposed to deactivate a profile marked as activeByDefault. This is not (always?) happening.

If you only have profile marking by default should be fine. To ensure only one profile (or the expected profiles) are active cd into the project and try

mvn -P bedework-3 help:active-profiles

6.3.6. Settings for build and install

The bw and install scripts now source a .bw file in the home directory which can set some defaults.

To alleviate some of the issues of having multiple maven profiles for bedework builds set the profile here with the bw-mvnProfile property. Note the setting needs to include the -P.

When building with maven the profile provides the location of the deployment properties. For the quickstart this is located in the bedework module at bedework/config/wildfly.deploy.properties.

When using the nobuild option you need to take a copy of that file and modify it according to the needs of your installation. Update the .bw file property bw_deployProps to define the location of the modified property file.

An example .bw file is:

#
# Defaults for bw script
#
echo "Setting defaults from .bw"
bw_mvnProfile="-P bedework_dev"

bw_deployProps="/home/myhome/bwstuff/myprops.properties"

6.3.7. Security

During the install an id and password will be set up so that the install process can reindex the data. These will allow use of the cli and the hawtio console.

6.3.8. Building

In many cases it is possible to simply cd in to the appropriate directory and do a mvn install with the bedework-3 profile. However there is a bw script which - while taking longer - does build all projects a module depends upon. This avoids the need to work out the dependency orderings of the independent projects. Thus

./bw bwcal

will build a lot of projects eventually building the client project which deploys an ear file.

6.3.9. Deploying

As part of the bedework project there is a maven plugin which uses a properties file to handle post-build deployment issues. Essentially the ear or war as built acts as a template for the deployer.

The deployment process may involve inserting filters for CAS, adding property values to web.xml files, cloning entire wars for calendar suites etc.

The file bedework/config/wildfly.deploy.properties is the quickstart version of that file.

When developing your own service the first thing to do is create a repository with your files and copy the above file into that repository. Then set the org.bedework.deployment.properties value in your maven settings.xml to point to that file.

DO NOT change the org.bedework.deployment.basedir property - unless you move the quickstart. This property is used to locate the wildfly instance.

6.4. Deploying servers

It is possible to deploy bedework in a number of configurations. There are the following major components to consider:

  • Database

  • OpenSearch

  • Calendar servers

  • Sync Engine

  • Event registration

  • Load balancer

The load balancer is only required if more than one calendar server is running. If there is only one instance then it is optional.

It might still be helpful to run the server behind a load balancer such as haproxy to restrict traffic to known services and allow filtering of bad ip addresses. Apache http server is an alternative.

All of the rest can be run on a single server but it will need a generous amount of memory and multiple processors for any significant load.

Running the database and OpenSearch on a separate instance helps spread the load. For greater reliability both could be run as clustered services and your organization may already provide database support you can use.

It should be possible to use hosted solutions for both database and OpenSearch.

Multiple calendar servers can be run to provide greater reliability and to handle higher loads. Bedework is not currently configured to handle session migration etc. If you do run multiple servers you MUST have a single database and OpenSearch service they both use.

It’s probably easiest to deploy the calendar server by simply installing the quickstart then removing any deployed modules not required. A later release will allow more choices over what gets deployed.

The sync engine and the event registration modules currently need to run on a single server. This is mostly because there is no mechanism to allocate outstanding sync requests between multiple servers.

Note that the calendar server does not use the database for the public events web presence or feeds. This woudl allow some different configurations - for example a single smaller server for public events administration and a number of front end servers for web presence and feeds.

6.5. Configuring wildfly

6.5.1. Adding an admin account

cd <quickstart-dir>
./wildfly/bin/add-user.sh  -a -dc wildfly/standalone/configuration

   What type of user do you wish to add?
    a) Management User (mgmt-users.properties)
    b) Application User (application-users.properties)
   (a): b

   Enter the details of the new user to add.
   Using realm 'ApplicationRealm' as discovered from the existing property files.
   Username : hawtio-admin
   Password recommendations are listed below. To modify these restrictions edit the add-user.properties configuration file.
    - The password should not be one of the following restricted values {root, admin, administrator}
    - The password should contain at least 8 characters, 1 alphabetic character(s), 1 digit(s), 1 non-alphanumeric symbol(s)
    - The password should be different from the username
   Password :
   Re-enter Password :
   What groups do you want this user to belong to? (Please enter a comma separated list, or leave blank for none)[  ]: admin
   About to add user 'hawtio-admin' for realm 'ApplicationRealm'
   Is this correct yes/no? yes
   Added user 'hawtio-admin' to file '/Users/xxx/dev/eap/wildfly-8.1.0.Final/standalone/configuration/application-users.properties'
   Added user 'hawtio-admin' to file '/Users/xxx/dev/eap/wildfly-8.1.0.Final/domain/configuration/application-users.properties'
   Added user 'hawtio-admin' with groups admin to file '/Users/xxx/dev/eap/wildfly-8.1.0.Final/standalone/configuration/application-roles.properties'
   Added user 'hawtio-admin' with groups admin to file '/Users/xxx/dev/eap/wildfly-8.1.0.Final/domain/configuration/application-roles.properties'
   Is this new user going to be used for one AS process to connect to another AS process?
   e.g. for a slave host controller connecting to the master or for a Remoting connection for server to server EJB calls.
   yes/no? no

Setting up databases

The quickstart is configured to use h2 for demonstration only. For production you will need to switch to something like postgresql or mysql.

postgresql

We’ll describe the process for the main calendar engine. The others are very similar.

All bedework and wildfly configuration files are in standalone/configuration/ - we’ll just refer to the relative path.

  • Name: caldb (you can change that if you wish)

  • Configuration: bedework/bwcore/dbconfig.xml

  • datasource: CalendarDS

Configure postgresql

You’ll probably need to configure postgres to allow your chosen role access to the server and databases.

Depending on how you’re running the system you may need to modify postgresql.conf and pg_haba.conf - both located by default in the daat directory.

If you’re running postgres on a separate system you may need to change the listen_addresses value and port:

listen_addresses = 'localhost'     # what IP address(es) to listen on;
                                   # comma-separated list of addresses;
                                   # defaults to 'localhost'; use '*' for all
                                   # (change requires restart)
port = 5432                        # (change requires restart)

Change "localhost" to "*" or a list of addresses.

5432 is the default port.

To allow your account access you will need to add a line or lines to pg_hba.conf near the end something like:

host    all             bedework        10.0.0.1/32        md5

This can be made more restrictive by naming the db.

Create a database with a known id/password. Ensure it is accessible from the host you are running bedework on.

The psql commands are something like:

create role bedework with login password 'xxxxxxxxx';
create database caldb owner bedework;

Set the hibernate dialect in the config file:

    <hibernateProperty>hibernate.dialect=org.hibernate.dialect.PostgreSQLDialect</hibernateProperty>

In standalone.xml replace the datasource definition with something like:

    <datasource jta="true" jndi-name="java:/CalendarDS" pool-name="CalendarDS" enabled="true" use-ccm="false">
       <connection-url>jdbc:postgresql://localhost:5432/caldb</connection-url>
       <driver>postgresql</driver>
       <pool>
         <min-pool-size>1</min-pool-size>
         <max-pool-size>50</max-pool-size>
         <prefill>true</prefill>
      </pool>
       <security>
         <user-name>xxxxx</user-name>
         <password>xxxxx</password>
       </security>
       <timeout>
        <idle-timeout-minutes>15</idle-timeout-minutes>
       </timeout>
        <validation>
          <validate-on-match>true</validate-on-match>
          <valid-connection-checker class-name="org.jboss.jca.adapters.jdbc.extensions.postgres.PostgreSQLValidConnectionChecker"></valid-connection-checker>

          <exception-sorter class-name="org.jboss.jca.adapters.jdbc.extensions.postgres.PostgreSQLExceptionSorter"></exception-sorter>
       </validation>
       <statement>
         <share-prepared-statements>false</share-prepared-statements>
       </statement>
     </datasource>

Ensure you also have the driver loaded - for example the drivers section which usually follows the datasources shoudllook somethign like:

You need to install the postgres driver - you’re either missing the driver declaration - something like:

<drivers>
    ...
    <driver name="postgresql" module="org.postgresql"/>
    ...
</drivers>

Ensure the jdbc driver is installed in the modules directory - something like:

wildfly/modules/org/postgresql/main/module.xml
wildfly/modules/org/postgresql/main/postgresql-8.4-701.jdbc4.jar

Module.xml for this example contains

<?xml version='1.0' encoding='UTF-8'?>

<module xmlns="urn:jboss:module:1.1" name="org.postgresql">
    <resources>
        <resource-root path="postgresql-8.4-701.jdbc4.jar"/>
    </resources>

    <dependencies>
        <module name="javax.api"/>
        <module name="javax.transaction.api"/>
    </dependencies>
</module>

Modify it appropriately for different versions.

Start wildfly and allow it to fully deploy. There will be many errors relating to the calendar database.

Delete the file wildfly/standalone/data/bedework/dumprestore/schema.sql

Use the cli to install the schema.

To run the cli you need the id and password you created when configuring wildfly. This id and password can also be used to access the hawtio console.

cd bw-cli/target/client/bin
./client -id admin-id -pw admin-pw
calschema export

This should install the schema. It will also create a file which can be manually installed if need be - use the psql client application

psql caldb < wildfly/standalone/data/bedework/dumprestore/schema.sql

Next you need to add some basic data. For this you need the full path to the initial data in wildfly/standalone/data/bedework/dumprestore/initbedework.xml

In the cli enter the command

restoreCal "/full/path/to/initbedework.xml"

The quotes are required. Some activity should ensue.

Reindex the data - again use the cli

rebuildidx

wait for it to terminate - then enter

listidx

The alias bwuser should be pointing at the index before the last one just created.

In the cli

makeidxprod index-name

replacing index-name with that last name - no quotes.

MySQL

TBD

Set the hibernate dialect in the config file:

    <hibernateProperty>hibernate.dialect=org.hibernate.dialect.MySQL5InnoDBDialect</hibernateProperty>

Unresolved directive in index.adoc - include::deployers/elasticsearch.adoc[]

6.6. Enabling and Disabling Public Events Registration

6.6.1. Enabling Public Events Registration

The public events registration system is enabled by default if you have installed Bedework with the data available in the quickstart.

6.6.2. Disabling Public Events Registration

If you wish to disable the public events registration system remove the "Eventreg admin token" from the System Preferences in the jmx console:

  • Log into the Bedework jmx console, e.g. http://localhost:8080/hawtio

    • Click "org.bedework.bwengine" in the left menu, then "service=System" in the right menu

    • Remove the value in the EventRegAdminToken field

    • Click "Apply Changes"

    • Invoke "saveConfig" to save your changes.

6.6.3. Enabling Public Events Registration (possibly after upgrade from a previous release)

First ensure that the CalWs interface is available. This is the SOAP service that the event registration service uses o communicate with bedework. In system.xml make sure the <calSoapWsURI> element is present and has the same value as the soap:address element in pubcalws-soap/wssvc.wsdl (this requires better explanation - it’s a file deployed in the bw-xml ear)

If you have upgraded from an older release, you may not have the data required for event registration in your system yet. Follow these steps to turn on the event registration system: (Please note: the process outlined below is only set up for the default quickstart and postgresql configs at the moment.)

  • Start up jboss and apacheDS

  • Log into the JMX-Console shipped with Bedework’s JBoss, e.g. http://localhost:8080/hawtio

    • Click "org.bedework.eventreg" in the left menu, then "service=Eventreg" in the right menu

      • If no database exists:

      • Set “create” and “export” attributes to "True"

      • Click "Apply changes" button (at the bottom of the list of attribute values)

      • Find the "schema" operation in the lower list, and click the “Invoke” button to export schema and create database

      • You should see a successful result; click "Back to MBean" button to return to "service=Eventreg"

      • Point at needed systems:

      • Set the WsdlUri attribute value to: http://localhost:8080/wsdls/pubcalws-soap/wssvc.wsdl

      • Set the TzsUri attribute value to: http://localhost:8080/tzsvr

      • Click "Apply changes" button

      • Admin token:

      • If no admin token exists, click “generateAdminToken”

      • You should see a successful result; click "Back to MBean" button to return to "service=Eventreg"

      • You should see a string such as "c0e93685-93cd-4bee-bed2-9996b89be28c" in the EventRegAdminToken attribute value.

      • Copy the EventRegAdminToken value (for use in the next step)

    • Click "org.bedework.bwengine" in the left menu, then "service=System" in the right menu

      • Paste the value into the EventRegAdminToken field

      • Click "Apply Changes" button

      • Invoke the "saveConfig" operation to save your changes.

  • Test:

    • Add a new public event. You should be able to select the checkbox "Users may register for this event", and make the event registerable.

    • Visit the event in public client — you should be able to register for it.

6.6.4. Notifications from EventReg

The event registration service will send notifications to the calendar engine when changes take place that might require notifying users. The event registration service calls the calendar engine notification web service (not to be confused with the notification engine). This web service allows the core engine to add notifications to the accounts of subscribed users. It is the job of the notification engine to forward those via email or some other service.

To configure notifications from eventreg you need to set the BwId, BwToken and BwUri properties in "service=Eventreg"and the NotifierId and NotifierToken values in "org.bedework.bwengine" → "notifications"

[[deploying-sync-engine] === Deploying the sync engine The synch engine handles the synchronization of external subscriptions with a bedework calendar - for example a Google web calendar or an ical feed from a department.

Currently such a synchronization must be carried out to a single calendar collection which only contains data from the external resource. Also only one way synchronization is supported - inbound to bedework.

These subscriptions are available to personal calendar users and to public events administrators. For personal calendar users the options are limited as it is intended only to mirror the external resource.

6.7. Initializing the database

If running with mysql the built in hibernate schema export doesn’t work - mysql jdbc does not support it.

The schema is simple however - it can be generated via the JMX mbeans or use the examples below - to install it manually, create a database - ensure UTF-8 is enabled

CREATE DATABASE `synchdb` DEFAULT CHARACTER SET utf8;
grant all on synchDb.* to '<id>'@'%' identified by '<pw>';

and then create the single table:

CREATE TABLE `bwsynch_subs` (
  `bwsyn_id` bigint(20) NOT NULL AUTO_INCREMENT,
  `bwsyn_seq` int(11) NOT NULL,
  `bwsyn_subid` varchar(250) NOT NULL,
  `bwsyn_owner` varchar(500) NOT NULL,
  `bwsyn_lrefresh` varchar(20) DEFAULT NULL,
  `bwsyn_errorct` int(11) DEFAULT NULL,
  `bwsyn_missing` char(1) NOT NULL,
  `bwsyn_connectorid_a` varchar(100) DEFAULT NULL,
  `bwsyn_conn_props_a` varchar(3000) DEFAULT NULL,
  `bwsyn_connectorid_b` varchar(100) DEFAULT NULL,
  `bwsyn_conn_props_b` varchar(3000) DEFAULT NULL,
  `bwsyn_props` varchar(3000) DEFAULT NULL,
  `bwsyn_dir` varchar(25) NOT NULL,
  `bwsyn_mstr` varchar(25) NOT NULL,
  PRIMARY KEY (`bwsyn_id`),
  UNIQUE KEY `UK_qptomm2syatpqumsl1udwk7be` (`bwsyn_subid`),
  KEY `bwsynidx_subid` (`bwsyn_subid`),
  KEY `bwsynidx_subowner` (`bwsyn_owner`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;

6.7.1. (Re)build bw-xml

The synch engine uses an extension of CalWS to communicate with bedework. It requires that the wsdl file contain the location of bedework. This is configured into the deploy.properties file - only one change for the synch engine should be necessary. Set the location of (one of) your application servers in the following.

# ------------------------------------------------------------------------
#       wsdls; ear bw-xml
# These go together - first name the wsdl directories and files...
org.bedework.app.bw-xml.bwwsdls.wsdl.synch=wssvc.wsdl

# then provide the global properties
org.bedework.global.synch.service.location=http://localhost:8080/synchws/

If you are running everything on one server then the quickstart setting above will do. Note that at the moment the synch engine can only work against a single bedework server. It can accept requests from any member of the cluster however. Keys

Generate a set of keys using the cli.

cd bwcli/dist/temp/shellscr/bwcli/ (or wherever your binary is)
./bwcli.sh
/usr/lib/jvm/java-8-oracle/bin/java -cp .:./classes:./resources:lib/bw-access-3.11.0.jar:lib/bw-annotations-3.11.0.jar:lib/bw-calfacade-3.11.0.jar:lib/bwcli-3.11.0.jar:lib/bw-ical4j-vcard-1.0.5.jar:lib/commons-collections4-4.0.jar:lib/commons-lang-2.3.jar:lib/commons-lang3-3.3.2.jar:lib/commons-logging.jar:lib/httpclient-osgi-4.3.3.jar:lib/ical4j-2.0.6.jar:lib/jackson-annotations-2.1.1.jar:lib/jackson-core-2.1.1.jar:lib/jackson-databind-2.1.1.jar:lib/jolokia-client-java-1.3.1.jar:lib/json-simple-1.1.1.jar:lib/log4j-1.2.8.jar:lib/rpiutil-3.11.0.jar org.bedework.bwcli.BwCli
JMX id:<your id>
Password:<your password>
cmd:genkeys gen
test with---->A variable of array type holds a reference to an object.
encrypts to-->BaUPfgTjzZxxbbW+lJACxmdo56tldkgfnr7LERkRTVyLQJh0kVt+GJZgJA1k9Wm+ojvpJCYFl34ybTy0vX2PM8Tu0+UsMKeV3HDi24NW6cH+C+QQ6XATLtskiBPhUQufpHBIKCke08PNh24xCoIk9+hllLgQQNCgVB1JQnQA0ak=
decrypts to-->A variable of array type holds a reference to an object.

Validity check succeeded

If you are using multiple servers copy the resulting key file from <quickstart>/wildfly/standalone/data/bedework/ on to each server. Ensure calendar server(s) can locate the synch engine.

The bwengine/synch settings are configured to use a jvm system property to locate the synch engine. In file <quickstart>/wildfly/standalone/configuration/bedework/bwengine/synch.xml you should see:

<?xml version="1.0" encoding="UTF-8" ?>

<synch xmlns="http://bedework.org/ns/" type="org.bedework.common.jmx.SynchConfigImpl">
  <connectorId>localBedework</connectorId>
  <managerUri>http://localhost:8080/synch/manager</managerUri>
  <wsdlUri>http://localhost:8080/wsdls/synch/wssvc.wsdl</wsdlUri>
</synch>

If you are using multiple servers change the host in <managerUri> to refer to your sync server.

6.7.2. Validating locations.

When an event arrives at the receiving end with an "X-BEDEWORK-LOCATION" property if the String value of the x-property is …​ === Locations in bedework

These are stored internally as location entities. They are created withthe admin client for public events or as a result of specifying or receiving locations for user clients.

6.7.3. Searching for public locations

There is a web service endpoint which can be used to search for public locations. It takes a filter expression as a parameter and will return <a limited?> number

A search takes the form:

<scheme-host-port>/locations/all?[params]

The params are * fexpr=expression * ?

The expression is a valid filter expression. Of particular interest are the following

  • loc_all=a-string

  • geo?

  • ?

For example

http://localhost:8080/cal/location/all.gdo?fexpr=loc_all=%27some%27

7. Development

7.1. Todo list

This is a (moderately) sorted list of features/changes etc for bedework. Could be a set of tickets but this is easier to read.

7.1.1. CoreEvents exceptions

Don’t do this:

    if ((val.getEntityType() != IcalDefs.entityTypeAvailable) &&
        (calendarNameExists(val, false, true) ||
          calendarNameExists(val, true, true))) {
      throwException(CalFacadeException.duplicateName, val.getName());
    }

Rollback and return an error indication. Should be a bad request - not a 500.

7.1.2. Index/updates after touch collection

The lastmod db update is not versioned as we get a lot and it causes frequent stale state exceptions.

The update should only take place if the new valueis after the old value.

Also when indexing - if we’re indexing because of a touch - we should ignore version conflict exceptions.

7.1.3. Filter x-properties

Filter x-properties in public clients/feeds - list of x-props to retain.

7.1.4. Api changes

Change the api to use response objects throughout. No exceptions. Allows for a better networked api.

7.1.5. Notifications

Need to be indexed in ES so that finding a notification for an entity is efficient (need to merge multiple notifications for same entity)

Change notifications for public events is probably not working correctly. We should be using the creator - or the owner of the alias - all public events have the same owner (public-user) and change notifications seem to be ending up in that bucket.

7.1.6. Limit interactions with db for updates.

For this we would do all interactions with ES and connect to and update db only as needed. Use sequence numbers to ensure db and index correctness. Benefits are shorter db interactions - only at point we update. Less complexity in web clients - no need to have conversations stretching across multiple requests. This can build on the work of the previous item. The web client code is already structured ro assume that it will do an explicit update of entities which should facilitate the change.

7.1.7. Move business logic out of webapps into core

Move as much as possible out of the current webapps module into the core APi implementation - this potentially allows a more RESTful style of client - possibly using the new jmap style interface being developed.

7.1.8. Categories

Use a larger category scheme as the basis for categories. Use SKOS based representation so that they are RDF friendly. Will allow for sub-categorization by event submitters.

7.1.9. Locations

Allow for use of external location sources such as geonames. Will allow good locations on external events.

7.1.10. Searching improvements

Search for events near a geo-location (requires locations to have geo)

Caching

Implement caching of feeder data as a built in feature of the quickstart.

7.1.11. Deployment of ears

Finish off the deployment process - it’s THAT close (is there an emoticon for 2 fingers very close together?) to allowing deployers to just replace the ears from prebuilt ears on the site. No builds required - server can detect an update is available.

7.1.12. Deployment of wildfly modules

If all - or many - required dependencies are deployed as wildfly modules it should reduce the size of the deployment and allow for even quicker startup.

7.1.13. Networked client api

Subset of svci but can be used for web client interactions.

7.1.14. Groups

Directory interface to directly interact with grouper for user and admin groups. Allow consumer only approach for external management of groups. Use extra ldap attributes to allow admin groups to be maintained in ldap.

7.1.15. Index logs in ES

Use kibana to get metrics etc

7.1.16. Timezones

Update UI to provide a search - possibly based on map. Use tzdist geo feature (being developed)

7.1.17. General work needed

  • Upgrade ES to latest - changes the query structure

  • Upgrade all libraries to latest

  • Preprocess the xsl to build the deployable language specific versions.

7.1.18. Recurring events

Currently I’m indexing all of the instances we allow - that can be many. The ideal would be not to index any instances but only the master and overrides. This has some problems.

Time range searches are easy. We have an indexed start and end and for the master we set it to be the entire (perhaps infinite) range so it always gets included in the result. For this case we could generate the instances based on the master.

For a query involving other conditions - e.g. categories the master may not appear.

Entity only

This is just the master

Overrides

This is master + overrides - again it may be filtered so we need to pull in the master if absent. This is in line with CalDAV (I think).

Expanded

This is a full expansion of all instances. If the master appears in the result we need to generate the appropriate instances for the selected time range.

If it doesn’t appear, but we have some members in the result - these are overrides and we need to carry out a secondary fetch of the master.

However the real problem here is paged queries. For the web clients (mostly) we allow a paged query of full expansions. The result is ordered by start date. This is easy to achieve when we index the instances as it’s just a time ranged query using ES paging.

If the instances aren’t indexed we need to retrieve all the events that match the query, at least produce the dtstarts and recurrence ids for every instance then deliver the matching instances in the batch.

7.2. Debugging the quickstart

Need to write a bunch of stuff here - I use intellij.

7.3. Tracing database interactions

Add the following to the wildfly logging configuration to see generated SQL and the values of the parameters:

  <logger category="org.hibernate.SQL">
      <level name="DEBUG"/>
  </logger>
  <logger category="org.hibernate.type.descriptor.sql">
      <level name="TRACE"/>
  </logger>

7.4. Running the caldav tester

The tester started as an Apple developed project to test CalDAV servers. It has since been taken over by CalConnect adn is in the process of being modified to make the tests more universally applicable.

7.4.1. Setup

To run some basic tests there is a script bw-caldavTest/src/main/resources/calconnect-tester/testbw/bw-QuickLook-CalDAV.sh

This script sets up the tester which needs to be cloned from the repository.

Running the tester requires that a number of users be provisioned in a particular state.

The quickstart data comes with users user01-user04. Each is setup with

  • cn: 01,test

  • objectclass: inetOrgPerson

  • objectclass: organizationalPerson

  • objectclass: person

  • objectclass: top

  • sn: user

  • givenname: 01

  • mail: user01@mysite.org

  • uid: user01

  • password: bedework

(Replace 01 with 02-04 for the rest)

8. Older Bedework Versions (3.10 and Prior)

3.10.4

December 14, 2016

  • Significant web client performance improvements

  • Fixes to sharing

  • Enhancements to the notification system

3.10.3

  • Core engine

    • Flag all entities as indexed

    • Fix hibernate query which pulled in too many objects

    • Fix bug in ES query which missed a number of recurrences

    • Fixes to synch report

    • Fixes to sharing

    • Work on notifications

    • Allow disable of ldap group check

    • Add code to stop autokill killing the indexer

    • Better cleanup of http connections

    • Fix up timestamps in ical

    • Dumprestore sets status of process

  • Caldav

    • Better handling of error conditions

    • Better error responses

    • Fixes to content type

    • Eventreg

    • Fixes for eventreg id

    • Use activemq to queue actions so they don’t get lost on restart

  • Webapps

    • Some fixes to the grid view

    • Remove a synchronized to reduce a bottleneck

    • Try refactor and recode calendar collection cloning for navigation

    • Fix copy of event

    • More cleanup of http connections

  • Util

    • Allow deployer to work with war files

    • Add jms classes extracted from sysevents module

    • Add jolokia module to allow a client to interact with jmx

  • Webdav

    • Minor propstat fix

    • Better error responses

Bedework 3.10.2

  • Performance Improvements

    • numerous updates to correct for bottlenecks and other performance issues allowing for significant load and large-scale deployments

  • External Subscriptions

    • significant improvements to external subscription handling

    • better handling of contacts and categories for public calendaring

  • Workflow (public calendaring)

    • new author and approver roles allow for moderated publishing of events by administrative groups

  • Cross-Suite Suggestions (public calendaring)

    • new suggestion mechanism allows different calendar suites to suggest and accept (or reject) events across teams

  • Event Registration (public calendaring)

    • significant improvements to event registration features

    • addition of custom fields to any registration form and a form builder for constructing and managing custom forms (extra text inputs, checkboxes, radio buttons, and select boxes)

    • email notifications (see Notifications)

  • Notifications (public calendaring)

    • In public calendaring, administrative users now receive notifications of actionable items (workflow approval requests, suggestions from other groups, and responses to each)

    • In event registration, the notification engine is used to communicate with registrants concerning registration, waitlisting, and event cancellation

  • Self Registration

    • Allows users to register with the calendaring system so that they may register for public events

  • Web Client Improvements

    • Improvements to the public administration client

      • Locations can be broken into locations and sub-locations (i.e. buildings and rooms)

      • Rooms can be added on-the-fly during event creation

      • Improved location and contact look-up during event creation

      • Numerous other stylistic and usability enhancements in the administrative client

    • Improvements to the public web client

      • improvements to accessibility compliance

  • Bug fixes

3.10

  • A largely-revamped public client, which features:

    • Responsive design - the public client will display reasonably on almost any screen size.

    • More powerful and flexible left-hand navigation - in just a few clicks, site visitors can ask for the "Arts events and Films taking place on West Campus"

    • Event filtering by string. – ex., “Arts events and Films on West Campus that include 'Sherlock'".

    • An endless stream of events - All events that match the criteria are presented, starting with today’s events (or any other date specified), and advancing into the future. An initial group of events are displayed on the page, and as the visitor scrolls towards the bottom of the page, the next group is presented.

    • Improved performance with fewer page reloads - most operations, such as adding or removing a filter, are done "in-page" (using Ajax calls).

  • Enhancements to indexing

    • Bedework 3.10 has a new search engine – ElasticSearch. In the Bedework context, ElasticSearch, provides better remote management of indexes, better scalability options, and much simpler configuration than Solr. ElasticSearch is used by, among others, Wikimedia, Foursquare Etsy, and GitHub.

    • Improved performance and scalability by directing most queries to an ElasticSearch index rather than directly to the database engine

  • More real-time site configuration

    • All configuration settings are now set through the JMX console, eliminating the need for rebuilds to reconfigure

  • An improved Quickstart

    • A smaller (~40%) Quickstart - no-longer-used code (such as webcache) and unused JBoss components have been removed

    • Better Quickstart documentation

3.8.0.13

April 17, 2012
* Administrative client: Support for image uploads and auto-generation of thumbnails during event creation and editing "Manage events" page now uses date range and date navigation to list events The event description field now tracks the number of characters used while the user types. Pending event topical areas are auto-selected (when possible) based on the selections made in the submissions client Pending event "preferred" and "all" listings are correctly selected when first editing a pending event "Preferred" and "all" listings now sorted Auto-focus first available field code updated to be more generally useful (based on work done by Eric Wittman Bug fix: pending event titles are correctly escaped on publish

  • Public events client:

    • Bug fix: correct ids on nodes of the calendar explorer menu (new feature in 3.8)

  • Submissions client:

    • Removed Confirmed/Tentative/Cancelled radio buttons

    • Correctly associate selected topical areas between submissions client and admin client (where possible)

  • Bedework servers / utils:

    • Configure Tomcat to fully support UTF-8 URLs

    • Bug fixes to dump/restore utilities

    • Bug fix: for long standing bug in date/time util.

    • Bug fix: missing tz and locale update

    • Bug fix: tombstone related bug in methods called by indexer

    • Bug fix: correct check for duplicate collection name

    • Bug fix: various DST, timezone fixes, support for UTC, etc.

    • Bug fix: blob caching fixes

3.8.0.11

February 27, 2012
* Fix issue preventing display of events on a DST boundary. * Fix iSchedule so it works again. * Fix change set so that added/deleted attendees get notified. * Storing of resources/attachments was broken. Required upgrade to hibernate 3.6 * Added some quota support because we can now store resources. * Added some missing datasource files. * Watch out for and fix bad PARTSTAT. * Don’t email freebusy requests. * SOAP namespace change.

3.8

January 30, 2012
* Synchronization Engine, providing more efficient synchronization of external data. read-only .ics subscriptions/feeds into Bedework are supported in this release.

  • CalWS-SOAP, a SOAP protocol for calendaring being developed by OASIS and CalConnect. Bedework 3.8 uses CalWS-SOAP for communication between the synchronization engine and Bedework.

  • WebDAV Synch, which is a draft RFC extension to WebDAV/CalDAV (https://tools.ietf.org/html/rfc6578), providing a more efficient method for client synchronization. WebDAV synch is currently supported by iCal - the Apple desktop calendaring client, and also by aCal - an Android CalDAV client (http://andrew.mcmillan.net.nz/projects/aCal).

  • New UI Feature - a theme setting that produces an explorer view of the calendars for navigation

  • Quickstart has a smaller footprint

  • New, simplified theme (using the new calendar explorer menu) providing easier integration with an organization’s web design.

  • Hypersonic (HSQL) is now packaged with the Quickstart instead of Derby. Derby never met our expectations that it could be deployed as the production DBMS for Bedework, and the advantages of HSQL, including Quickstart support for the Scheduling Assistant, are compelling.

  • Structural changes to projects.

  • All references to Bedework documentation are updated in the web clients to point to the Bedework website and latest documentation wiki. Previously, some references were to the older, deprecated bedework.org site, which created some confusion in previous releases.

  • Various bugfixes

3.7

March 10, 2011
* The Bedework personal client has been simplified, and presents new displays for FreeBusy, and for scheduling and managing meetings.

  • Bedework 3.7 uses CardDAV for managing contacts, and provides a new, standalone address book web client as well as a significantly improved and enhanced CardDAV V4 server.

  • Improved internationalization - the web clients are distributed with language strings in Spanish (all clients) and German (public and personal clients). The Spanish translations are the result of a collaboration between the Universidad Pública de Navarra and their Pamplona colleagues at Universidad de Navarra, and the German translation is the contribution of Werner Frerichs of the University of Kiel.

  • Personal calendaring client UI has been upgraded, with particular attention to Scheduling and Free/Busy

  • Improved environmentals - Reduced memory footprint in the quickstart; logging overhead has been decreased

  • Public and private calendar display names can now be changed, providing a means to safely modify the labeling of calendars over time as well as stronger internationalization.

  • Addressbook enhancements, including a CardDAV V4 server, support for groups, and a stand-alone address book web client appropriate for deployment within multiple applications

  • An initial version of the CalWS restful web service API is available in the system shipped with the quickstart. See: http://www.calconnect.org/pubdocs/CD1011%20CalWS-Rest%20Restful%20Web%20Services%20Protocol%20for%20Calendaring.pdf

  • Bug fixes to all servers and clients

3.6

February 3, 2010 * Core Bedework services packaged in JBoss * Spanish translation of public calendaring themes shipped with quickstart * Bug fixes and final enhancements * See Bedework 3.6 Milestone page for information about post-release bug fixes.

3.6 release candidate 1

January 9, 2010
* Public calendaring New default public theme based on Duke/Yale themes Feed URL and Widget Builder for generating rss, json, xml, and ical feeds as well as embeddable javascript widgets A "feeder" application that serves as a common source for public data feeds and widgets A web cache application for storing and serving the feeds and widgets Internationalized and modularized themes New mobile theme for iPhones and other smartphones

  • System notifications now built on JMS (ApacheMQ) allowing more modular design of the services

    • Indexing reworked as outboard process

    • Scheduling reworked as outboard process

    • Logging of system notifications

  • Performance improvements

    • in CalDAV

    • event retrieval (system wide)

  • Other Enhancements

    • improved ical subscriptions, user and public clients

    • UI support of deleting collections

    • improvements to scheduling in the user client

    • subscription coloring in the user client

    • general bugfixes

    • improvements to documentation

3.5

July 17, 2009
* Bug fixes and final enhancements * See href="http://www.bedework.org/trac/bedework/milestone/Bedework%203.5[Bedework 3.5 Milestone page] for information about post-release bug fixes.

3.5 release candidate 2

June 3, 2009
* Bug fixes and final enhancements * Further support for draft 0.7 of CalDAV scheduling * Improved personal client user interface

3.5 release candidate 1

May 19, 2009
* Performance improvements reduce JVM memory usage decrease database system load

  • Large-scale restructuring of data and administrative UI to map to new conceptual model of the single calendar pool

  • public events submissions client enhanced with added workflow in the admin client for pending events, including email notification when an event is published

  • admin client: cross-tagging of events by administrative groups (currently locate events by searching)

  • Support for draft 0.7 of CalDAV scheduling

  • Stronger support for xproperties

3.5 preview release

January 29, 2009
* Public events optimized for a single calendar pool model

  • simplifies public events calendaring

  • filtering can be applied to all collections allowing for fine-grained control over subscriptions from within the user interfaces; no longer necessary to filter in the xslt of public client

  • administrative users tag events by topical area based on subscriptions within each calendar suite. The system then assigns appropriate categories to events.

  • administrative users can tag events with as many topical areas as appropriate

  • categories are maintained by superusers

  • categories can be set on all collections

  • adding a user to a calendar suite group will allow the user to administer the calendar suite

    • Subscriptions greatly improved

  • all subscriptions are reimplimented as calendar aliases

  • subscriptions now appear in caldav clients such as Mozilla Lightning or Apple’s iCal

  • users can apply filters to aliases allowing for fine-grained control over subscriptions in the user client

  • subscriptions to external ical feeds available in public and personal clients

    • Apache DS ldap directory ships with quickstart

  • used for user accounts, authentication, and the new CardDAV server

  • quickstart more closely resembles a production system

    • CardDAV server first release

  • address book widget used in personal client queries attendees for meetings

    • Timezone server in use

  • provides standardized timezone service

    • CalDAV improvements

  • Support for draft 6 of CalDAV scheduling

  • Support for storing resources (e.g. files, attachments) within the folder hierarchy

3.4.1.1

June 3, 2008

  • Support for RFC-2445 x-properties

  • Inclusion of image URL for public events

  • Improvements to public event submission web client

  • Improvements to CalDAV and WebDAV

  • Improvements to dump/restore

  • Improvements to access control

  • Better support for driving public events client with categories

  • Fixes to scheduling - better support for COUNTER

  • Support deletion of non-empty calendars via CalDAV

  • Performance improvements to Lucene indexing

  • Bug fix for null parameters in x-props (thanks to Roberto Polli)

3.4.1

March 18, 2008

  • CalDAV: many improvements; greatly enhanced interoperability with Mozilla Lightning and Apple’s iCal; support for subscriptions to web calendars through the CalDAV server, allowing Bedework to expose user subscriptions to desktop clients; improved support for CalDAV filters

  • Addition of Public Events Submission web client (beta) which allows non-admin users to suggest public events. (configuration details)

  • Much better support for RSS and Javascript feeds including the addition of category filters and date ranges

  • Rudimentary interface for adding named CalDAV filters that can be used with the public web client providing powerful filtering features

  • Improvements to scheduling

  • Improvements to recurring event support

  • Improvements to freebusy

  • Improvemenst to locale support

  • Improvements to Lucene searching

  • Improvements to access control

  • Better handling of tasks

  • Better timezone handling

  • Numerous UI improvements including updated web template for mobile devices and an improved administrative interface

  • Bugfix to set all character encoding to utf-8 (thanks to Martin Blom)

3.4

September 14, 2007

  • Better standards support: A much more complete implementation of the calendaring standards RFC2445, RFC2446, RFC791 (CalDAV) and the CalDAV scheduling draft standard.

  • A reasonably complete implementation of iTIP scheduling, such as simple (non-recurring) iTIP scheduling, display of attendee FreeBusy information, sending invitations, update of attendee status, canceling meetings. There is some support of recurring meetings and modifications to instances partially works.

  • Portal support: A number of changes to make the user interface more portal friendly.

  • CalDAV: Changes/fixes have been implemented to improve interoperability with Apple’s iCal, and there is significant support now for CalDAV scheduling.

  • UI improvements to access control and recurring events.

  • Initial support for FREEBUSY URL and calendar subscription URL.

  • Apache Ant: Upgrade to 1.7 to make apt task available for further development.

3.3.1

April 25, 2007

  • Access fixes. Note this requires a change to the /public folder. Previously read + write-content was sufficient for administrative privileges. Now it must be read + write-content + bind (may also want unbind to allow deletions)

  • A number of bugfixes to CalDAV support.

  • Implemented some missing CalDAV features.

    • attachments now work,

    • copy/move/rename partially works.

    • Freebusy information can be stored

    • Tasks (todos)

    • Search filters

  • Bug fixes for recurring events

  • Timezones shared correctly

  • Oracle now builds and deploys without alterations to generated schema.

  • Fixed up restore so that it can handle UWcal 2.3.2 data

  • 12/24 mode works correctly

  • One-shot free and busy url works allow users to publicize their busy time.

  • Scheduling is now close to completion and largely usable. Some of the more esoteric features still require work, e.g. instances of recurring meetings, COUNTER is untested.

  • UI supports all access control features.

  • Import and export of calendars completed

3.3.1 preview release

February 23, 2007

  • CalDAV - Fixes to problems uncovered in the interoperability testing at CalConnect.

  • Recurring events - Fixes to some bugs, especially in the sharing of these events.

  • Timezones - Fixes to problems sharing events with 'private' timezones, such as those added via CalDAV or imported as ics.

  • Access control - Fixed some bugs in setting ACLs. The UI has been updated to enable all features of access control.

  • Scheduling - Implemented additional functionality. Scheduling is still incomplete but more features are exposed in 3.3.1 to allow further testing. We have successfully sent an invitation, posted a response, and observed the resulting event in the calendar. Scheduling support is "fragile" but progressing rapidly.

  • Bedework 3.3.1 is much more Oracle-friendly than previous versions. Based on work done by Julian Ball at Queens, and Chris Mann at Maryland, we have significantly overhauled the XML schema, with the intent of eliminating or at least drastically reducing the modifications Oracle users would have to make.

3.3

January 24, 2007

  • Java 1.5 and Tomcat 5.5 as implementation requirements

  • Lucene searching has been implemented for all clients

  • Categories have been reinstated

  • Many changes to CalDAV to better support various clients.

  • Significant progress supporting OpenConnector

  • Support for recipients and attendees

  • iCal import and export of multiple events / whole calendars

  • Improved timezone displays / support

  • Support for floating time values

  • Support for storing UTC time values

  • Recurring event support

  • Human readable text fields now stored with language information to facilitate internationalization

  • Scheduling (meeting requests, replies etc) are now mostly implemented

  • Significant improvements to the web user interfaces, XHTML user clients & dojo based widgets

  • Bug fixes (e.g. access control) and housekeeping (e.g. all tables given consistent names)

3.3 release candidate

December 15, 2006

  • A subset of the 3.3 release, above

3.3 preview release

November 22, 2006

  • Lucene searching added

  • Recurring and annotated event support rewritten

  • Added a String table to facilitate internationalization

  • Category support resuscitated, categories attached to calendars - implement changes to event categories

  • Java 5 language features to facilitate development

  • Fixes to access control and improved ui support

  • Added back-end support for todos

  • Support for scheduling operations

  • CalDAV improvements

3.2

August 15, 2006

  • SVN Restructuring complete

  • CalDAV improvements

3.1

August 1, 2006

  • Restructuring of system into multiple SVN projects

  • Working personal calendars released

  • Freebusy aggregator added

  • CalDAV server restructured to allow use as a proxy

3.1 release candidate 4

June 22, 2006

  • Restructuring of stylesheet directories to better support calendar suites

  • Initial release of jsr-168 compliant portlet for use within uPortal

3.1 release candidate 3

June 19, 2006

  • Further interface updates and bug fixes

  • Calendar Suites (departmental calendars) updated and highlighted

3.1 release candidate 2

June 12, 2006

  • Bug fixes

  • Upgrade to struts 1.2.9

  • Introduction of Calendar Suites:

    • the Bedework system can now be defined as a collection of "calendar suites" which allows for the implementation of departmental calendars.

3.1 release candidate 1

May 19, 2006

  • Dump/restore and schema:

    • Now zipped up with shell script for running stand-alone

  • Caldav fixes to bring up to draft 12

  • Prototype free/busy aggregator

  • Personal Client:

    • sharing of events (as well as calendars and free-busy)

    • bug fixes and interface updates

  • Admin and Public Clients:

    • brought up-to-date with all changes

3.1pre

May 5, 2006

  • Personal Client:

    • Preferences management in place

    • Can select destination calendar when importing an event or adding an event reference

    • Updates to access control

    • Schema changes

    • Graphical updates (also to public client)

3.1pre

April 21, 2006

  • Personal Client:

    • Full calendar management in place

    • Free/Busy display

    • Basic sharing of calendars and free/busy

3.1pre

April 14, 2006

  • Personal Client: Fixed up personal client so basic functionality is restored. Client has been given a graphical overhaul.

  • Free/Busy: Preliminary work on free/busy

  • Lucene full text searching: Added classes to support Lucene indexing :sectnums:

Previous versions of this document