cf/238

The cf-release v238 was released on June 27, 2016.

IMPORTANT - Known issue: The WebDAV blobstore and Cloud Controller API / Clock / Worker jobs are unable to start after a VM restart because creation of the run directory for each process, /var/vcap/data/sys/run/*, was moved to the bosh pre-start script. The jobs are unable to start because /var/vcap/data/sys/run is mounted on a temporary file-system and the bosh pre-start script is not executed on VM restart, only deployment. A fix is in the pipeline for CF-240. To workaround this issue, operators can do a bosh deploy, which will recognize the failing jobs and properly create the run directory. - v238 includes a fix for CVE-2016-4468, UAA SQL Injection. The mitigation is to upgrade to cf-release v238 - Cloud Controller and other components of capi-release now use bosh pre-start job-lifecycle scripts for many startup tasks including database migrations. This capability requires bosh-release v206+ (1.3072.0) and requires releases deployed with 3125+ stemcells.

Contents: - CC and Service Broker APIs - DEA-Warden-HM9000 Runtime - Buildpacks and Stacks - Identity - Routing - Loggregator - Internal Components - Job Spec Changes - Recommended BOSH Stemcell Versions - Recommended Versions of Additional Releases

CC and Service Broker APIs

CC API Version: 2.57.0

Service Broker API Version: 2.9

CAPI Release

• Add blobstore timeout configuration details
• Add configuration to run multiple blobstore nginx workers per core details
• Update nginx to 1.11.1 details
• Bridge components only support properties.capi details

Cloud Controller

• Make minimum candidate stagers configurable details
• Use hm9000 internal address when making requests details
• Add missing event types to API documentation details
• Enforce space quota on route creation details
• Client author should be able to follow CC API docs to configure total reserved route ports when creating a space quota details
• Retry blobstore requests before failing details
• hm9000 client handles socket error when internal hm9000 address does not exist details
• Emit error when consul is down details
• Add optional description to security group rule details
• Domain helper used in check_for_domain_overlap doesn’t work when a second domain appears in list in addition to the system domain details
• /v2/routes and /v2/apps/:guid/routes and /v3/apps/:guid/routes return a deprecated url format for domains. details
• Emit error when consul is down details
• Allow Shared Domains to be seeded through the manifest details
• Sequel sql_log_level is ‘debug’, not ‘debug2’ details
• Move database migrations and seeding into bosh pre-start. Move buildpack installation into bosh post-start. Run cloud controller scripts as vcap user. details
• Updating service broker with non-unique service plan name fails to provide offending service and plan info. details
• EXPERIMENTAL: When Cloud Controller starts an app on Diego and has a service binding containing volume_mounts, it should desire an LRP with volume mounts details
• V3 Experimental
  • As a space developer, I can map a route to a specific process type on a specific port details
  • As a space auditor, I should NOT be able to download packages or droplets details
  • v3 process examples in docs should show stats link details
  • Droplet memory_limit field should be staging_memory_in_mb details
  • Droplet disk_limit field should be staging_disk_in_mb details
  • As a SpaceAuditor, I expect to never see sensitive information details
  • As a SpaceManager, I expect to have oznly READ access for all V3 endpoints details
  • As a space developer, I can get the list of droplets associated with a package details
  • As an API consumer, I should be able to filter /v3/droplets and /v3/apps/:guid/droplets details
• Service Broker API
  • Add service_id and plan_id to last_operation calls to service brokers details
  • Support for broker operation identifier for provision details
  • Support for broker operation identifier for deprovision details
  • Support for broker operation identifier for update details
  • EXPERIMENTAL: Translate service broker volume mounts to diego volume mounts details

TPS

• Support ActualLRPCrashedEvent from BBS in TPS details

Pull Requests and Issues

• cloudfoundry/cloud_controller_ng#551: Missing service/plan id for async last_operation call details
• cloudfoundry/cloud_controller_ng#573: SpaceManager / SpaceAuditor could not see private domain details
• cloudfoundry/cloud_controller_ng#597: The “Updating an App” documentation makes it look like I can update the detected_start_command details

DEA-Warden-HM9000 Runtime

• Bumped to ruby 2.3.1
• Improved HM9000 performance

Known issues - Container metrics via CLI are 100x larger than reality.

Buildpacks and Stacks

Support for .profile pre-runtime hooks. Documentation can be found here

stacks

updated to 1.67.0 (from 1.56.0)

1.67.0

1.66.0

1.65.0

1.64.0

1.63.0

1.62.0

1.61.0

1.60.0

1.59.0

1.58.0

1.57.0

java-buildpack

updated to v3.7.1 (from v3.7)

v3.7.1

nodejs-buildpack

updated to v1.5.15 (from v1.5.14)

v1.5.15

php-buildpack

updated to v4.3.14 (from v4.3.12)

v4.3.14

v4.3.13

python-buildpack

updated to v1.5.6 (from v1.5.5)

v1.5.6

ruby-buildpack

updated to v1.6.19 (from v1.6.17)

v1.6.19

v1.6.18

staticfile-buildpack

updated to v1.3.9 (from v1.3.8)

v1.3.9

Identity

Updated to UAA Release 3.4.1 - UAA 3.4.1 Release Notes - UAA 3.4.0 Release Notes

Routing

• Operator can now use manifest property uaa.ca_cert to configure a custom CA used to sign the SSL cert hosted by UAA details
• Quota attribute total_reserved_route_ports is now unlimited for BOSH Lite manifest generated using ./scripts/generate-bosh-lite-dev-manifest details
• Manifest properties have been updated for gorouter; see below details, details
• Some processes on the router job no longer run as root details
• Gorouter now uses cgo netdns (instead of Go’s DNS resolver), which supports EDNS0 details

Loggregator

• Add loggregator etcd ssl properties to cf-release template details
• Add handshake timeout and set DisableKeepAlive in noaa consumer details
• Initial work on tagged metrics: changes to dropsonde. Not yet ready for use.
• Doppler supports Etcd TLS details
• Metron supports Etcd TLS details
• TC supports Etcd TLS details
• Syslog Drain Binder supports Etcd TLS details
• Include Forwarded Addresses in HttpStartStop details
• Loggregator components emit metrics regarding open file handles/connections details
• Pull AZ information from BOSH in trafficontroller details
• Pull Zone (when available) From Bosh 2.0 in Metron and Doppler details
• Remove preferred_protocol property from docs details
• Traffic Controller should timeout when failing to talk to the UAA or CC over SSL details
• cloudfoundry/noaa #15: Set consumer to open after connection goroutines are closed details
• cloudfoundry/sonde-go #2: Make top-level package go-gettable details

Internal Components

consul

• consul-release was bumped from v80 to v92. Significant changes:
  • Fixes a bug in confab, the wrapper program for orchestrating the start of the consul binary. The previous behaviour was that if confab is started a second time, it will kill the previously running consul process, leaving nothing running. The new correct behaviour is for confab to fail if consul is already running, but not kill consul. details
  • Bump consul binary from version 0.5.2 to 0.6.4, and consul-template binary from version 0.9.0 to 0.14.0. details
  • Long-running processes in the consul_agent job now run as vcap rather than root. details

etcd and etcd-metrics-server

• etcd-release was bumped from v48 to v57. Significant changes:
  • Long-running processes in the etcd job now run as vcap rather than root. details
  • Long-running processes in the etcd_metrics_server job now run as vcap rather than root. details
  • etcd_metrics_server job supports talking to etcd via mutual TLS. details
  • Add etcd-proxy job in service of zero-downtime upgrades from a non-TLS etcd cluster to a TLS etcd cluster. The zero-downtime upgrade work is currently a work in progress. details

postgres

No changes.

nats and nats_stream_forwarder

• nats-release was bumped from 01ee06a4cab572a87417a25a886ad933bfa183a0 to 219e93bdb8a8e9fc0734fb0640b8b8d6edc14c3f. Significant changes:
  • Long-running processes in the nats job now run as vcap rather than root. details
  • Long-running processes in the nats_stream_forwarder job now run as vcap rather than root. details

Job Spec Changes

• Add property cc.minimum_candidate_stagers, the number of candidate DEAs for staging, with default of 5.
• Add property hm9000.port for configuring how Cloud Controller communicates with DEAs, with no default. This is used to generate hm9000.internal_url in the Cloud Controller configuration. If not using DEAs / HM9000, this port is unused, but required.
• Add properties cc.resource_pool.webdav_config.blobstore_timeout, cc.packages.webdav_config.blobstore_timeout, cc.droplets.webdav_config.blobstore_timeout, cc.buildpacks.webdav_config.blobstore_timeout, each with defaults of 5 seconds.
• Add property blobstore.nginx_workers_per_core with default of 2. This is used when configuring blobstore instances serving WebDAV.
• Add fog_aws_storage_options to blobstore configuration properties. These each accept a hash, with the only valid key being encryption. Set fog_aws_storage_options to {'encryption' => 'AES256'} in order to encrypt files at rest in S3.
  • cc.resource_pool.fog_aws_storage_options
  • cc.packages.fog_aws_storage_options
  • cc.droplets.fog_aws_storage_options
  • cc.buildpacks.fog_aws_storage_options
• CC Bridge Properties moved from properties.diego to properties.capi:
  • diego.cc_uploader -> capi.cc_uploader
  • diego.nsync -> capi.nsync
  • diego.stager -> capi.stager
  • diego.tps -> capi.tps
• Gorouter property changes:
  • Removed router.skip_oauth_tls_verification
  • Removed metron_endpoint.host
  • Removed metron_endpoint.port
  • Removed metron_endpoint.dropsonde_port
  • Removed dropsonde.enabled
  • Renamed routing-api.port to routing_api.port
  • Renamed routing-api.auth_disabled to routing_api.auth_disabled
  • Added metron.port

Recommended BOSH Stemcell Versions

• AWS: light-bosh-stemcell-3232.11-aws-xen-hvm-ubuntu-trusty-go_agent
• vSphere: bosh-stemcell-3232.11-vsphere-esxi-ubuntu-trusty-go_agent
• OpenStack: N/A
• BOSH-Lite: bosh-stemcell-3147-warden-boshlite-ubuntu-trusty-go_agent

These are soft recommendations; several different versions of the stemcells are likely to work fine with this version of cf-release and the corresponding versions of the additional releases listed below.

Recommended Versions of Additional Releases

These versions are soft recommendations, as several different versions of these releases may work correctly with this version of cf-release. - Diego release v0.1476.0. Release notes for v0.1476.0 · v0.1475.0 · v0.1474.0 · v0.1473.0. - Garden-Linux release v0.338.0. Release notes for v0.338.0. - etcd release v57. Release notes for v57 · v56 · v55 · v54 · v53 · v52 · v51 · v50. - cflinuxfs2-rootfs release v1.16.0. Release notes for v1.16.0 · v1.15.0 · v1.14.0 · v1.13.0 · v1.12.0 · v1.11.0 · v1.10.0 · v1.9.0 · v1.8.0 · v1.7.0 · v1.6.0.

- name: "cf"
  version: "238"
  url: "https://bosh.io/d/github.com/cloudfoundry/cf-release?v=238"
  sha1: sha256:

bosh upload-release --sha1=sha256: \
  "https://bosh.io/d/github.com/cloudfoundry/cf-release?v=238"