== Meeting information == * #cloud-init: Cloud-init bi-weekly status, 13 May at 16:25 — 17:32 UTC * Full logs at [[http://ubottu.com/meetingology/logs/cloud-init/2019/cloud-init.2019-05-13-16.25.log.html]] == Meeting summary == === Previous Actions === The discussion about "Previous Actions" started at 16:31. * ''LINK:'' https://cloud-init.github.io/status-2019-01-28.html#status-2019-01-28 * ''LINK:'' https://bugs.launchpad.net/cloud-init/+bug/1801364 * ''ACTION:'' Odd_Bloke confirm correct status of https://bugs.launchpad.net/cloud-init/+bug/1801364 === Recent Changes === The discussion about "Recent Changes" started at 16:35. * ''LINK:'' https://launchpad.net/cloud-init/+milestone/19.1 === In-progress Development === The discussion about "In-progress Development" started at 16:46. * ''LINK:'' https://wiki.ubuntu.com/CloudinitUpdates * ''LINK:'' https://trello.com/c/oxv61bPl/1099-sru-191-xenial-bionic-cosmic-and-disco * ''LINK:'' https://copr.fedorainfracloud.org/coprs/g/cloud-init/el-testing/ * ''LINK:'' https://code.launchpad.net/~chad.smith/cloud-init/+git/cloud-init/+merge/367301 * ''LINK:'' https://code.launchpad.net/~chad.smith/cloud-init/+git/cloud-init/+merge/367302 * ''LINK:'' https://code.launchpad.net/~chad.smith/cloud-init/+git/cloud-init/+merge/367297 * ''LINK:'' https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1828637 * ''LINK:'' https://trello.com/b/hFtWKUn3/daily-cloud-init-curtin === Office hours (next ~30 mins) === The discussion about "Office hours (next ~30 mins)" started at 17:02. * ''LINK:'' https://cloudinit.readthedocs.io/en/latest/topics/tests.html# * ''LINK:'' https://cloud-init.github.io * ''ACTION:'' Odd_Bloke/rharper/AnhVoMSFT followup on recommended approach for getting kernel/userspace timestamp with or without python-dbus package dependency == Vote results == == Action items, by person == * AnhVoMSFT * Odd_Bloke/rharper/AnhVoMSFT followup on recommended approach for getting kernel/userspace timestamp with or without python-dbus package dependency == Done items == * (none) == People present (lines said) == * blackboxsw (163) * AnhVoMSFT (20) * ubot5` (19) * meetingology (5) * powersj (3) == Full Log == 16:25 #startmeeting Cloud-init bi-weekly status 16:25 Meeting started Mon May 13 16:25:37 2019 UTC. The chair is blackboxsw. Information about MeetBot at http://wiki.ubuntu.com/meetingology. 16:25 16:25 Available commands: action commands idea info link nick 16:26 hi folks, we are restarting the cloud-init community status meeting so we can have a community forum to discuss current cloud-init development and try to improve momentum in the cloud-init project efforts. 16:28 over the last couple of months we had taken a break from this meeting. But, I think it's shown us that there is demand and usefulness in continuing hosting these meetings. So, from here on, we expect to post the upcoming date of status meetings in this channel's room topic. 16:29 Again the goal of this meeting is to give people a chance to see what work is currently being performed on upstream cloud-init as well as a forum to discuss or raise concerns or get eyes on potential bugs or features. 16:30 Generally the topics we would like to cover are the following: (and interjections/suggestions are always welcome 16:30 topics: Previous Actions, Recent Changes, In-progress Development, Office Hours. 16:31 #topic Previous Actions 16:32 so, looking at our last status minutes from the last meeting, (on Jan 28.... :( ) we had one action carried over 16:32 #link https://cloud-init.github.io/status-2019-01-28.html#status-2019-01-28 16:33 review and land https://code.launchpad.net/~rjschwei/cloud-init/+git/cloud-init/+merge/356098 16:33 that's done. as well as an openstack bug... 16:33 #link https://bugs.launchpad.net/cloud-init/+bug/1801364 16:33 Ubuntu bug 1801364 in cloud-init "persisting OpenStack metadata fails" [Undecided,Confirmed] 16:33 Odd_Bloke: has landed this bug, I think it needs to be marked fix committed though 16:34 Odd_Bloke: is out today at a meeting, but I'll mark that action so we can make sure that bug status is properly reported (if there are no other branches to land for it) 16:34 #action Odd_Bloke confirm correct status of https://bugs.launchpad.net/cloud-init/+bug/1801364 16:34 * meetingology Odd_Bloke confirm correct status of https://bugs.launchpad.net/cloud-init/+bug/1801364 16:34 Ubuntu bug 1801364 in cloud-init "persisting OpenStack metadata fails" [Undecided,Confirmed] 16:34 That;s about it for actions from previous meeting. 16:35 #topic Recent Changes 16:36 Here we list current content that has landed in cloud-init since last meeting..... because it has been a really long time since last meeting, I'm going to redact this to the last month or so of content to avoid getting banned in this channel for spam 16:37 - Azure: Return static fallback address as if failed to find endpoint 16:37 [Jason Zions (MSFT)] 16:37 - release 19.1 (LP: #1828479) 16:37 - freebsd: add chpasswd pkg in the image [Gonéri Le Bouder] 16:37 - tests: add Eoan release [Paride Legovini] 16:37 Launchpad bug 1828479 in cloud-init "Release 19.1" [Undecided,Fix released] https://launchpad.net/bugs/1828479 16:37 hah, I don't think that worked 16:37 ok breaking that content up 16:37 - Azure: Return static fallback address as if failed to find endpoint 16:37 [Jason Zions (MSFT)] 16:37 - release 19.1 (LP: #1828479) 16:37 - freebsd: add chpasswd pkg in the image [Gonéri Le Bouder] 16:37 - tests: add Eoan release [Paride Legovini] 16:37 - cc_mounts: check if mount -a on no-change fstab path 16:37 [Jason Zions (MSFT)] (LP: #1825596) 16:37 - replace remaining occurrences of LOG.warn [Daniel Watkins] 16:37 Launchpad bug 1825596 in cloud-init "Azure reboot with unformatted ephemeral drive won't mount reformatted volume" [High,Fix released] https://launchpad.net/bugs/1825596 16:37 - DataSourceAzure: Adjust timeout for polling IMDS [Anh Vo] 16:37 - Azure: Changes to the Hyper-V KVP Reporter [Anh Vo] 16:37 - git tests: no longer show warning about safe yaml. 16:37 - tools/read-version: handle errors [Chad Miller] 16:37 - net/sysconfig: only indicate available on known sysconfig distros 16:37 (LP: #1819994) 16:37 - packages: update rpm specs for new bash completion path 16:37 [Daniel Watkins] (LP: #1825444) 16:37 - test_azure: mock util.SeLinuxGuard where needed 16:37 [Jason Zions (MSFT)] (LP: #1825253) 16:37 - setup.py: install bash completion script in new location [Daniel Watkins] 16:37 - mount_cb: do not pass sync and rw options to mount 16:37 Launchpad bug 1819994 in Provider for Plainbox - Canonical Certification Server "cloud-init selects sysconfig netconfig renderer if network-manager is installed on Ubuntu" [Critical,Confirmed] https://launchpad.net/bugs/1819994 16:37 [Gonéri Le Bouder] (LP: #1645824) 16:37 - cc_apt_configure: fix typo in apt documentation [Dominic Schlegel] 16:37 - Revert "DataSource: move update_events from a class to an instance..." 16:37 [Daniel Watkins] 16:37 Launchpad bug 1825444 in cloud-init (Ubuntu) ""make rpm" fails on CentOS after commit 947d3c208" [Undecided,Fix released] https://launchpad.net/bugs/1825444 16:37 - Change DataSourceNoCloud to ignore file system label's case. 16:37 [Risto Oikarinen] 16:38 Launchpad bug 1825253 in cloud-init "Unit tests with filesystem-related mocks fail in SeLinuxGuard when run on RHEL or CentOS" [Undecided,Fix released] https://launchpad.net/bugs/1825253 16:38 Launchpad bug 1645824 in cloud-init "NoCloud source doesn't work on FreeBSD" [Medium,Fix released] https://launchpad.net/bugs/1645824 16:38 - cmd:main.py: Fix missing 'modules-init' key in modes dict 16:38 [Antonio Romito] (LP: #1815109) 16:38 - ubuntu_advantage: rewrite cloud-config module 16:38 - Azure: Treat _unset network configuration as if it were absent 16:38 [Jason Zions (MSFT)] (LP: #1823084) 16:38 - DatasourceAzure: add additional logging for azure datasource [Anh Vo] 16:38 - cloud_tests: fix apt_pipelining test-cases 16:38 Launchpad bug 1815109 in cloud-init "cloud-final.service: "cloud-init modules --mode final" exit with "KeyError: 'modules-init'" after upgrade to version 18.2" [Medium,Fix released] https://launchpad.net/bugs/1815109 16:38 - Azure: Ensure platform random_seed is always serializable as JSON. 16:38 [Jason Zions (MSFT)] 16:38 - net/sysconfig: write out SUSE-compatible IPv6 config [Robert Schweikert] 16:38 - tox: Update testenv for openSUSE Leap to 15.0 [Thomas Bechtold] 16:38 Launchpad bug 1823084 in cloud-init "DataSourceAzure doesn't rebuild network-config after reboot" [High,Fix released] https://launchpad.net/bugs/1823084 16:38 - net: Fix ipv6 static routes when using eni renderer 16:38 [Raphael Glon] (LP: #1818669) 16:38 Launchpad bug 1818669 in cloud-init "ipv6 static routes configured for eni are incorrect" [Undecided,Fix released] https://launchpad.net/bugs/1818669 16:38 - Add ubuntu_drivers config module [Daniel Watkins] 16:38 - doc: Refresh Azure walinuxagent docs [Daniel Watkins] 16:38 - tox: bump pylint version to latest (2.3.1) [Daniel Watkins] 16:38 - DataSource: move update_events from a class to an instance attribute 16:38 [Daniel Watkins] (LP: #1819913) 16:38 - net/sysconfig: Handle default route setup for dhcp configured NICs 16:38 [Robert Schweikert] (LP: #1812117) 16:38 - DataSourceEc2: update RELEASE_BLOCKER to be more accurate 16:38 Launchpad bug 1819913 in cloud-init "cloud-init on xenial may generate network config on every boot" [Undecided,Fix released] https://launchpad.net/bugs/1819913 16:38 [Daniel Watkins] 16:38 - cloud-init-per: POSIX sh does not support string subst, use sed 16:38 Launchpad bug 1812117 in cloud-init "route files are not written on SUSE distros" [Undecided,Fix released] https://launchpad.net/bugs/1812117 16:38 (LP: #1819222) 16:38 - Support locking user with usermod if passwd is not available. 16:38 Launchpad bug 1819222 in cloud-init "cloud-init-per no longer works due to bashisms" [Undecided,Fix released] https://launchpad.net/bugs/1819222 16:38 - Example for Microsoft Azure data disk added. [Anton Olifir] 16:38 - clean: correctly determine the path for excluding seed directory 16:38 [Daniel Watkins] (LP: #1818571) 16:38 - helpers/openstack: Treat unknown link types as physical 16:38 Launchpad bug 1818571 in cloud-init "cloud-init clean removes seed directory even when --seed is not specified" [High,Fix released] https://launchpad.net/bugs/1818571 16:38 [Daniel Watkins] (LP: #1639263) 16:38 Launchpad bug 1639263 in cloud-init " cloud-init Unknown network_data link type: macvtap" [Undecided,Fix released] https://launchpad.net/bugs/1639263 16:38 - drop Python 2.6 support and our NIH version detection [Daniel Watkins] 16:38 - tip-pylint: Fix assignment-from-return-none errors 16:38 - net: append type:dhcp[46] only if dhcp[46] is True in v2 netconfig 16:38 [Kurt Stieger] (LP: #1818032) 16:38 - cc_apt_pipelining: stop disabling pipelining by default 16:38 [Daniel Watkins] (LP: #1794982) 16:38 - tests: fix some slow tests and some leaking state [Daniel Watkins] 16:38 Launchpad bug 1818032 in cloud-init "sysconfig renders BOOTPROTO=dhcp even if dhcp=false in v2 network-config" [High,Fix released] https://launchpad.net/bugs/1818032 16:38 - util: don't determine string_types ourselves [Daniel Watkins] 16:38 Launchpad bug 1794982 in cloud-init "drop /etc/apt/apt.conf.d/90cloud-init-pipelining in 16.04+" [Medium,Fix released] https://launchpad.net/bugs/1794982 16:39 Azure and SuSE and FreeBSD foks have been quite a bit of work and we have a number of other community contributors to thank this round 16:41 AnhVoMSFT: jasonzio (MSFT) and robjo (SuSE) thanks again for your efforts 16:43 Also, as you can see, Canonical added someone to our ranks for cloud-init development. Odd_Bloke is a permanent addition to our team and landing a lot of solid content for cloud-init. 16:43 welcome Dan! 16:43 thanks rharper Odd_Bloke powersj blackboxsw for all the reviewing and advice 16:44 From upstream side of the house, we have cut our first official upstream release of cloud-init. Version 19.1 sports 5 months of development, 24 contributors from 20 top-level domains. Details of that release are sent to our mailinglist and posted to the luanchpad milestone 16:44 #link https://launchpad.net/cloud-init/+milestone/19.1 16:46 :) ! it's nice how involved your teams have been thanks again. 16:46 #topic In-progress Development 16:47 So, for the upstream release of 19.1: Canonical has just started our SRU (Stable Release Update) process where we validate the upstream release before pushing it back into earlier Ubuntu series 16:47 We follow this upstream release process for Ubuntu 16:47 #link https://wiki.ubuntu.com/CloudinitUpdates 16:48 I've queue to release branches for review as of Friday 05/10. I have queued Ubuntu Disco and published to disco-proposed pocket already. 16:48 So if someone deploys disco on your favorite cloud or hardware, you can add a disco-proposed apt source to /etc/apt/sources.list and you sound be able to install and or test 19.1 16:49 I made the 19.1.1 SRU cut which included the commitish baa478546d8cac98a706010699d64f8c2f70b5bf which was the tip containing " Azure: Return static fallback address as if failed to find endpoint" as the last commit 16:49 so jasonzio's latest azure branch made it 16:50 expectation is that I'll get most of verification done over this week and we can hopefully (pending no SRU regressions discovered) publish to Xenial, Bionic, Cosmic and Disco the beginning of next week (05/20/2019) 16:51 I just added the following card to track this work, it'll move to DONE when published 16:51 #link https://trello.com/c/oxv61bPl/1099-sru-191-xenial-bionic-cosmic-and-disco 16:51 AnhVoMSFT, thank you for all your contributions 16:52 is there a package available publicly for 19.1 that we can download and apply it to existing images and run through our perf. regression test? 16:52 also today, we'll post 19.1 to our copr el-testing repo at 16:52 #link https://copr.fedorainfracloud.org/coprs/g/cloud-init/el-testing/ 16:52 for centos. 16:53 AnhVoMSFT, daily PPA: https://launchpad.net/~cloud-init-dev/+archive/ubuntu/daily 16:53 AnhVoMSFT: we have daily repos that 16:53 thx powersj 16:53 * powersj disappears 16:54 AnhVoMSFT, so that is built from tip of master (which currently contains exactly the same commit set as will be published to each series) 16:54 AnhVoMSFT: once the following branches are "merged" for each series, xenial-proposed, bionic-proposed, cosmic-proposed will also contain the final bits for each series 16:54 thanks powersj blackboxsw 16:55 #link https://code.launchpad.net/~chad.smith/cloud-init/+git/cloud-init/+merge/367301 16:55 #link https://code.launchpad.net/~chad.smith/cloud-init/+git/cloud-init/+merge/367302 16:55 #link https://code.launchpad.net/~chad.smith/cloud-init/+git/cloud-init/+merge/367297 16:55 so when we land those, it should be a matter of an hour or so until -proposed is accessible to test those final packages 16:56 AnhVoMSFT: in the SRU process, we still will have to verify and test those bits before release. There is a tracking bug for the overall process here 16:56 #link https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1828637 16:56 Ubuntu bug 1828637 in cloud-init (Ubuntu) "sru cloud-init (18.5-45 to 19.1.1) Xenial, Bionic, Cosmic, Disco" [Undecided,In progress] 16:56 ^ this but will go to fix released when we pass verification and publish the SRU 16:57 which will be promotion from xenial-proposed to xenial-updates for public availability on all images 16:57 again the estimate on this is probably next week for getting through testing 16:58 the rest of the cloud-init team is meeting this week for feature planning for the next cycle. But generally we will be tracking our ongoing work @ our public trello board 16:59 #link https://trello.com/b/hFtWKUn3/daily-cloud-init-curtin 17:00 If there are there any features or significant gaps that people have interest in that we could try to put on our radar feel free to drop us a line here in channel or on the mailinglist @ cloud-init@lists.launchpad.net 17:01 I think that about wraps up current in-flight work 17:02 #topic Office hours (next ~30 mins) 17:03 This section is allocated generally for folks to chat about any burning questions that we have. Bring your topics, bugs, feature requests, merge proposals or questions and we'll try getting some eyes or discussion on the 17:03 *them. 17:04 Generally we should have a couple cloud-init devs hanging out in channel during this time to help build momentum or unblock folks. Today may be a bit light in attendance due to a remote work trip a number of people are attending. 17:04 if our next work requires importing another module (python-dbus), are there any guidelines wrt to using additional python modules not currently part of cloudinit package? 17:06 AnhVoMSFT: generally adding new dependencies is something we try to avoid if possible due to impacting the minimal cloud images with additional dependencies. From ubuntu's perspective we tend to try to look at an ubuntu minimal image to see if the dependency is already there (and that makes things simple for the addition). 17:06 If new dependency is also something that can be optional instead of strict it makes adding that dependency easier. 17:06 For example: python-jsonschema is an optional package dependency 17:07 we bookend the imports in cloud-init with a try/except ImportError: pass and make sure that cloud-init still behaves if it is not there. 17:07 i will have to look into how to make it optional - certainly we can go that way if that makes thing easier 17:07 what we are looking at is how to get the kernel init timestamp and systemd init timestamp - both of these are available as KernelTimestamp and UserSpaceTimestamp that are available as part of the systemd's system dbus 17:07 This type of approach has been helpful in supporting Centos 6 where jsonschema wasn't available in stock images 17:08 normally you can get those info from "systemctl show", but the issue is systemctl internally use systemd's format_timestamp, which trims the precision down to "seconds" level. (format_timestamp_us would give us what we need) 17:11 AnhVoMSFT: interesting. since it seems like the dependency is one one or two values and not the full content of python-dbus library/modules I wonder if it might be better to see if those values are available in /proc or /sys somewhere as it might be fairly simple to instrument our own. (and avoid the time cost of importing larger python modules) 17:11 I guess if we really want to avoid another module, we can use os.stat to key off ctime of /proc and /proc/1 - however, the ctimes of /proc and /proc/1 are a bit earlier than the actual KernelTimestamp and UserSpaceTimestamp. They do have the advantage of being more widely available. Systemd and dbus might not exist on all distros 17:11 ... at least something to peek at to see if it is low hanging fruit that is easy to handle 17:12 Right, I would appreciate if someone with more knowledge on Linux systems can provide some advices here 17:12 the dependency is really on those values, and the dbus library seems a bit overkill for it 17:13 rharper: Odd_Bloke may have some thoughts on that too. but that conversation may have to wait a bit due to time difference. 17:14 AnhVoMSFT: again, what call/values would you like to make/get from python-dbus just for clarity 17:15 bus=dbus.SystemBus() systemd1=bus.get_object('org.freedesktop.systemd1','/org/freedesktop/systemd1') manager=dbus.Interface(systemd1, 'org.freedesktop.systemd1.Manager') props_iface=dbus.Interface(manager,'org.freedesktop.DBus.Properties') props_iface.Get('org.freedesktop.systemd1.Manager','KernelTimestamp') 17:15 oops - multiline didn't seem to go so well 17:15 it's all good thx 17:16 alternately, I support we could shell out via util.subp to make a cli call to handle that to avoid a python package dependency, but again it'd probably be better/faster to avoid that 17:17 right, we're trying to avoid adding more util.subp call (in fact we have a workitem to reduce the numbers of the existing util.subp call the Azure DS makes) 17:18 Like mentioned previously, we're looking at the /proc and /proc/1 ctime, they're cheap using the os.stat module and exist in all distros. 17:19 oops typo "alternatively , I support" /// I suppose 17:19 yeah that seems like the most non-contentious approach (processing /proc/1 ctime) 17:20 uncontentious rather 17:22 another thing we're looking at is cloud-init cloudtests so that we can have some azure cloud test running as part of cloud-init check-in gate 17:22 excellent! 17:23 if you guys have any pointers/tips on what it takes to enable cloud-tests please let us know 17:23 one of our team members will be looking at it 17:24 any blockers that you are aware of to date? Generally the biggest lift is getting the platform support added to the cloud-tests 17:25 I know powersj added more CI docs related to this a few months ago at 17:25 #link https://cloudinit.readthedocs.io/en/latest/topics/tests.html# 17:25 I'll try to see if there was additional documentation tossed around 17:26 the benefit if Azure platform is added to cloud-init tip is that'd we would likely add some of that testing to our SRU verification process each release we are doing. I'm excited about that 17:27 right now, it's manual for Azure testing... per something like this https://github.com/cloud-init/ubuntu-sru/blob/master/manual/azure-sru-18.5.21.txt 17:27 I'm not aware of any blockers, we just barely started looking at it 17:27 these links are very helpful, I'll pass them along 17:28 the above script is tedious to write up, and only is really testing upgrade path and any *new* azure-specific bugs/features that are fixed in the current SRU, since most of the other integration testing covers general case 17:28 thanks AnhVoMSFT that's good to hear 17:29 are there any other topics folks have for today's meeting? Otherwise we can adjourn until next meeting 17:29 next meeting will be May 27th, same time 17:30 and I'll post status notes for the meeting to our github pages 17:30 #link https://cloud-init.github.io 17:31 before I forget. let's get an action to followup on kernel/userspace timestamp discussion 17:31 thanks blackboxsw 17:32 #action Odd_Bloke/rharper/AnhVoMSFT followup on recommended approach for getting kernel/userspace timestamp with or without python-dbus package dependency 17:32 * meetingology Odd_Bloke/rharper/AnhVoMSFT followup on recommended approach for getting kernel/userspace timestamp with or without python-dbus package dependency 17:32 thank you AnhVoMSFT 17:32 see folks next time, or anytime in channel 17:32 #endmeeting Generated by MeetBot 0.1.5 (http://wiki.ubuntu.com/meetingology)