16:04 <blackboxsw> #startmeeting Cloud-init bi-weekly status meeting
16:04 <meetingology> Meeting started Mon Apr 30 16:04:15 2018 UTC.  The chair is blackboxsw. Information about MeetBot at http://wiki.ubuntu.com/meetingology.
16:04 <meetingology> 
16:04 <meetingology> Available commands: action commands idea info link nick
16:04 <blackboxsw> hi folks, sorry for the mis-representation of when our cloud-init status meeting date. It's time for another episode/update of the happenings in cloud-init.
16:05 <blackboxsw> Next meeting will be in two weeks: May 7th
16:05 <blackboxsw> at 16:00 UTC
16:06 <blackboxsw> The last couple weeks on the upstream side of the house has been a big push to get testing and stability into master for the Ubuntu Bionic release freeze
16:06 <blackboxsw> ... I'd better start with the topic
16:07 <blackboxsw> #topic Recent Changes
16:07 <blackboxsw> The last couple weeks on the upstream side of the house has been a big push to get testing and stability into master for the Ubuntu Bionic release freeze.
16:07 <robjo> May 7th would be 1 week from today that should be May 14th
16:07 <blackboxsw> robjo: gah, I did it again. Thank you... glad someone's listening.    Next cloud-init status meeting Monday May 14th 16:00 UTC
16:08 <blackboxsw> #topic #cloud-init Reviews: http://bit.ly/ci-reviews | Meeting minutes: https://goo.gl/mrHdaj | Next status meeting: Monday 5/14 16:00 UTC | cloud-init 18.2 released (03/28/2018)
16:08 <blackboxsw> ok topic agrees in channel now, so I don't botch it at the end of meeting
16:09 <blackboxsw> Along with a blitz for stability in Bionic the following changes have been shepherded into tip of master
16:09 <blackboxsw> - Add reporting events and log_time around early source of blocking time
16:09 <blackboxsw> [Ryan Harper]
16:09 <blackboxsw> - IBMCloud: recognize provisioning environment during debug boots.
16:09 <blackboxsw> (LP: #1767166)
16:09 <blackboxsw> - net: detect unstable network names and trigger a settle if needed
16:09 <blackboxsw> [Ryan Harper] (LP: #1766287)
16:09 <blackboxsw> - IBMCloud: improve documentation in datasource.
16:09 <ubot5> Launchpad bug 1767166 in cloud-init (Ubuntu) "IBMCloud datasource does not recognize provisioning in debug mode." [Medium,Confirmed] https://launchpad.net/bugs/1767166
16:09 <blackboxsw> - sysconfig: dhcp6 subnet type should not imply dhcpv4 [Vitaly Kuznetsov]
16:09 <blackboxsw> - packages/debian/control.in: add missing dependency on iproute2.
16:09 <ubot5> Launchpad bug 1766287 in cloud-init (Ubuntu) "18.04 minimal images on GCE intermittently fail to set up networking " [Undecided,In progress] https://launchpad.net/bugs/1766287
16:09 <blackboxsw> (LP: #1766711)
16:09 <blackboxsw> - DataSourceSmartOS: add locking of serial device.
16:09 <blackboxsw> [Mike Gerdts] (LP: #1746605)
16:09 <blackboxsw> - DataSourceSmartOS: sdc:hostname is ignored [Mike Gerdts] (LP: #1765085)
16:09 <ubot5> Launchpad bug 1766711 in cloud-init (Ubuntu Bionic) "cloud-init missing dependency on iproute2" [Medium,Fix committed] https://launchpad.net/bugs/1766711
16:09 <blackboxsw> - DataSourceSmartOS: list() should always return a list
16:09 <blackboxsw> [Mike Gerdts] (LP: #1763480)
16:09 <ubot5> Launchpad bug 1746605 in cloud-init "DataSourceSmartOS needs locking" [Medium,Fix committed] https://launchpad.net/bugs/1746605
16:09 <blackboxsw> - schema: in validation, raise ImportError if strict but no jsonschema.
16:09 <blackboxsw> - set_passwords: Add newline to end of sshd config, only restart if
16:09 <blackboxsw> updated. (LP: #1677205)
16:09 <ubot5> Launchpad bug 1765085 in cloud-init "DataSourceSmartOS ignores sdc:hostname" [Medium,Fix committed] https://launchpad.net/bugs/1765085
16:09 <blackboxsw> - pylint: pay attention to unused variable warnings.
16:09 <blackboxsw> - doc: Add documentation for AliYun datasource. [Junjie Wang]
16:09 <blackboxsw> - Schema: do not warn on duplicate items in commands. (LP: #1764264)
16:09 <ubot5> Launchpad bug 1763480 in cloud-init "DataSourceSmartOS list() should always return a list" [Medium,Fix committed] https://launchpad.net/bugs/1763480
16:09 <ubot5> Launchpad bug 1677205 in cloud-init "cloud-init eats final EOL of sshd_config" [Medium,Fix committed] https://launchpad.net/bugs/1677205
16:09 <ubot5> Launchpad bug 1764264 in juju 2.3 "bionic cloud-init 18.2 WARNING Juju's 'runcmd' stanza" [High,Triaged] https://launchpad.net/bugs/1764264
16:10 <blackboxsw> the general theme has been: new IBMCloud datasource support for cloud-init, SmartOS datasource work by mgerdts, and some json schema improvements
16:12 <blackboxsw> so background on IBM, is that their support used to be ConfigDrive based datasource only, but there is now some additional support for different IBM boot/provisioning  stages, hence a new datasource that can support different boot modew
16:12 <blackboxsw> *boot modes
16:14 <blackboxsw> over the last two weeks we've landed an SRU into xenial and artful: 18.2-4-g05926e48-0ubuntu1~16.04.1  and   bionic sits at 18.2-14-g6d48d265-0ubuntu1
16:15 <mgerdts> On the SmartOS side, my changes are driven by our adoption of bhyve (moving away from kvm/qemu).  qemu provides a dhcp server VMs could fall back to if could-init was missing or misbehaving.  bhyve doesn't have that, so I've been working on getting cloud-init to be more stable with the bhyve serial metadata service.
16:15 <blackboxsw> Also, to our continuous integration on jenkins we now have an additional test for proposed packages in ubuntu for the bionic release to make sure ubuntu doesn't break across pending upgrades
16:15 <blackboxsw> #link https://jenkins.ubuntu.com/server/job/cloud-init-integration-proposed-b/
16:16 <blackboxsw> that integration tests hits the suite of platforms lxd, kvm and ec2
16:16 <blackboxsw> excellent mgerdts, and thanks for the blitz on these branches
16:17 <blackboxsw> looks like there are a few still in our review queue that we'll be able to get through once the dust settles on the bionic release  (which should be this week)
16:17 <blackboxsw> #link https://code.launchpad.net/~cloud-init-dev/cloud-init/+git/cloud-init/+ref/master/+activereviews
16:19 <mgerdts> Is now the right time to discuss bug 1765801, or is that later?
16:19 <ubot5> bug 1765801 in cloud-init "network should be optionally reconfigured on every boot" [Undecided,Confirmed] https://launchpad.net/bugs/1765801
16:20 <blackboxsw> I think over the last 2 weeks there have been a couple of requests in channel for how someone goes about getting newer cloud init into RHEL7, if anyone on the line today knows the contact point or process for that it'd be helpful. larsks doesn't seem to be around
16:20 <blackboxsw> mgerdts: probably in about 10 mins. thanks for brining it up
16:20 <blackboxsw> hopefully less.
16:20 <blackboxsw> ok I think that's it for recent changes, next topic (in-progress dev, then office hours (and bug discussion))
16:21 <blackboxsw> #topic In-progresss Development
16:21 <blackboxsw> We'll make this one short:
16:22 <blackboxsw> for ubuntu : bionic just went feature freeze last week, our team has a couple of IBM-related cheanges that we are pulling together for a quick SRU into xenial/artful to handle upgrade path from configdrive -> IBMCloud that we are working on the beginning of this week
16:22 <blackboxsw> we are also trying to wrap up validation of a Bionic SRU per the following bug
16:22 <blackboxsw> #link https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1767412
16:22 <ubot5> Launchpad bug 1767412 in cloud-init (Ubuntu Bionic) "SRU cloud-init 18.2-27-g6ef92c98-0ubuntu1" [Medium,Fix committed]
16:23 <blackboxsw> which grabs a number of the updates I listed in the last topic
16:23 <blackboxsw> since Ubuntu tends to sync all changes from tip into each release stream
16:23 <mgerdts> Is there any chance the SmartOS changes can piggy back on that IBM SRU
16:24 <mgerdts> asked too soon - I see they are mentioned in that bug.
16:25 <blackboxsw> mgerdts: no worries. good ask. probably not for this IBM SRU into xenial/artful which is going to be an exception to our update rule and only be a single cherry pick, but planning a folllowup SRU in about 2 weeks which will pull all changes from tip into artful/xenial/bionic/chunky releases
16:25 <mgerdts> ok
16:25 <blackboxsw> the cherry pick is to fasttrack it for IBM into xenial with minimal risk.
16:25 <blackboxsw> and we want to pull in all your changes if we can (and perform additional validation)
16:25 <blackboxsw> so the next SRU is our target
16:26 <blackboxsw> Also inprogress is some more Azure work on pre-provisioning that should land shortly:
16:26 <blackboxsw> #link https://code.launchpad.net/~jocha/cloud-init/+git/cloud-init/+merge/344192
16:27 <blackboxsw> as well as some builddeb fixes and network configuration printout fixes from smoser
16:29 <blackboxsw> smoser and rharper also worked out  some issues on specific google regions where cloud-init was getting hit by a race condition. Cloud-init started up before the kernel/udev was able to rename network devices to stable names like ens4 etc, so cloud-init's network configuration written ended up breaking because it represented devices like eth0 etc.
16:29 <blackboxsw> there are a couple of branches in flight to fix this issue:
16:29 <blackboxsw> #link https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/344181
16:30 <blackboxsw> #link https://code.launchpad.net/~raharper/cloud-init/+git/cloud-init/+merge/344198
16:30 <blackboxsw> ok I think that's it for in-progress work. So we'll head to office hours so we can chat bugs, branches reviews etc
16:31 <blackboxsw> #topic Office Hours (next ~30 mins)
16:31 <blackboxsw> We'll be hanging out here for anyone who wants more eyes on a review, feature discussions or bug triage....
16:32 <blackboxsw> well, some of us will be :) a couple of us are at a feature planning conference for the week.
16:33 <mgerdts> In https://code.launchpad.net/~mgerdts/cloud-init/+git/cloud-init/+merge/343712 smoser said that he was concerned about how this would interact with eventual network hotplug
16:34 <mgerdts> There doesn't seem to be a timeline for network hotplug and the lack of network autoreconfig on reboot is has popped up a couple times in the past week.  This is just with a couple early adopters and internal users.
16:34 <blackboxsw> #link https://code.launchpad.net/~mgerdts/cloud-init/+git/cloud-init/+merge/343712
16:35 <blackboxsw> just to track it in the meeting
16:35 <mgerdts> So coming up with some mechanism to make this work soon is pretty important to us.
16:35 <mgerdts> gotcha, will be sure to do that in the future.
16:35 * blackboxsw reads up on that link
16:35 <blackboxsw> no worries, I'm pedantic :)
16:36 <mgerdts> That's how you got chosen to run the meeting, I suppose.  :)
16:36 <blackboxsw> yeah network hotplug will have a long tail as far as feature develpment (agreed). I believe it's on our charter for this next quarter. but that's what is being discussed this week
16:37 <blackboxsw> heh on meeting comment ;) too true
16:39 <blackboxsw> so mgerdts your branch allows metadata to set maintain_network to allow cloud-init to control network configuration each reboot with a True value
16:39 <blackboxsw> ?
16:39 <mgerdts> yes
16:39 <mgerdts> if it's not set to true in our metadata, the traditional behavior stays.
16:40 <mgerdts> That is, in the default path, any customization that someone does in the guest will not get whacked.
16:41 <robjo> cloud-netconfig handles hotplug https://github.com/SUSE/Enceladus/tree/master/cloud-netconfig contributions for other distros welcome
16:42 <blackboxsw> nice reference robjo
16:42 <blackboxsw> #link https://github.com/SUSE/Enceladus/tree/master/cloud-netconfig
16:43 <robjo> We currently have no GCE specific information but that is easy enough to add. The GCE guest environment handles this and we use the GCE guest environment code in our images in GCE
16:43 <blackboxsw> mgerdts: so can a user turn off that feature on an instance once they've already deployed, or is it create-time only
16:43 <mgerdts> It can be flipped at any time, in the current implementation.
16:44 <mgerdts> current implementation is only in a development branch
16:47 <blackboxsw> mgerdts: the only things I can see being an issue with the maintain network in cloud-init is that we are adding the cost of another function call  && metdata dict parse to look for a signal about maintaining the network. I agree that cloud-init having granularity between is_new_instance vs just re-do network, is something that cloud-init should have.
16:48 <blackboxsw> we probably need to discuss this too with rharper about what short-term vision we can get to while we await our network hotplug support in cloud-init proper
16:49 <blackboxsw> I'd tend to agree that waiting on fully baked hotplug solution is probably too long in this case
16:49 <blackboxsw> as that runway will be at least 2 months I'd think
16:50 <blackboxsw> ok, I'll take an action item to resolve this if we can by next meeting
16:50 <mgerdts> Not only that, but support for it will likely require changes in the host as well.  We tend not to do host updates very often, so it could be a year or more after the feature is available in images before it will be useful.
16:51 <blackboxsw> #action blackboxsw to have discussions w/ team on datasource maintaining network on each reboot per  https://code.launchpad.net/~mgerdts/cloud-init/+git/cloud-init/+merge/343712
16:51 * meetingology blackboxsw to have discussions w/ team on datasource maintaining network on each reboot per  https://code.launchpad.net/~mgerdts/cloud-init/+git/cloud-init/+merge/343712
16:51 <mgerdts> thanks
16:51 <blackboxsw> good topic.
16:52 <mgerdts> Is there another place that is good to catch up with larsks or other people that can offer guidance on for redhat/centos?
16:52 <blackboxsw> let's see, anything else folks want to chat about? stagnant reviews, bugs of interest etc?
16:53 * blackboxsw looks at the last cloud-init community summit attendees list to see if rhel folks have another contacts that was supposed to replace larsks
16:53 <stanguturi> Chad, Is it possible that someone from cloud-init team can take a look at https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1766538
16:53 <ubot5> Launchpad bug 1766538 in cloud-init (Ubuntu) "network customization with cloud-init does not work on Ubuntu18.04 Beta2 Server" [Undecided,Confirmed]
16:55 <blackboxsw> mgerts, ryan mccabe is a potential contact too, looks like he's not here either today.
16:56 <mgerdts> ok, thanks
16:56 <blackboxsw> hrm, yeah not certain what mechanism is used to get cloud-init updated into RedHat mgerdts. Maybe filing a redhat bug about the request
16:57 <blackboxsw> mgerdts: https://bugzilla.redhat.com/ maybe
16:58 <blackboxsw> stanguturi: yes we can, we are trying to sort and understand any bugs against Bionic that we can
16:58 <mgerdts> ok, I can try that.
16:58 <blackboxsw> #link https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1766538
16:58 <ubot5> Launchpad bug 1766538 in cloud-init (Ubuntu) "network customization with cloud-init does not work on Ubuntu18.04 Beta2 Server" [Undecided,Confirmed]
16:58 <stanguturi> blackboxsw: Thanks
17:00 <blackboxsw> stanguturi: ok, so this is netplan + cloud-init related right?
17:00 <stanguturi> blackboxsw: Yes.
17:01 <akik> what does network hotplug mean in cloud-init context?
17:02 * blackboxsw tries to remember what vmware datasource does, (like writing files direct to network /etc/network/interfaces.d)
17:02 <blackboxsw> akik: https://hackmd.io/M1Tae41PQBC7a9qMsurTJw?both  is a shared document for comment on hotplug in cloud-init
17:02 <blackboxsw> #link https://hackmd.io/M1Tae41PQBC7a9qMsurTJw?both
17:03 * blackboxsw looks to see if there was a better doc hrm
17:03 <stanguturi> blackboxsw: Oh. But in the case of netplan, why does cloud-init remembers?
17:04 <akik> blackboxsw: does it mean that cloud-init stays running, waiting for new network interfaces to appear?
17:04 <blackboxsw> akik: right, it would mean that you wouldn't have to reboot cloud-init if devices get added at a later time (post-boot)
17:05 <blackboxsw> cloud-init would listen to some sort of event channel and react, re-write, and apply network config to add new devices
17:06 <akik> would it do the same thing as you could do with ansible or puppet? sorry i'm trying to understand why you would do it with cloud-init
17:08 <blackboxsw> akik: you would try to do it with cloud-init if you didn't want to rely on additional configuration management solutions if the only thing you needed was network config to reflect reality (not full system configuration and system automation)
17:09 * blackboxsw has more puppet/chef background than ansible.
17:09 <blackboxsw> cloud-init does currently detect and write network configuration based on what the user/cloud-metadata tell us is the proper config for the instance
17:10 <akik> i only thought of cloud-init to do the initial configuration
17:10 <blackboxsw> so it would follow that if the metadata could dynamically tell the instance that network config has changed, cloud-init should probably try to react to that to fix the config to match the updated network configuration
17:11 <blackboxsw> akik: correct. cloud-init current only handle initial boot config and leaves the rest up whatever mechanism someone uses to update detailed config after that boot
17:11 <akik> ok thanks
17:12 <blackboxsw> akik: and we'd make that feature configurable (handle hotplug:True/False) so if users have other services handling hotplug cloud-init wouldn't collide
17:13 <blackboxsw> ok I think we're hitting the end of office hours. please feel free to continue discussion, we all poke around here throughout the day as our primary means of communication
17:13 <blackboxsw> thanks robjo akik stanguturi and mgerdts for the lively discussion
17:13 <blackboxsw> stanguturi: I'll dig up more info on that bug today
17:13 <mgerdts> thank you
17:13 <blackboxsw> as always notes will be here
17:14 <blackboxsw> #link https://cloud-init.github.io/
17:14 <blackboxsw> #endmeeting