01:00:17 <cprofitt> #startmeeting ubuntu-friendly 01:00:17 <meetingology> Meeting started Wed Feb 6 01:00:17 2013 UTC. The chair is cprofitt. Information about MeetBot at http://wiki.ubuntu.com/meetingology. 01:00:17 <meetingology> 01:00:17 <meetingology> Available commands: #accept #accepted #action #agree #agreed #chair #commands #endmeeting #endvote #halp #help #idea #info #link #lurk #meetingname #meetingtopic #nick #progress #rejected #replay #restrictlogs #save #startmeeting #subtopic #topic #unchair #undo #unlurk #vote #voters #votesrequired 01:00:35 <cprofitt> #meetingtopic welcome back 01:00:46 <cprofitt> hello all -- and welcome to the Ubuntu Friendly meeting 01:01:00 <cprofitt> can everyone that is here for the meeting just let me know you are in the room 01:02:09 <SergioMeneses> o/ 01:02:12 <phillw> o/ 01:02:40 <cprofitt> thanks for coming everyone 01:02:51 <cprofitt> our agenda is: 01:02:59 <pleia2> o/ 01:03:15 <cprofitt> hexr, wiki cleanup, how to improve parts of the Ubuntu Friendly system 01:03:24 <cprofitt> your questions and any advice/ideas you all have 01:03:49 <cprofitt> baloons are you here? 01:03:56 <SergioMeneses> #link https://wiki.ubuntu.com/UbuntuFriendly/Meetings 01:04:29 <cprofitt> alright... I might have baloons take over if he comes in, but let me tell you a bit about hexr 01:04:51 <cprofitt> #meetingtopic hexr 01:05:12 <cprofitt> #link https://launchpad.net/hexr 01:06:30 <cprofitt> hexr, from my understanding, will potentially make use of the same data Ubuntu Friendly does, but it will do so to see what harware components have been tested 01:06:50 <cprofitt> so it will seek to find out of a specific Nvidia card has been tested or a specific wireless card 01:07:06 <cprofitt> and not look at systems, but components 01:07:21 <cprofitt> #link https://launchpad.net/~hexr-dev/+members#active 01:07:36 <cprofitt> the developers are listed on that page 01:08:24 <cprofitt> I think knowing that could help Ubuntu Friendly add some more functionality to the site -- where people might be able to sort systems based on a desired video card or wireless card... 01:08:43 <cprofitt> for now it is just something for us to know as we move forward 01:09:22 <cprofitt> #topic wiki clean up 01:09:41 <cprofitt> how many of you have had time to look at the wiki page for UF? 01:09:58 <cprofitt> #link https://wiki.ubuntu.com/UbuntuFriendly/ 01:10:53 <cprofitt> would anyone be willing to take on reviewing the pages to ensure that they are up-to-date? 01:11:00 <SergioMeneses> cprofitt, we can put some icons 01:11:04 <SergioMeneses> cprofitt, I can do it 01:11:30 <cprofitt> #action SergioMeneses will review the Ubuntu Friendly wiki 01:11:30 * meetingology SergioMeneses will review the Ubuntu Friendly wiki 01:11:34 <cprofitt> thanks SergioMeneses 01:11:45 <cprofitt> please feel free to ask for help from others as you do that 01:12:05 <cprofitt> if anyone else knows of pages that need updates please let SergioMeneses or me know 01:12:14 <SergioMeneses> cprofitt, ok 01:12:26 <chilicuil> ok =) 01:12:36 <cprofitt> #topic improvements 01:13:12 <cprofitt> I know from personal experience we have some things to improve moving forward 01:13:21 <cprofitt> phillw: did you want to discuss some of the thoughts you had? 01:13:56 <cprofitt> anyone else? 01:14:28 <cprofitt> #subtopic submissions 01:14:34 <phillw> cprofitt: just to mention that you and laptop team are quite similar and I hope there is a good flow of informations between you both :) 01:14:47 <cprofitt> phillw: I would hope so as well. 01:15:01 <cprofitt> One of the things that got me involved was submissions 01:15:29 <cprofitt> right now it is my understanding that submissions are not getting in to UF 01:15:35 <cprofitt> but I am not sure why that is 01:15:40 <pleia2> phillw: hm, I thought the laptop testing team was for testing Ubuntu on laptops for upcoming releases, not reporting on how well supported laptops are in current releases 01:16:31 <SergioMeneses> Im totally agree with pleia2, 01:17:10 <phillw> pleia2: ergo, one a release is out, that information is available to what is then the 'current' release? 01:17:21 <phillw> *once* 01:18:00 <cprofitt> I would like to see two things done to improve in regards to submissions 01:18:05 <pleia2> phillw: but it would be results for when it was in development, so maybe the graphics card for that one laptop finally did end up working once the release happened, the results would be from when it wasn't working :\ 01:18:15 <pleia2> for example 01:18:39 <phillw> i see, I'll have a chat with the laptop team :) 01:18:49 <cprofitt> pleia2: I agree.. the results from pre-release could would not be invluded in Ubuntu Friendly 01:19:19 <pleia2> they're both valueable and needed projects, for sure, I just thing they provide different data to different audiences 01:19:25 <pleia2> s/thing/think 01:19:28 <cprofitt> but the laptop testing team could, upon release, be a group of people who ran the tests for UF 01:19:36 <pleia2> cprofitt: indeed! 01:20:05 <cprofitt> and from my understanding the database we are pulling from might actually be the same -- but with different criteria for what information is included 01:20:13 <cprofitt> and also differences in how it gets displayed 01:20:21 <pleia2> it makes sense for both projects to test the same things 01:20:27 <cprofitt> but I would need one of the developers to give me more information on that 01:20:52 <cprofitt> I would like to see two things done to improve in regards to submissions 01:21:19 <cprofitt> 1. some system be put in place to make it easy for those submitting results to know that the results were processed 01:21:31 <cprofitt> if the result was invalid - let them know why 01:22:05 <cprofitt> as a person who has submitted results and never saw my results in UF I know how that can cause someone to lose interest in contributing 01:22:52 <pleia2> +1 01:23:13 <cprofitt> 2. I would also want to have a group of people that could look at invalid results to ensure that how the data is being processed is accurate -- that the automated system is not makring valid results as invalid 01:23:30 <cprofitt> this is particularly important as hardware changes... 01:23:52 <cprofitt> failure to test a card reader when there is no such device present should not invalidate results 01:24:04 <cprofitt> any other comments in regards to submissions? 01:24:58 <cprofitt> #subtopic checkbox 01:25:10 <cprofitt> that brings me to one more idea 01:25:24 <cprofitt> this one is not mine, but one that has been discussed in the qa channel 01:25:52 <cprofitt> there may be a need to refine checkbox a bit to allow for a 'simple' test 01:26:06 <cprofitt> or in the case of hexr a test of specific hardware 01:26:36 <cprofitt> the simple test would be an abbreviated test to make sure that basic things like wireless work etc. 01:27:07 <cprofitt> I also had cases when I tested where the automated test 'failed' itself, but I knew the device was working 01:27:39 <cprofitt> I think there should be some sort of mechanism for making a note of that issue built in to checkbox so that changes can be addressed with those issues 01:28:03 <cprofitt> having devices 'fail' a test when they are actually working undercuts people's faith in the process and the results. 01:28:20 <cprofitt> any thoughts in that area? 01:28:32 <roadmr> o/ 01:28:43 <cprofitt> yes, roadmr 01:29:02 <roadmr> hello! first, if an automated test fails when the device is working, that means a bug that we'd like to get fixed 01:29:21 <roadmr> I understand this adds a lot of friction to things, so maybe thinking of ways to make this more effortless may be worthwhile 01:29:31 * cprofitt nods 01:29:42 <roadmr> second, the automated tests are in general more reliable but they are also quite valuable in high-volume environments 01:30:07 <roadmr> if we assume that people can be trusted, we can potentially use manual test cases instead of automated ones that are most unreliable 01:30:30 <roadmr> even so, it would be good to collect some information that can be automatically analyzed, maybe server-side 01:30:46 <roadmr> this way we can potentially re-score a submission given the raw data, even after-the-fact 01:30:51 <pleia2> admittedly there is some trust to an extent already in the testing (you can hear a sound, etc) 01:30:59 <cprofitt> I too, would prefer to get the automated tests working... I think human interaction always introduces some potential for error 01:31:08 <roadmr> this would however introduce a "we're collecting stuff and sending it up to canonical" factor that some people don't like 01:31:19 <cprofitt> I do like rescoring a submission after the fact 01:31:33 <roadmr> so this is an area for improvement but we always have the option of doing this manually 01:31:47 <cprofitt> roadmr: in this case I think the 'collecting data' part is voluntary 01:31:54 <cprofitt> at least with checkbox 01:32:15 <roadmr> cprofitt: yes, well in order to determine what's going on, some extensive logs would need to be collected 01:32:22 * cprofitt nods 01:32:30 <roadmr> cprofitt: but keeping the user in control would be the best 01:32:48 <roadmr> pleia2: hehe, we sought to remove the human factor from that by adding an automated audio_test :) so the computer listens to itself 01:32:56 <cprofitt> could that be an optional part of checkbox -- that a user would opt-in to collecting additional data? 01:32:57 <roadmr> anyway, those are my ideas on this 01:33:32 <pleia2> roadmr: does it work? I guess I haven't run it in a bit :) 01:33:35 <cprofitt> I appreciate the feedback and information roadmr 01:33:35 <roadmr> cprofitt: sure, with the current checkbox architecture it feels a bit complex to do, but we can always change that, we should bend the tool to the needs, not the other way around 01:33:51 <roadmr> pleia2: as long as you have a microphone and speakers, it works and it's very reliable 01:33:59 <pleia2> nice 01:34:15 <cprofitt> roadmr: +1 we want to make it easy for people to collect data for this type of use -- improving user experience 01:34:58 <cprofitt> roadmr: do you have any insight as to why results are not getting included in to Ubuntu Friendly or is that outside of your scope? 01:35:42 <roadmr> cprofitt: I have a pretty good idea why, yes 01:35:52 <cprofitt> could you go a bit in to that for us? 01:36:13 <roadmr> cprofitt: two components are involved in this, the first is checkbox, which runs the tests, assembles a report and pushes that to launchpad 01:36:45 <roadmr> the second is the UF website/application, which pulls the reports (submissions) from launchpad and adds the results to the friendly database (what you see in the website) - does the scoring, aggregation and such 01:37:14 <roadmr> for a submission to appear at all, it needs to contain (and pass) all tests identified as "core" 01:37:30 <roadmr> the basics for a computer to be usable, if any core tests are failed the submission will not appear in UF 01:37:52 <roadmr> if it passes the "core" tests, it gets 3 stars, and any additional tests it passes increase its rating, up to 5 stars 01:38:06 <roadmr> a 5-star rating means that all the components we test work perfectly 01:38:14 <roadmr> so why would a submission not appear? 01:38:39 <roadmr> some of the core tests are automated, and checkbox is not very good at 1) telling you one of them failed, and 2) letting you rerun/amend the results 01:39:19 <roadmr> checkbox itself has no idea of core tests, so it can't tell you "hey, your card reader didn't work and that means you won't make it into UF - please rerun this before submitting" 01:39:42 <cprofitt> is there currenlty anyway for a person who has submitted the test to see their submissions on launchpad? 01:40:16 <roadmr> that's one reason, the second is that since the UF website needs some love, there may have been a bit of "drift" into the tests checkbox submits and performs, the UF site has not been updated as checkbox whitelists (sets of tests to run) change 01:40:23 <roadmr> those would be the two main reasons 01:40:36 <roadmr> cprofitt: there's a cryptic url that you an access to see your submissions, let me find it 01:40:53 <cprofitt> so, it sounds like the key is giving the user some feedback as to if their device failed a core test... and also allowing for them to resubmit 01:40:53 <roadmr> cprofitt: https://launchpad.net/~/+hwdb-submissions 01:41:37 <roadmr> cprofitt: yep, again, something that's a bit difficult to do with the current checkbox architecture which is very linear :( think of the difference between linear and non-linear video editing 01:41:47 <roadmr> in a sense, with checkbox we can only ffwd, rewind, and look at the results at the very end 01:41:58 * cprofitt nods 01:42:00 <roadmr> with no/little chance of going back to a specific test. 01:42:08 <cprofitt> exactly... 01:42:28 <cprofitt> it is also my understanding there is no ability to 'pause' a test and come back and pick up where you left off 01:42:49 <roadmr> there is, but it's very awkward heheh 01:42:51 <cprofitt> roadmr: is there currently any team that reviews the failed submissions? 01:43:07 <roadmr> what you do basically is murder checkbox, when you restart it does its best to pick up where it left off 01:43:30 <roadmr> it gives you three choices: rerun the last test, skip to the next one (in case the last test outright crashed the system), or start anew 01:43:42 * cprofitt nods 01:43:50 <roadmr> the message is quite terse and the choices are somewhat ambiguous, so most people are/will be confused by this 01:44:36 <roadmr> cprofitt: reviewing failed submissions, not really, no. It can be done but pretty much the only person who can and knows how to do it is jedimike, by looking at how the UF results importer processed a submission 01:44:48 <roadmr> he'd have more specifics on how to do it, but it's done on an ad-hoc basis 01:45:00 <cprofitt> roadmr: thanks 01:45:07 <roadmr> np :) 01:45:14 <cprofitt> I asked because I saw that as part of the launchpad team description 01:45:27 <cprofitt> does anyone else have any questions for roadmr? 01:46:19 <cprofitt> I would like to set an initial priority for the team moving forward 01:46:33 <cprofitt> I think that we have two high value items 01:47:12 <cprofitt> 1. Improving the UF website process so that there is some feedback to the person who submitted the system as to the fact the results were processed but failed 01:47:35 <cprofitt> 2. Documenting the current process - so others can understand it as we start to look at improving it 01:47:42 <cprofitt> are there any other items people see? 01:48:47 <cprofitt> roadmr: would you be willing to procude a flow chart of what you described that we could them use on the wiki? 01:48:57 <cprofitt> just a rough outline of how the process currently flows? 01:49:08 <roadmr> cprofitt: sure 01:49:22 <roadmr> cprofitt: there's some documentation (an outline plus more detailed links) here: https://friendly.ubuntu.com/what-is-ubuntu-friendly/ 01:49:23 <cprofitt> #action roadmr will produce a flow chart of the current process 01:49:23 * meetingology roadmr will produce a flow chart of the current process 01:49:53 <cprofitt> #vote agree that improving feedback on submissions is important in moving foward 01:49:53 <meetingology> Please vote on: agree that improving feedback on submissions is important in moving foward 01:49:53 <meetingology> Public votes can be registered by saying +1, +0 or -1 in channel, (private votes don't work yet, but when they do it will be by messaging the channel followed by +1/-1/+0 to me) 01:49:57 <cprofitt> +1 01:49:57 <meetingology> +1 received from cprofitt 01:50:12 <roadmr> +1 definitely heh 01:50:12 <meetingology> +1 definitely heh received from roadmr 01:50:33 <chilicuil> +1 01:50:33 <meetingology> +1 received from chilicuil 01:50:35 <SergioMeneses> +1 01:50:35 <meetingology> +1 received from SergioMeneses 01:50:44 <cprofitt> pleia2: ? 01:50:48 <cprofitt> phillw: ? 01:50:56 <phillw> +1 01:50:56 <meetingology> +1 received from phillw 01:51:05 <phillw> sorry wasnm't sure if i was to vote :) 01:51:23 <cprofitt> #endvote 01:51:23 <meetingology> Voting ended on: agree that improving feedback on submissions is important in moving foward 01:51:23 <meetingology> Votes for:5 Votes against:0 Abstentions:0 01:51:23 <meetingology> Motion carried 01:52:01 <cprofitt> I think for now we should move towards that; after we get that feedback improved we need to look to make some improvements to checkbox as roadmr mentioned 01:52:16 <cprofitt> any other topics, questions or thoughts? 01:52:37 <roadmr> o/ 01:52:42 <cprofitt> yes, roadmr 01:53:00 <SergioMeneses> when is our next meeting? 01:53:10 <roadmr> hehe, one thing that we *can* potentially do with current checkbox is make it submit directly to a (possibly updated) UF application, so feedback is quicker 01:53:33 <roadmr> this needs writing a "plugin" and an accompanying web application, so it would need some coordination 01:53:33 <cprofitt> +1 roadmr that should be explored 01:53:51 <roadmr> we're still on time to make that happen before Ubuntu Feature Freeze, but we'd have to start working soon 01:54:13 * cprofitt nods 01:54:24 <roadmr> ideally we sould ask jedimike if there's a way to submit directly to the UF application, because we may not have a lot of manpower to come up with a new one 01:54:31 <roadmr> sould/should 01:54:46 <cprofitt> SergioMeneses: I would like us to meet again in two weeks -- I think to be fair to some of the UTC 01:54:49 <roadmr> that may be a quick way to reduce the decoupling between submission and actual result processing 01:55:22 <cprofitt> folks we should have the meeting earlier; though that would require a different person to chair the meeting 01:55:36 <SergioMeneses> perfect 01:55:41 <cprofitt> roadmr: I know jedimike has spoken to me about that so I will try to touch bases with him about that 01:55:56 <cprofitt> are any of you in the UTC 0/+1 TZ? 01:57:01 <phillw> 'im on utc 01:57:01 <cprofitt> alright... I will send a message to the list and see if I can get a person in that TZ to chair a meeting. 01:57:02 <roadmr> cprofitt: I'm EST (UTC-5) 01:57:19 <cprofitt> phillw: would you be willing to chair a meeting on the 19th? 01:57:27 <cprofitt> roadmr: I am on your TZ as well 01:57:33 <phillw> I can if no one steps forward :) 01:57:49 <SergioMeneses> Im UTC-5 01:57:55 <cprofitt> phillw: ok, I will send the request to the list and you will be the fall back 01:58:36 <cprofitt> #action next meeting February 19th UTC 17:00 - cprofitt will send a message to the list to get a chair 01:58:36 * meetingology next meeting February 19th UTC 17:00 - cprofitt will send a message to the list to get a chair 01:58:36 <phillw> what time on 19th are you thinking of? 01:59:05 <phillw> yeah, 1700 utc would be okay if no one steps forward. 01:59:09 <SergioMeneses> 17 utc 01:59:12 * cprofitt nods 01:59:20 <cprofitt> yes, SergioMeneses so noon our time 01:59:52 <SergioMeneses> cprofitt, it's not problem for me 01:59:58 <cprofitt> thanks SergioMeneses phillw pleia2 chilicuil roadmr for attending. I think this has been very productive for an initial start 02:00:04 <cprofitt> #endmeeting