cam(4) timeouts in bhyve/kyua runs up on Jenkins?

[Neel Natu]

Hi NGie,

On Mon, May 25, 2015 at 1:40 PM, Garrett Cooper <yaneurabeya at gmail.com> wrote:
> On Apr 28, 2015, at 0:54, Alexander Motin <mav at FreeBSD.org> wrote:
>
>> Hi.
>>
>> On 27.04.2015 21:17, Garrett Cooper wrote:
>>> On Apr 27, 2015, at 11:16, Garrett Cooper <yaneurabeya at gmail.com>
>>> wrote:
>>>
>>>> I was looking at the console log for the latest kyua run and I’ve
>>>> noticed that it’s timing out a bit more [1] than it was
>>>> previously [2]. I’ve seen some of your commits recently to cam(4)
>>>> dealing with bhyve — has there been a performance regression
>>>> there? Thanks! -NGie
>>>>
>>>> 1.
>>>> https://jenkins.freebsd.org/job/FreeBSD_HEAD-tests2/940/console
>>>> 2.
>>>> https://jenkins.freebsd.org/job/FreeBSD_HEAD-tests2/983/console
>>>
>>> (Sorry for not being more explicit for the archives) These are the
>>> timeouts I’m referring to:
>>>
>>> ahcich0: is 00000000 cs 00000000 ss 1f000000 rs 1f000000 tfd 50
>>> serr 00000000 cmd 1000dc17 (ada0:ahcich0:0:0:0):
>>> WRITE_FPDMA_QUEUED. ACB: 61 08 a8 54 1e 40 00 00 00 00 00 00
>>> (ada0:ahcich0:0:0:0): CAM status: Command timeout
>>> (ada0:ahcich0:0:0:0): Retrying command

Can you try to reproduce with a virtio-blk instead of ahci-hd?

It will help narrow down the problem because both ahci-hd and
virtio-blk emulations share a common backend to read/write to the
disk.

best
Neel

>>
>> Last time I was more working on bhyve host disk emulation, rather then
>> on cam(4) running on guest. Considering that, what guest and what host
>> versions are you running? Is there any other load on host except this VM
>> that could cause I/O delays high enough to trigger timeouts? What are
>> you using to back the virtual disk (file, zvol, ...)?
>
> I have no idea what the Jenkins slaves are running in terms of configuration/version/etc. You’ll have to ask jenkins-admin@…
> Thanks!
> -NGie