From owner-zfs-devel@FreeBSD.ORG Sat Jun 19 23:47:47 2010 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 00EF7106564A for ; Sat, 19 Jun 2010 23:47:47 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (chello089077043238.chello.pl [89.77.43.238]) by mx1.freebsd.org (Postfix) with ESMTP id 4AEE48FC22 for ; Sat, 19 Jun 2010 23:47:46 +0000 (UTC) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id 599AB45FC0; Sun, 20 Jun 2010 01:21:20 +0200 (CEST) Received: from localhost (chello089077043238.chello.pl [89.77.43.238]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id 5120D45F83 for ; Sun, 20 Jun 2010 01:21:15 +0200 (CEST) Date: Sun, 20 Jun 2010 01:21:01 +0200 From: Pawel Jakub Dawidek To: zfs-devel@FreeBSD.org Message-ID: <20100619232101.GI6850@garage.freebsd.pl> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.4.2.3i X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 9.0-CURRENT amd64 X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-0.6 required=4.5 tests=BAYES_00,RCVD_IN_SORBS_DUL autolearn=no version=3.0.4 Cc: Subject: abc X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 19 Jun 2010 23:47:47 -0000 abc -- Pawel Jakub Dawidek http://www.wheelsystems.com pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! From owner-zfs-devel@FreeBSD.ORG Sat Jun 19 23:47:47 2010 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0966A106566B for ; Sat, 19 Jun 2010 23:47:47 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (chello089077043238.chello.pl [89.77.43.238]) by mx1.freebsd.org (Postfix) with ESMTP id 4E4A88FC23 for ; Sat, 19 Jun 2010 23:47:46 +0000 (UTC) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id 20B6845E86; Sun, 20 Jun 2010 01:17:51 +0200 (CEST) Received: from localhost (chello089077043238.chello.pl [89.77.43.238]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id 0D34945DF4 for ; Sun, 20 Jun 2010 01:17:45 +0200 (CEST) Date: Sun, 20 Jun 2010 01:17:31 +0200 From: Pawel Jakub Dawidek To: zfs-devel@FreeBSD.org Message-ID: <20100619231731.GH6850@garage.freebsd.pl> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="17/8oYur5Y32USnW" Content-Disposition: inline User-Agent: Mutt/1.4.2.3i X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 9.0-CURRENT amd64 X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-0.6 required=4.5 tests=BAYES_00,RCVD_IN_SORBS_DUL autolearn=no version=3.0.4 Cc: Subject: Test. X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 19 Jun 2010 23:47:47 -0000 --17/8oYur5Y32USnW Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Test. --=20 Pawel Jakub Dawidek http://www.wheelsystems.com pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --17/8oYur5Y32USnW Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (FreeBSD) iEYEARECAAYFAkwdUAsACgkQForvXbEpPzR3XwCgp3pIBvSM1iiC78tZ/OO2srG+ RjUAoN7MiaqchFSakl5XSuh9xvvgLuPe =VGr/ -----END PGP SIGNATURE----- --17/8oYur5Y32USnW-- From owner-zfs-devel@FreeBSD.ORG Sat Jun 19 23:55:39 2010 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 768D91065670 for ; Sat, 19 Jun 2010 23:55:39 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (chello089077043238.chello.pl [89.77.43.238]) by mx1.freebsd.org (Postfix) with ESMTP id BB7E08FC1C for ; Sat, 19 Jun 2010 23:55:38 +0000 (UTC) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id BEC4A45CDC; Sun, 20 Jun 2010 01:55:36 +0200 (CEST) Received: from localhost (chello089077043238.chello.pl [89.77.43.238]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id 26E22456B1 for ; Sun, 20 Jun 2010 01:55:31 +0200 (CEST) Date: Sun, 20 Jun 2010 01:55:17 +0200 From: Pawel Jakub Dawidek To: zfs-devel@FreeBSD.org Message-ID: <20100619235517.GJ6850@garage.freebsd.pl> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="gTY1JhLGodeuSBqf" Content-Disposition: inline User-Agent: Mutt/1.4.2.3i X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 9.0-CURRENT amd64 X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-0.6 required=4.5 tests=BAYES_00,RCVD_IN_SORBS_DUL autolearn=no version=3.0.4 Cc: Subject: Welcome! X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 19 Jun 2010 23:55:39 -0000 --gTY1JhLGodeuSBqf Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Hello:) Ok, so we now have a mailing list. It is not private, archives can be read through mailman's web interface, but subscriptions are moderated by me for now. Currently we have the following guys here: Josh Carter / Spectra Logic Randy Chou / Panzura Pawel Jakub Dawidek / FreeBSD Samir Desai / Spectra Logic Justin Gibbs / Spectra Logic Steve Jung / Panzura Xin Li / iXsystems, FreeBSD Michael Lin / Panzura Martin Matuska / FreeBSD Ravi Mulam / Panzura Matt Olander / iXsystems Steve Pate / Spectra Logic Tushar Tambay / Spectra Logic John Taylor / Panzura Ganesh Varadarajan / Spectra Logic I hope I got all the names and companies right. If you would like to add someone, let me know privately. To speed things up I went ahead and just subscribed all of you, instead of sending invitiations. If you don't want to be here, please forgive me and let me know (also privately), I'll remove you from the list ASAP. The idea of this list is to coordinate efforts of companies and individuals directly involved in FreeBSD/ZFS development. I must say that after spending few years on FreeBSD/ZFS alone and failing to get more people involved, I'm very happy to see so many names here:) --=20 Pawel Jakub Dawidek http://www.wheelsystems.com pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --gTY1JhLGodeuSBqf Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (FreeBSD) iEYEARECAAYFAkwdWOQACgkQForvXbEpPzS5ZQCfYSvpuswzh7IXolplrNCyVfXG O4AAnAqcQWmyqUzfdRbYgc446IosLtI2 =m4z5 -----END PGP SIGNATURE----- --gTY1JhLGodeuSBqf-- From owner-zfs-devel@FreeBSD.ORG Sun Jun 20 09:19:03 2010 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5099E1065672 for ; Sun, 20 Jun 2010 09:19:03 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from mail.vx.sk (core.vx.sk [188.40.32.143]) by mx1.freebsd.org (Postfix) with ESMTP id 887298FC16 for ; Sun, 20 Jun 2010 09:19:02 +0000 (UTC) Received: from core.vx.sk (localhost [127.0.0.1]) by mail.vx.sk (Postfix) with ESMTP id A3C69CC69C for ; Sun, 20 Jun 2010 11:01:13 +0200 (CEST) X-Virus-Scanned: amavisd-new at mail.vx.sk Received: from mail.vx.sk ([127.0.0.1]) by core.vx.sk (mail.vx.sk [127.0.0.1]) (amavisd-new, port 10024) with LMTP id VdAbqzF1D51R for ; Sun, 20 Jun 2010 11:01:11 +0200 (CEST) Received: from [10.9.8.1] (chello085216227216.chello.sk [85.216.227.216]) by mail.vx.sk (Postfix) with ESMTPSA id 53191CC695 for ; Sun, 20 Jun 2010 11:01:11 +0200 (CEST) Message-ID: <4C1DD8D9.5090404@FreeBSD.org> Date: Sun, 20 Jun 2010 11:01:13 +0200 From: Martin Matuska User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; sk; rv:1.8.1.23) Gecko/20090812 Lightning/0.9 Thunderbird/2.0.0.23 Mnenhy/0.7.5.0 MIME-Version: 1.0 To: zfs-devel@FreeBSD.org X-Enigmail-Version: 1.0.1 Content-Type: text/plain; charset=windows-1250 Content-Transfer-Encoding: 7bit Cc: Subject: ZFS development strategy X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 20 Jun 2010 09:19:03 -0000 Hi Pawel, Xin and all members of this list. First of all, I would like to thank Pawel for all the work on porting ZFS to FreeBSD and for creating this list. I hope this list will help us to exchange new ideas, help each other with existing issues or discuss how things should be implemented. As the very first topic, I would like to discuss our ZFS development strategy. To coordinate our work as best as possible, it would be very nice to have a common set of goals and targets or at least some basic strategy guidelines we should follow. I understand everybody has some kind of "own" agenda, but we should e.g. avoid reimplementing things 2 times (and sometimes both in different ways). At current state, we have quite well-tested post-v14 in head, stable/8, post-v13 in stable/7, and latest work (post-v26) in pjd's perforce tree. My main effort until now has been finding and importing bugfixes and/or important improvements (e.g. parallel traversal, l2arc) from OpenSolaris that do or may affect FreeBSD users into head and stable. If I continue this effort I could easily land in v15 as a pre-step for v26 (v15 introduces group and user quotas and splits the python code as well). What I also plan to do is creating a python dummy port for the python code part. So I would maily like to ask the following questions about ourt ZFS: 1. Are we interested in setting up some goals / guidelines for our ZFS in FreeBSD to help coordinating our work? 2. Is there a (very rough) estimate when v26 might find its way into wider testing and head? 3. If the answer for 2. lies in far future, does it make sense to go v15 as a step inbetween (including or not-including the python dummy port)? Thank you very much, mm From owner-zfs-devel@FreeBSD.ORG Tue Jun 22 17:00:43 2010 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4A05E1065670; Tue, 22 Jun 2010 17:00:43 +0000 (UTC) (envelope-from dewcopper@gmail.com) Received: from mail-yw0-f181.google.com (mail-yw0-f181.google.com [209.85.211.181]) by mx1.freebsd.org (Postfix) with ESMTP id E621A8FC13; Tue, 22 Jun 2010 17:00:42 +0000 (UTC) Received: by ywh11 with SMTP id 11so3657076ywh.7 for ; Tue, 22 Jun 2010 10:00:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:sender:received :in-reply-to:references:date:x-google-sender-auth:message-id:subject :from:to:content-type; bh=fz/GRygvqQbJzOtg5eMUJV3qT9/ZWfJTgDOes4p7AOk=; b=rSasFyYsVsJbW15owLpHg7f2qWPdfY2v6Km/QE5EJBapAA/2Q5UxQhGaOBceYJtoBl h9T03OzGWNI5821bLqrhJV02V981mLM4M3an0Mp/pvjQ+hly9EGIlySgFtzrCU66I2V2 WI7taY3MwMpwozgJ7fULf5Sp9z5Fe7SZfE9io= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:content-type; b=M6gItA8LYA0l6StcoMiaUI2hOfbOYnRfyxl1Giz5UNrIGpzIl7p+ydtgvmY7ptltGr kvnhy1fuZ9LoDyRA3CMwpEUrQXH2y3OhqWoxGFJVxY1zTU/3zjhEQsUKum80tdKnV7Rp 6noMczzdR7HgOfL0hOKH9GuC0fCf++1JjxpLc= MIME-Version: 1.0 Received: by 10.150.172.1 with SMTP id u1mr6082693ybe.240.1277224300482; Tue, 22 Jun 2010 09:31:40 -0700 (PDT) Sender: dewcopper@gmail.com Received: by 10.150.191.5 with HTTP; Tue, 22 Jun 2010 09:31:40 -0700 (PDT) In-Reply-To: <4C1DD8D9.5090404@FreeBSD.org> References: <4C1DD8D9.5090404@FreeBSD.org> Date: Tue, 22 Jun 2010 09:31:40 -0700 X-Google-Sender-Auth: LAoe_36N7Xnliedqa1qsLZq6YUM Message-ID: From: Tushar Tambay To: Martin Matuska , zfs-devel@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 Cc: Subject: Re: ZFS development strategy X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 22 Jun 2010 17:00:43 -0000 Hi all, I haven't got everyones skype ids yet and I do not have a conf call # that has a toll free dial-in from poland and india. So first question - does anyone have a conf call number we can use ? If not, just as a backup, can you folks send out you skype ids please ? On 6/20/10, Martin Matuska wrote: > Hi Pawel, Xin and all members of this list. > > First of all, I would like to thank Pawel for all the work on porting > ZFS to FreeBSD and for creating this list. > I hope this list will help us to exchange new ideas, help each other > with existing issues or discuss how things should be implemented. > > As the very first topic, I would like to discuss our ZFS development > strategy. > > To coordinate our work as best as possible, it would be very nice to > have a common set of goals and targets or at least some basic strategy > guidelines we should follow. I understand everybody has some kind of > "own" agenda, but we should e.g. avoid reimplementing things 2 times > (and sometimes both in different ways). > > At current state, we have quite well-tested post-v14 in head, stable/8, > post-v13 in stable/7, and latest work (post-v26) in pjd's perforce tree. > > My main effort until now has been finding and importing bugfixes and/or > important improvements (e.g. parallel traversal, l2arc) from OpenSolaris > that do or may affect FreeBSD users into head and stable. If I continue > this effort I could easily land in v15 as a pre-step for v26 (v15 > introduces group and user quotas and splits the python code as well). > What I also plan to do is creating a python dummy port for the python > code part. > > So I would maily like to ask the following questions about ourt ZFS: > 1. Are we interested in setting up some goals / guidelines for our ZFS > in FreeBSD to help coordinating our work? > 2. Is there a (very rough) estimate when v26 might find its way into > wider testing and head? > 3. If the answer for 2. lies in far future, does it make sense to go v15 > as a step inbetween (including or not-including the python dummy port)? > > Thank you very much, > mm > _______________________________________________ > zfs-devel@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/zfs-devel > To unsubscribe, send any mail to "zfs-devel-unsubscribe@freebsd.org" > -- Sent from my mobile device tyt phone: 408 203 9736 (c) - From owner-zfs-devel@FreeBSD.ORG Tue Jun 22 20:23:23 2010 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5D07D1065670 for ; Tue, 22 Jun 2010 20:23:23 +0000 (UTC) (envelope-from dewcopper@gmail.com) Received: from mail-yw0-f181.google.com (mail-yw0-f181.google.com [209.85.211.181]) by mx1.freebsd.org (Postfix) with ESMTP id 174448FC1C for ; Tue, 22 Jun 2010 20:23:22 +0000 (UTC) Received: by ywh11 with SMTP id 11so3777536ywh.7 for ; Tue, 22 Jun 2010 13:23:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:sender:received:date :x-google-sender-auth:message-id:subject:from:to:content-type; bh=1B3d5kKzLyDlhOf3MZPxzjeHYHLB/hL06Ljmuk1PQhc=; b=jIDOIMoBb2kpJCiSnF6yMFUfe7Pc9wLpZO0lhGSTS29UkUxV9McaGcKNBnt3sqxb+l VYYY6yi1kIguUqzmwDfWv0Je/EoZKAb+DjPRoUdqW++z9N7N0i9mHfWWrTUqMcyvFp9Z mlScJ7/SdafOTJ0tS3qRC6GbLPKDHnJTFh/k4= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:date:x-google-sender-auth:message-id:subject :from:to:content-type; b=jfTv3PxfHvAF1sg3sNyA/FT6FZ+5kWIm7fIE7wmNC1C6g+47TXsKeLZKkCx/C0YPnh xF+y9Zz5TaXLTiV9taMbRSH6tyO7BBUh9cjhYaaIFehvnNOjPbN2geqLQytNuQfDNjLb 3OQpxz61GNCdgAnoyvO7nBPtZuMmgH6RuVyqs= MIME-Version: 1.0 Received: by 10.150.208.19 with SMTP id f19mr6761359ybg.208.1277238202175; Tue, 22 Jun 2010 13:23:22 -0700 (PDT) Sender: dewcopper@gmail.com Received: by 10.150.191.5 with HTTP; Tue, 22 Jun 2010 13:23:22 -0700 (PDT) Date: Tue, 22 Jun 2010 13:23:22 -0700 X-Google-Sender-Auth: JeZXc8J3q-h9eT8G05Kl1JFeGw4 Message-ID: From: Tushar Tambay To: zfs-devel@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: zfs/bsd conf call minutes X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 22 Jun 2010 20:23:23 -0000 date : 7/22/10 attendees : justin, pawel, xin, ganesh, samir, steve, tushar, john development branch & checkin logistics - /project/zfs/stable/8 will be our primary collaborative branch - everyone should have checkin privileges for this already - justin will do a weekly merge of changes/fixes from the bsd/head and bsd/stable branches to /project/zfs/stable/8 and /project/zfs/current - pawel will request martin to do a weekly merge of changes/fixes from /user/pjd/zfs branch to the /project/zfs/stable/8 and /project/zfs/current - ganesh will send out a description of zfs test suite layout to zfs-devel@freebsd.org alias and run the discussion to decide where the test suite will be checked in - samir will establish a checkin criteria based on zfs test suite - we will delay discussing the details of translation layer required to support zfs v24 on the official 8.x BSD release until we get a better feel for stability/performance of zfs zfs / python - on opensolaris, zfs utilities have been re-written in python (earlier in C) - python is not available in base bsd disribution (only in ports) - we will keep zfs utilities in python and include zfs in base bsd distribution but users will need to configure their systems with the python port to get all the zfs functionality zfs/dedupe - suspected dedupe performance issue needs to be quantified with further testing - dedupe performance primarily tied to how much of the index can be cached. - zfs has the facility to introduce a second level cache using (say) SSD storage but this needs testing/verification. - pawel suspects a priority/scheduling issue with dedupe - under heavy dedupe load, system becomes unresponsive/deadlocked. - no dedupe tests exist in recently ported test suite zfs/memory issue - KVA exhaustion is a problem but limited to 32bit platforms - besides KVA, there are other memory usage issues (though not very severe). pawel's recent fix to reclaim memory used by name-cache should address some of these - BSD memory tuning is primarily for ufs. so assumptions about inode and other sizes need to be revisited for zfs. zfs / io - (pawel) currently zfs commits changes in transaction groups every 10 (earlier 30) seconds and flushes whenever ARC is full. - (justin) zfs flushing to disk is v. bursty even when applied i/o load is very uniform. - zfs needs better flush behind algorithms for (application) i/o - justin will fix broken disk elevator code. he has seen data corruption / unmountable zfs file systems that are caused by this. zfs / zvols - bioflush has no sync/async flag. a big problem for zfs/zvol - currently all i/o is treated as async and callers need to call another interface to force flush - ... pretty sure i've got some details wrong on this. justing - can you please correct this ? next conf call - 7/20/2010 cheers, -- tyt phone: 408 203 9736 (c) - From owner-zfs-devel@FreeBSD.ORG Tue Jun 22 21:47:21 2010 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 567F91065690 for ; Tue, 22 Jun 2010 21:47:21 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from mail.vx.sk (core.vx.sk [188.40.32.143]) by mx1.freebsd.org (Postfix) with ESMTP id A83DE8FC2F for ; Tue, 22 Jun 2010 21:47:20 +0000 (UTC) Received: from core.vx.sk (localhost [127.0.0.1]) by mail.vx.sk (Postfix) with ESMTP id 6F8A3D4415 for ; Tue, 22 Jun 2010 23:47:19 +0200 (CEST) X-Virus-Scanned: amavisd-new at mail.vx.sk Received: from mail.vx.sk ([127.0.0.1]) by core.vx.sk (mail.vx.sk [127.0.0.1]) (amavisd-new, port 10024) with LMTP id SO7xvB23ru+v for ; Tue, 22 Jun 2010 23:47:17 +0200 (CEST) Received: from [10.9.8.1] (chello085216227216.chello.sk [85.216.227.216]) by mail.vx.sk (Postfix) with ESMTPSA id 232E0D4408 for ; Tue, 22 Jun 2010 23:47:17 +0200 (CEST) Message-ID: <4C212F68.5060701@FreeBSD.org> Date: Tue, 22 Jun 2010 23:47:20 +0200 From: Martin Matuska User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; sk; rv:1.8.1.23) Gecko/20090812 Lightning/0.9 Thunderbird/2.0.0.23 Mnenhy/0.7.5.0 MIME-Version: 1.0 To: zfs-devel@FreeBSD.org X-Enigmail-Version: 1.0.1 Content-Type: text/plain; charset=windows-1250 Content-Transfer-Encoding: 7bit Cc: Subject: ZFS v15 patch X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 22 Jun 2010 21:47:21 -0000 Dear ZFS developers, As the user/group quotas feature is too much attractive for my needs, I couldn't resist and have created (and debugged + tested) a ZFS v15 patch for head (applies cleanly against stable/8 as well). It is a backport of several onnv-revisions, always consulting pjd's p4 tree and includes four post-9396 related user/groupquota bugfixes. The bootcode (zfsimpl.h) is properly updated to support v15 as well, the python part is modified (paths, smb support, ioctls). The patch, list of imported revisions and a preliminary but working version of the pyzfs port can be downloaded from this link: http://people.freebsd.org/~mm/patches/zfs/v15/ You can download 8.1-RC1-amd64 with zfsv15 (incl. boot support) mfsBSD ISOs from here: http://mfsbsd.vx.sk/iso/8.1rc1-zfsv15.iso (without symbols, 98.7M) http://mfsbsd.vx.sk/iso/8.1rc1-zfsv15-debug.iso (with symbols, 187.5M) Login/password into the ISOs: root/mfsroot run "zfsinstall" for a zfs-on-root installation (don't forget -V 15 if trying to install) The amd64 ISO's may be tested in virtualbox or real-world systems as well. Regarding new ioctl's: I have added all new ioctl's in the patch as new ones (numbers 49 to 52). This makes the old kernel module work with new library / utilities (tested) or any other binary that references these ioctl's. Regarding the python port: The current port installs the library into PYTHON_SITELIBDIR and pyzfs.py into /usr/lib/zfs/ Here is an idea how the port could work in the future: a) FreeBSD installs /usr/lib/zfs/pyzfs.py (or whatever location desired) by default with #!/usr/bin/env python b) the pyzfs library is installed from ports (dummy or distfile-bound port) Version 15 introduces user and group quota accounting and could be a very good step in preparation towards v26. The code runs impressingly stable (I didn't manage to reproduce any panics or deadlocks yet). It would be very great if this could get widely tested. I guess we could go with this code in direction stable/8 (head first), because it is backwards compatible (zfs and zpool work with older kernel module - only new features don't work, the same for the boot changes, boot from older pools is supported). Any ideas, suggetions, etc. are welcome! Cheers, mm From owner-zfs-devel@FreeBSD.ORG Wed Jun 30 05:18:29 2010 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C582A106566C; Wed, 30 Jun 2010 05:18:29 +0000 (UTC) (envelope-from dewcopper@gmail.com) Received: from mail-gw0-f54.google.com (mail-gw0-f54.google.com [74.125.83.54]) by mx1.freebsd.org (Postfix) with ESMTP id 6AEA78FC18; Wed, 30 Jun 2010 05:18:29 +0000 (UTC) Received: by gwj16 with SMTP id 16so263313gwj.13 for ; Tue, 29 Jun 2010 22:18:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:sender:received:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; bh=QjJJasVF/rLKcSNDu0oTrF5GPYGB9Kxs+TWnMp0Z6Z0=; b=Yz4illwPRbUDyujN7drSFHZQGCXhpVzDui2WKfS+m2DRcA9V6+Lj31+hWWqdnjK1cs aLL1iC/bEgx2Qj0BXcKtm6c498cK8zlAM41PNn7tHE96hcvyXUol69qC3sZw+FoN+mQS DCjziSo5XRI8TJc4RTTTWd8J7OgkL5x1zsAm4= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:date:x-google-sender-auth:message-id:subject :from:to:cc:content-type; b=v/Fcgt00kDAkFHzRXOjjAB7BGHkb0lwJn7Zg+e/1EMofT/j5ZldYPo3piQ8V5jITkM c/610JhQnhnF1dN1q/W9x1ZH2Y/8atHuJXCIXvMl9kltviPfWTz59rgUoy61K4OC2ZAu bsx7fvXRJXiUn75R1rKETZR1YtlxoyeyG1aSA= MIME-Version: 1.0 Received: by 10.90.107.5 with SMTP id f5mr6880432agc.18.1277875100600; Tue, 29 Jun 2010 22:18:20 -0700 (PDT) Sender: dewcopper@gmail.com Received: by 10.151.101.5 with HTTP; Tue, 29 Jun 2010 22:18:20 -0700 (PDT) Date: Tue, 29 Jun 2010 22:18:20 -0700 X-Google-Sender-Auth: EfsTd1tIPy91cLayGS_dahuNTK0 Message-ID: From: Tushar Tambay To: pawel Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: zfs-devel@freebsd.org Subject: latest zfs bits on stable 8 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 30 Jun 2010 05:18:29 -0000 hi pawel, we are ready to check in some changes to the common zfs tree, but before that happens, the zfs bits in /project/zfs/stable/8 will need to be updated to the latest from your tree. my understanding is that you were going to request martin to do this. is there any further update on this ? thanks in advance, -- tyt phone: 408 203 9736 (c) - From owner-zfs-devel@FreeBSD.ORG Wed Jun 30 07:05:04 2010 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2F0B3106566B for ; Wed, 30 Jun 2010 07:05:04 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (chello089077043238.chello.pl [89.77.43.238]) by mx1.freebsd.org (Postfix) with ESMTP id 759C28FC13 for ; Wed, 30 Jun 2010 07:05:03 +0000 (UTC) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id 24AEC45E6F; Wed, 30 Jun 2010 09:05:01 +0200 (CEST) Received: from localhost (chello089077043238.chello.pl [89.77.43.238]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id E8E1745E48; Wed, 30 Jun 2010 09:04:55 +0200 (CEST) Date: Wed, 30 Jun 2010 09:04:37 +0200 From: Pawel Jakub Dawidek To: Tushar Tambay Message-ID: <20100630070437.GD1816@garage.freebsd.pl> References: Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="YToU2i3Vx8H2dn7O" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.3i X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 9.0-CURRENT amd64 X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-0.6 required=4.5 tests=BAYES_00,RCVD_IN_SORBS_DUL autolearn=no version=3.0.4 Cc: zfs-devel@freebsd.org Subject: Re: latest zfs bits on stable 8 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 30 Jun 2010 07:05:04 -0000 --YToU2i3Vx8H2dn7O Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Tue, Jun 29, 2010 at 10:18:20PM -0700, Tushar Tambay wrote: > hi pawel, >=20 > we are ready to check in some changes to the common zfs tree, but before > that happens, the zfs bits in /project/zfs/stable/8 will need to be updat= ed > to the latest from your tree. >=20 > my understanding is that you were going to request martin to do this. is > there any further update on this ? Well, I'm meeting Martin in two days at meetBSD and I'd like to explain the whole thing to him there. My idea is to sync to more or less working changeset (last changeset of user/pjd/zfs/... doesn't even compile atm) and then start to sync only selected changes (bug fixes mostly). What I'm doing in user/pjd/zfs/... is to sync all the changes from OpenSolaris and we don't want that in projects/zfs/..., because changes from OpenSolaris are comming faster than we can test them and fix them, so we have to decide where exactly is the point we want to focus on and don't bring any new features from OpenSolaris to projects/zfs/... - from this point we should only sync bug fixes from usr/pjd/zfs/... I'll follow up after meetBSD. --=20 Pawel Jakub Dawidek http://www.wheelsystems.com pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --YToU2i3Vx8H2dn7O Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (FreeBSD) iEYEARECAAYFAkwq7IUACgkQForvXbEpPzTDAACdEtY5lOlxXrTLHcuHd5cwHCEd YVgAn3yTGhYKL1nL5uxzyAnDP62juetH =9G27 -----END PGP SIGNATURE----- --YToU2i3Vx8H2dn7O-- From owner-zfs-devel@FreeBSD.ORG Wed Jun 30 07:24:23 2010 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D8E86106564A; Wed, 30 Jun 2010 07:24:23 +0000 (UTC) (envelope-from dewcopper@gmail.com) Received: from mail-gx0-f182.google.com (mail-gx0-f182.google.com [209.85.161.182]) by mx1.freebsd.org (Postfix) with ESMTP id D4DC38FC0C; Wed, 30 Jun 2010 07:24:21 +0000 (UTC) Received: by gxk7 with SMTP id 7so292605gxk.13 for ; Wed, 30 Jun 2010 00:24:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:sender:received :in-reply-to:references:date:x-google-sender-auth:message-id:subject :from:to:cc:content-type; bh=fiUzXI3zCIMUbB/a3SReyQAYb+jpRxLIGLbTp3nZBmM=; b=IvK5JOqfyCgCKkdDxQFNK+Bvn1f9bfPw3Dmg2yJGJiY1mSoQ2KZcosbYSUgWmVfYNu YfTfzju9OvWkNDyeLAFS5fwfx1A7ENFQAdfoSeWi0KMMDu1Ec9kF10qwiAXZ8gh+RU/6 reQ+6XUZjvLcr4tiMBnSehsIlKk1hHLJYdXCc= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; b=ulrNpVVPxiMnP4Fvph3xfmOVrn48T+ORkjM3I2q2XSwyZKUDPQK1NsuoKEFTOBPsRk 1451CJuJGzFok19hnMsqi5j7H0hHcw2Vd7uwU2tb/2CTicnwG4j78AnzJhzRVNOg+Dis e/FJnpDVU7MD9UhNFepeFRfZbrRniPb0NHf3o= MIME-Version: 1.0 Received: by 10.90.63.18 with SMTP id l18mr5695659aga.154.1277882643890; Wed, 30 Jun 2010 00:24:03 -0700 (PDT) Sender: dewcopper@gmail.com Received: by 10.151.101.5 with HTTP; Wed, 30 Jun 2010 00:24:03 -0700 (PDT) In-Reply-To: <20100630070437.GD1816@garage.freebsd.pl> References: <20100630070437.GD1816@garage.freebsd.pl> Date: Wed, 30 Jun 2010 00:24:03 -0700 X-Google-Sender-Auth: UzeTFJKRBFHG-AM2iXnTee5FU-c Message-ID: From: Tushar Tambay To: Pawel Jakub Dawidek Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: zfs-devel@freebsd.org Subject: Re: latest zfs bits on stable 8 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 30 Jun 2010 07:24:23 -0000 thanks pawel ! On Wed, Jun 30, 2010 at 12:04 AM, Pawel Jakub Dawidek wrote: > On Tue, Jun 29, 2010 at 10:18:20PM -0700, Tushar Tambay wrote: > > hi pawel, > > > > we are ready to check in some changes to the common zfs tree, but before > > that happens, the zfs bits in /project/zfs/stable/8 will need to be > updated > > to the latest from your tree. > > > > my understanding is that you were going to request martin to do this. is > > there any further update on this ? > > Well, I'm meeting Martin in two days at meetBSD and I'd like to explain > the whole thing to him there. > > My idea is to sync to more or less working changeset (last changeset of > user/pjd/zfs/... doesn't even compile atm) and then start to sync only > selected changes (bug fixes mostly). > > What I'm doing in user/pjd/zfs/... is to sync all the changes from > OpenSolaris and we don't want that in projects/zfs/..., because changes > from OpenSolaris are comming faster than we can test them and fix them, > so we have to decide where exactly is the point we want to focus on and > don't bring any new features from OpenSolaris to projects/zfs/... - from > this point we should only sync bug fixes from usr/pjd/zfs/... > > I'll follow up after meetBSD. > > -- > Pawel Jakub Dawidek http://www.wheelsystems.com > pjd@FreeBSD.org http://www.FreeBSD.org > FreeBSD committer Am I Evil? Yes, I Am! > -- tyt phone: 408 203 9736 (c) - From owner-zfs-devel@FreeBSD.ORG Thu Jul 1 08:07:06 2010 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 292BD106566C for ; Thu, 1 Jul 2010 08:07:06 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from mail.vx.sk (core.vx.sk [188.40.32.143]) by mx1.freebsd.org (Postfix) with ESMTP id 5E2448FC1B for ; Thu, 1 Jul 2010 08:07:05 +0000 (UTC) Received: from core.vx.sk (localhost [127.0.0.1]) by mail.vx.sk (Postfix) with ESMTP id 74451DAC56 for ; Thu, 1 Jul 2010 10:07:04 +0200 (CEST) X-Virus-Scanned: amavisd-new at mail.vx.sk Received: from mail.vx.sk ([127.0.0.1]) by core.vx.sk (mail.vx.sk [127.0.0.1]) (amavisd-new, port 10024) with LMTP id 0p22fpFjI33q for ; Thu, 1 Jul 2010 10:07:02 +0200 (CEST) Received: from [10.9.8.1] (188-167-78-139.dynamic.chello.sk [188.167.78.139]) by mail.vx.sk (Postfix) with ESMTPSA id 8343CDAC48 for ; Thu, 1 Jul 2010 10:07:02 +0200 (CEST) Message-ID: <4C2C4CA6.4050808@FreeBSD.org> Date: Thu, 01 Jul 2010 10:07:02 +0200 From: Martin Matuska User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; sk; rv:1.8.1.23) Gecko/20090812 Lightning/0.9 Thunderbird/2.0.0.23 Mnenhy/0.7.5.0 MIME-Version: 1.0 To: zfs-devel@FreeBSD.org X-Enigmail-Version: 1.0.1 Content-Type: multipart/mixed; boundary="------------070804070500090504050209" Cc: Subject: Bugfixes backported to Solaris 10 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 01 Jul 2010 08:07:06 -0000 This is a multi-part message in MIME format. --------------070804070500090504050209 Content-Type: text/plain; charset=windows-1250 Content-Transfer-Encoding: 7bit Hi all, attached is a list of bugfixes and corresponding onnv-revisions that have been backported to Solaris 10 until today (kernel patch 142901-09). Information if the corresponding bugfix is already in head/stable is included. --------------070804070500090504050209 Content-Type: text/plain; name="zfs_bugfixes_solaris_10.txt" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="zfs_bugfixes_solaris_10.txt" List of ZFS v15 bugfixes backported by Solaris 10 142901-09 --------- 9515:d3b739d9d043 (not imported, head-v16-extended-v3.patch) 6586537 async zio taskqs can block out userland commands 9790:e276ee006ff6 (not imported) 6747441 GRUB/vdev_get_bootpath, spa_get_rootconf, zpool_get_physpath should take care of spare vdev 6844158 assertion failed: vd->vdev_ops == &vdev_mirror_ops, file: ../../common/fs/zfs/zvol.c, line: 1054 10896:007ca67ca400 (not imported) 6793877 lockd and Apache can block ZFS force-unmounting on behalf of clients 10801:e0bf032e8673 (not imported, head-v16-extended-v3.patch) 6822816 assertion failed: zap_remove_int(ds_next_clones_obj) returns ENOENT 11546:42ea6be8961b (not imported) 6833999 3-way deadlock in dsl_dataset_hold_ref() and dsl_sync_task_group_sync() 10474:0e96dd3b905a (head: r208130, stable/8: r208288) 6847118 hang in ZFS during rollback 10800:469478b180d9 (not imported, head-v16-extended-v2.patch) 6880764 fsync on ZFS is broken if writes are greater than 32kb on a hard crash and no log attached 10810:b6b161a6ae4a (not imported, head-v16-extended-v3.patch) 6892298 buf->b_hdr->b_state != arc_anon, file: ../../common/fs/zfs/arc.c, line: 2849 142901-10 --------- 11321:506b7043a14c (head: r208131, stable/8: r208288) 6847615 deadlock between zfs_dirent_lock and zfs_rmdir 11454:6e69bacc1a5a (not imported, head-v16-extended-v3.patch) 6898245 suspended zpool should not cause rest of the zfs/zpool commands to hang 142901-12 --------- 12079:13822b941977 (not imported) 6939941 problem with moving files in ZFS 142901-13 --------- 10890:499786962772 (not imported, head-v16-extended-v3.patch) 6807339 spurious checksum errors when replacing a vdev 11249:6c30f7dfc97b (not imported, head-v16-extended.patch) 6906110 bad trap panic in zil_replay_log_record 6906946 ZFS replay isn't handling uid/gid correctly 142901-14 --------- 10839:cf83b553a2ab (head: r209101, stable/8: r209274) 6836714 arc_read_done may try to byteswap undefined data 10575:2a8816c5173b (not imported) 6882227 spa_async_remove() shouldn't do a full clear --------------070804070500090504050209-- From owner-zfs-devel@FreeBSD.ORG Tue Jul 6 21:22:50 2010 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D5D6D106566B for ; Tue, 6 Jul 2010 21:22:50 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (chello089077043238.chello.pl [89.77.43.238]) by mx1.freebsd.org (Postfix) with ESMTP id 2E2978FC1A for ; Tue, 6 Jul 2010 21:22:48 +0000 (UTC) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id C7EE145E8E; Tue, 6 Jul 2010 23:22:46 +0200 (CEST) Received: from localhost (chello089077043238.chello.pl [89.77.43.238]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id 980C845D8D for ; Tue, 6 Jul 2010 23:22:40 +0200 (CEST) Date: Tue, 6 Jul 2010 23:22:19 +0200 From: Pawel Jakub Dawidek To: zfs-devel@FreeBSD.org Message-ID: <20100706212219.GD1724@garage.freebsd.pl> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="mJm6k4Vb/yFcL9ZU" Content-Disposition: inline User-Agent: Mutt/1.4.2.3i X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 9.0-CURRENT amd64 X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-0.6 required=4.5 tests=BAYES_00,RCVD_IN_SORBS_DUL autolearn=no version=3.0.4 Cc: Subject: Some updates. X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Jul 2010 21:22:50 -0000 --mJm6k4Vb/yFcL9ZU Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Hi. During meetBSD in Poland few days ago we had some time to discuss few ZFS-related things. Martin Matuska agreed to work on merging bug fixes from user/pjd/zfs/... to projects/zfs/head/... and to projects/zfs/stable/8/... I'd suggest merging everything from user/pjd/zfs/... except for @179537, which doesn't compile yet to projects/zfs/.... Without this change it should more or less work. Martin is going to upgrade ZFS we have in HEAD to v15. This is also what Solaris is using. --=20 Pawel Jakub Dawidek http://www.wheelsystems.com pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --mJm6k4Vb/yFcL9ZU Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (FreeBSD) iEYEARECAAYFAkwznosACgkQForvXbEpPzQ+qgCg3j+XtM5MN1jWWmMe6tAFmwL+ TGgAnjKHKS12ybyg37OliT4a292yC3Lq =cfsc -----END PGP SIGNATURE----- --mJm6k4Vb/yFcL9ZU-- From owner-zfs-devel@FreeBSD.ORG Wed Jul 7 06:37:38 2010 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8B0F41065675 for ; Wed, 7 Jul 2010 06:37:37 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from mail.vx.sk (core.vx.sk [188.40.32.143]) by mx1.freebsd.org (Postfix) with ESMTP id BEB788FC18 for ; Wed, 7 Jul 2010 06:37:36 +0000 (UTC) Received: from core.vx.sk (localhost [127.0.0.1]) by mail.vx.sk (Postfix) with ESMTP id 688A0F0227 for ; Wed, 7 Jul 2010 08:37:35 +0200 (CEST) X-Virus-Scanned: amavisd-new at mail.vx.sk Received: from mail.vx.sk ([127.0.0.1]) by core.vx.sk (mail.vx.sk [127.0.0.1]) (amavisd-new, port 10024) with LMTP id zwzMfr22hAbJ for ; Wed, 7 Jul 2010 08:37:33 +0200 (CEST) Received: from [10.9.8.1] (188-167-78-139.dynamic.chello.sk [188.167.78.139]) by mail.vx.sk (Postfix) with ESMTPSA id 5909CF0220 for ; Wed, 7 Jul 2010 08:37:33 +0200 (CEST) Message-ID: <4C3420AD.8040709@FreeBSD.org> Date: Wed, 07 Jul 2010 08:37:33 +0200 From: Martin Matuska User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; sk; rv:1.8.1.23) Gecko/20090812 Lightning/0.9 Thunderbird/2.0.0.23 Mnenhy/0.7.5.0 MIME-Version: 1.0 To: zfs-devel@FreeBSD.org X-Enigmail-Version: 1.0.1 Content-Type: text/plain; charset=windows-1250 Content-Transfer-Encoding: 7bit Cc: Subject: Status update on v15 import X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 07 Jul 2010 06:37:38 -0000 Hi all, I have some update on the status of the v15 upgrade: - crafted a new patch that follows steps of Solaris 10 going v15 (many more revisions added) - released a CFT on freebsd-current@, with very detailed patch information including Solaris 10 patch numbers - positive testing reports from running on i386, amd64 and sparc systems (and cross-importing pools, etc.) - the patch is now ready for head, I will commit it right after 8.1 gets released - an UPDATING entry will give notice about pool upgrading and that the new utilities need a new kernel I have tested the binary compatibility with older utilities as well, and it works properly, here is the matrix: Old kernel, old userland: OK Old kernel, new userland: ERROR New kernel, old userland: OK New kernel, new userland: OK So this fullfills the import constraint into stable/8, I will tag the commit with a 2-month MFC after: Link to the patch information file, including all imported revisions, bug-ids and reference to Solairis 10 patch numbers: http://people.freebsd.org/~mm/patches/zfs/v15/head-v15-v3.html http://people.freebsd.org/~mm/patches/zfs/v15/head-v15-v3.txt Direct link to the patch: http://people.freebsd.org/~mm/patches/zfs/v15/head-v15-v3.patch And I repeat: the patch applies cleanly against stable/8 as well (and works). Cheers, mm From owner-zfs-devel@FreeBSD.ORG Wed Jul 7 14:34:34 2010 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D3460106564A; Wed, 7 Jul 2010 14:34:34 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from harmony.bsdimp.com (bsdimp.com [199.45.160.85]) by mx1.freebsd.org (Postfix) with ESMTP id 7B0AE8FC1F; Wed, 7 Jul 2010 14:34:34 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by harmony.bsdimp.com (8.14.3/8.14.1) with ESMTP id o67EWjcX031759; Wed, 7 Jul 2010 08:32:45 -0600 (MDT) (envelope-from imp@bsdimp.com) Date: Wed, 07 Jul 2010 08:33:01 -0600 (MDT) Message-Id: <20100707.083301.149607579557318410.imp@bsdimp.com> To: mm@FreeBSD.org From: "M. Warner Losh" In-Reply-To: <4C3420AD.8040709@FreeBSD.org> References: <4C3420AD.8040709@FreeBSD.org> X-Mailer: Mew version 6.3 on Emacs 22.3 / Mule 5.0 (SAKAKI) Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit Cc: zfs-devel@FreeBSD.org Subject: Re: Status update on v15 import X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 07 Jul 2010 14:34:34 -0000 In message: <4C3420AD.8040709@FreeBSD.org> Martin Matuska writes: : Hi all, : : I have some update on the status of the v15 upgrade: : : - crafted a new patch that follows steps of Solaris 10 going v15 (many : more revisions added) : - released a CFT on freebsd-current@, with very detailed patch : information including Solaris 10 patch numbers : - positive testing reports from running on i386, amd64 and sparc systems : (and cross-importing pools, etc.) : - the patch is now ready for head, I will commit it right after 8.1 gets : released : - an UPDATING entry will give notice about pool upgrading and that the : new utilities need a new kernel : : I have tested the binary compatibility with older utilities as well, and : it works properly, here is the matrix: : : Old kernel, old userland: OK : Old kernel, new userland: ERROR : New kernel, old userland: OK : New kernel, new userland: OK : : So this fullfills the import constraint into stable/8, I will tag the : commit with a 2-month MFC after: Actually, old kernel + new userland == ERROR does not necessarily fulfill the import constraints. What does ERROR mean here? For -current it might be alright (might here depends on what the consequences are). For -stable, the bar is much higher. Again, it might be OK, but what are the consequences of this error? While the project might not support totally arbitrary movements with kernel and userland, each case is judged based on the consequences of the errors. Warner : Link to the patch information file, including all imported revisions, : bug-ids and reference to Solairis 10 patch numbers: : http://people.freebsd.org/~mm/patches/zfs/v15/head-v15-v3.html : http://people.freebsd.org/~mm/patches/zfs/v15/head-v15-v3.txt : : Direct link to the patch: : http://people.freebsd.org/~mm/patches/zfs/v15/head-v15-v3.patch : : And I repeat: the patch applies cleanly against stable/8 as well (and : works). : : Cheers, : mm : _______________________________________________ : zfs-devel@freebsd.org mailing list : http://lists.freebsd.org/mailman/listinfo/zfs-devel : To unsubscribe, send any mail to "zfs-devel-unsubscribe@freebsd.org" : : From owner-zfs-devel@FreeBSD.ORG Thu Jul 8 10:51:48 2010 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8D6B8106564A for ; Thu, 8 Jul 2010 10:51:48 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from mail.vx.sk (core.vx.sk [188.40.32.143]) by mx1.freebsd.org (Postfix) with ESMTP id D0ABC8FC08 for ; Thu, 8 Jul 2010 10:51:47 +0000 (UTC) Received: from core.vx.sk (localhost [127.0.0.1]) by mail.vx.sk (Postfix) with ESMTP id DE6C4F361A; Thu, 8 Jul 2010 12:51:46 +0200 (CEST) X-Virus-Scanned: amavisd-new at mail.vx.sk Received: from mail.vx.sk ([127.0.0.1]) by core.vx.sk (mail.vx.sk [127.0.0.1]) (amavisd-new, port 10024) with LMTP id 0Z2eVyt1K5Ff; Thu, 8 Jul 2010 12:51:44 +0200 (CEST) Received: from [10.9.8.1] (188-167-78-139.dynamic.chello.sk [188.167.78.139]) by mail.vx.sk (Postfix) with ESMTPSA id B6B9EF3610; Thu, 8 Jul 2010 12:51:44 +0200 (CEST) Message-ID: <4C35ADC2.7060900@FreeBSD.org> Date: Thu, 08 Jul 2010 12:51:46 +0200 From: Martin Matuska User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; sk; rv:1.8.1.23) Gecko/20090812 Lightning/0.9 Thunderbird/2.0.0.23 Mnenhy/0.7.5.0 MIME-Version: 1.0 To: "M. Warner Losh" References: <4C3420AD.8040709@FreeBSD.org> <20100707.083301.149607579557318410.imp@bsdimp.com> In-Reply-To: <20100707.083301.149607579557318410.imp@bsdimp.com> X-Enigmail-Version: 1.0.1 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: zfs-devel@FreeBSD.org Subject: Re: Status update on v15 import X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 Jul 2010 10:51:48 -0000 As for good news, I have fixed this as well. Now it is: Old kernel, old userland: OK (both zfs and zpool work) Old kernel, new userland: OK (both zfs and zpool work) New kernel, old userland: OK (both zfs and zpool work) New kernel, new userland: OK (both zfs and zpool work) The problem was that in v15 the check for internal private datasets was moved out of userland to kernel. So it was very easy to introduce a very simple compatibility fix and check for "$" as the first character in appropriate userland parts, too. The patch for the internal private datasets is here: http://people.freebsd.org/~mm/patches/zfs/v15/head-v15-v3-compat.patch (head-v15-v3.patch needs to be applied first) Dňa 7. 7. 2010 16:33, M. Warner Losh wrote / napísal(a): > In message: <4C3420AD.8040709@FreeBSD.org> > Martin Matuska writes: > : Hi all, > : > : I have some update on the status of the v15 upgrade: > : > : - crafted a new patch that follows steps of Solaris 10 going v15 (many > : more revisions added) > : - released a CFT on freebsd-current@, with very detailed patch > : information including Solaris 10 patch numbers > : - positive testing reports from running on i386, amd64 and sparc systems > : (and cross-importing pools, etc.) > : - the patch is now ready for head, I will commit it right after 8.1 gets > : released > : - an UPDATING entry will give notice about pool upgrading and that the > : new utilities need a new kernel > : > : I have tested the binary compatibility with older utilities as well, and > : it works properly, here is the matrix: > : > : Old kernel, old userland: OK > : Old kernel, new userland: ERROR > : New kernel, old userland: OK > : New kernel, new userland: OK > : > : So this fullfills the import constraint into stable/8, I will tag the > : commit with a 2-month MFC after: > > Actually, old kernel + new userland == ERROR does not necessarily > fulfill the import constraints. What does ERROR mean here? For > -current it might be alright (might here depends on what the > consequences are). For -stable, the bar is much higher. Again, it > might be OK, but what are the consequences of this error? > > While the project might not support totally arbitrary movements with > kernel and userland, each case is judged based on the consequences of > the errors. > > Warner > > : Link to the patch information file, including all imported revisions, > : bug-ids and reference to Solairis 10 patch numbers: > : http://people.freebsd.org/~mm/patches/zfs/v15/head-v15-v3.html > : http://people.freebsd.org/~mm/patches/zfs/v15/head-v15-v3.txt > : > : Direct link to the patch: > : http://people.freebsd.org/~mm/patches/zfs/v15/head-v15-v3.patch > : > : And I repeat: the patch applies cleanly against stable/8 as well (and > : works). > : > : Cheers, > : mm > : _______________________________________________ > : zfs-devel@freebsd.org mailing list > : http://lists.freebsd.org/mailman/listinfo/zfs-devel > : To unsubscribe, send any mail to "zfs-devel-unsubscribe@freebsd.org" > : > : > _______________________________________________ > zfs-devel@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/zfs-devel > To unsubscribe, send any mail to "zfs-devel-unsubscribe@freebsd.org" > From owner-zfs-devel@FreeBSD.ORG Thu Jul 8 19:05:20 2010 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 786D41065670; Thu, 8 Jul 2010 19:05:20 +0000 (UTC) (envelope-from delphij@delphij.net) Received: from tarsier.geekcn.org (tarsier.geekcn.org [IPv6:2001:470:a803::1]) by mx1.freebsd.org (Postfix) with ESMTP id 212288FC0A; Thu, 8 Jul 2010 19:05:20 +0000 (UTC) Received: from mail.geekcn.org (tarsier.geekcn.org [211.166.10.233]) by tarsier.geekcn.org (Postfix) with ESMTP id 1D75AA59968; Fri, 9 Jul 2010 03:05:15 +0800 (CST) X-Virus-Scanned: amavisd-new at geekcn.org Received: from tarsier.geekcn.org ([211.166.10.233]) by mail.geekcn.org (mail.geekcn.org [211.166.10.233]) (amavisd-new, port 10024) with LMTP id BlRyfZGc+oNQ; Fri, 9 Jul 2010 03:05:04 +0800 (CST) Received: from delta.delphij.net (drawbridge.ixsystems.com [206.40.55.65]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by tarsier.geekcn.org (Postfix) with ESMTPSA id 08A44A59980; Fri, 9 Jul 2010 03:04:54 +0800 (CST) DomainKey-Signature: a=rsa-sha1; s=default; d=delphij.net; c=nofws; q=dns; h=message-id:date:from:reply-to:organization:user-agent: mime-version:to:cc:subject:references:in-reply-to: x-enigmail-version:openpgp:content-type:content-transfer-encoding; b=GOaK6S1Bh68IBnPqEKbTa73U4hu5/n1TPjyuIG+akapHEGsUVG8ZQpJRALRrc+7ih XYP0VgVDdp0svhFRVqQOg== Message-ID: <4C362153.5010604@delphij.net> Date: Thu, 08 Jul 2010 12:04:51 -0700 From: Xin LI Organization: The Geek China Organization User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.1.10) Gecko/20100629 Thunderbird/3.0.5 ThunderBrowse/3.3 MIME-Version: 1.0 To: Martin Matuska References: <4C3420AD.8040709@FreeBSD.org> <20100707.083301.149607579557318410.imp@bsdimp.com> <4C35ADC2.7060900@FreeBSD.org> In-Reply-To: <4C35ADC2.7060900@FreeBSD.org> X-Enigmail-Version: 1.0.1 OpenPGP: id=3FCA37C1; url=http://www.delphij.net/delphij.asc Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Cc: zfs-devel@FreeBSD.org Subject: Re: Status update on v15 import X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: d@delphij.net List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 Jul 2010 19:05:20 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 On 2010/07/08 03:51, Martin Matuska wrote: > As for good news, I have fixed this as well. > > Now it is: > > Old kernel, old userland: OK (both zfs and zpool work) > Old kernel, new userland: OK (both zfs and zpool work) > New kernel, old userland: OK (both zfs and zpool work) > New kernel, new userland: OK (both zfs and zpool work) > > The problem was that in v15 the check for internal private datasets was > moved out of userland to kernel. > So it was very easy to introduce a very simple compatibility fix and > check for "$" as the first character in appropriate userland parts, too. > > The patch for the internal private datasets is here: > http://people.freebsd.org/~mm/patches/zfs/v15/head-v15-v3-compat.patch > > (head-v15-v3.patch needs to be applied first) That's nice! Thanks for working on this! (Reading the code it seems that '$' is the indicator of hidden dataset regardless whether it's the first character. -CURRENT kernel uses the same strchr() as the patch did.) Cheers, - -- Xin LI http://www.delphij.net/ FreeBSD - The Power to Serve! Live free or die -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.15 (FreeBSD) iQEcBAEBCAAGBQJMNiFTAAoJEATO+BI/yjfBcR0IAIsoMjj2Y+0nHNd5lt6/uPVG S+8V0cNj1pTEmg3nqMcLdmTwYrFnS8ueXdJna295Ky2Qkv4KKRtK2cAHQGr5PMPD WF7twJpU7hvBvfO5gduRulZ9yMxchXsWEOezg6nFHvfqjBTIatUrEhWTPM5GHSoD junaAn+McO8p+ysMj1pR0Vwb1YqVOHT+ZQzqt57XH/JsiDD97Qpq+44D+QLltCmd xK8Ht+0tuxbrWgBLoCPbClO3wplpdhi8vdh4GhfUrry9WzHKtHJuhYq3e10K6ZVu yVRrkqb0Z8jFaAtju9CFx2lnExCm1x0+rDu3564C7wLsUWK634wxy0yW3IdEkI8= =2WbQ -----END PGP SIGNATURE----- From owner-zfs-devel@FreeBSD.ORG Tue Jul 20 16:41:01 2010 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 744E4106566B for ; Tue, 20 Jul 2010 16:41:01 +0000 (UTC) (envelope-from dewcopper@gmail.com) Received: from mail-pw0-f54.google.com (mail-pw0-f54.google.com [209.85.160.54]) by mx1.freebsd.org (Postfix) with ESMTP id 497148FC14 for ; Tue, 20 Jul 2010 16:41:01 +0000 (UTC) Received: by pwj9 with SMTP id 9so2579423pwj.13 for ; Tue, 20 Jul 2010 09:41:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:sender:received:date :x-google-sender-auth:message-id:subject:from:to:content-type; bh=eUuaz7lCoi4jfUJyksZZe+jcvCEFwvX/WlPK0fZsDgE=; b=JE+9KJM4Dj6PKyeVKUjzkBivAwkDTlGnrZ660ROb5UeJHgWcuAyCBRLjxVok+uV+Wg URHouP86oc5XUDUa2lw98KCvwzH8EcGAJYYmpzcJAKmc8dSuSMI0Es1+JXvwGcvvfu7K lJaVA8m0c50JcUzL4BEvoFatfnVihdw1gASew= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:date:x-google-sender-auth:message-id:subject :from:to:content-type; b=rdKs/IemXnlQp74Ka6nTpdpTD3ocDeY0Bdtey9pexawihoFhzN19qBh8zmnXYrkZS4 JxY93jg25rXL/zSySK1CKJEYTOa201d6WKB86Al7a3sjVXnRL3vB5Svx9Wz0A/sOmtZc oJXDLwu2Uf/bS/e3lpPiH4Tfda1P2j07o4cOI= MIME-Version: 1.0 Received: by 10.142.148.10 with SMTP id v10mr9535691wfd.105.1279644059397; Tue, 20 Jul 2010 09:40:59 -0700 (PDT) Sender: dewcopper@gmail.com Received: by 10.229.218.146 with HTTP; Tue, 20 Jul 2010 09:40:58 -0700 (PDT) Date: Tue, 20 Jul 2010 09:40:58 -0700 X-Google-Sender-Auth: rvn6ZePQa0g2Rhs46yDJJvQ9g4I Message-ID: From: Tushar Tambay To: zfs-devel@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: conf call today X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Jul 2010 16:41:01 -0000 hi all, we are supposed to be having our second monthly call today at 10 AM PST but i am guessing that most people have forgotten that it was today :-) apologies for not sending out a reminder mail yesterday. i meant to but did not get around to it. i propose we move the call this week to thursday 10 AM PST. please respond if that does that not work for you. cheers, -- tyt phone: 408 203 9736 (c) - From owner-zfs-devel@FreeBSD.ORG Thu Jul 22 17:14:03 2010 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4DF1B1065673 for ; Thu, 22 Jul 2010 17:14:03 +0000 (UTC) (envelope-from dewcopper@gmail.com) Received: from mail-gw0-f54.google.com (mail-gw0-f54.google.com [74.125.83.54]) by mx1.freebsd.org (Postfix) with ESMTP id 06D9C8FC16 for ; Thu, 22 Jul 2010 17:14:02 +0000 (UTC) Received: by gwj23 with SMTP id 23so258387gwj.13 for ; Thu, 22 Jul 2010 10:14:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:sender:received :in-reply-to:references:date:x-google-sender-auth:message-id:subject :from:to:content-type; bh=+KJVZMAmFWS/bkiMD+ZLHVFB2RbKtm0dJ80cgY2AkV4=; b=ecLyjWKzCW4burpufKRSK7zeblN8F96lIG9QisT+AhtT+dRNqfNVRmxs3sxLAoOsoG oP8we2vDsFCiE0kwur0Pxo4yI0mJC2Pr5nrEWm3+BoXYPkM1b9bmJmId/YzsTlHZ9Kqv jp4fT9Zzi1ltUmYw0vCS0+iHcwI8H7paRzH6c= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:content-type; b=iLxBt0rx5CeTHYd9jhtacSv8/V8ue/67uPHQnJaKlNAdSD4gjuFNag+A3uGJrhdZfx ZiOSMbNXWhstOov0X/TE7BtfyVm9SxCBswu4z5ZP1k7WuKkJMtsZeKWebL0zXltGokg8 B41gB996f6HlSCFzaCSMp2maM3XjC4sqbyIQI= MIME-Version: 1.0 Received: by 10.224.66.167 with SMTP id n39mr1454247qai.391.1279818842342; Thu, 22 Jul 2010 10:14:02 -0700 (PDT) Sender: dewcopper@gmail.com Received: by 10.229.14.97 with HTTP; Thu, 22 Jul 2010 10:14:02 -0700 (PDT) In-Reply-To: References: Date: Thu, 22 Jul 2010 10:14:02 -0700 X-Google-Sender-Auth: Cr18G1JU5q30KRMsawPraUF8dCs Message-ID: From: Tushar Tambay To: zfs-devel@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: Re: conf call today X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Jul 2010 17:14:03 -0000 hi all, samir, ganesh, pawel and i are on the conference call the call at the moment. if i see anyone else on skype, i will add them. cheers, On Tue, Jul 20, 2010 at 9:40 AM, Tushar Tambay wrote: > hi all, > > we are supposed to be having our second monthly call today at 10 AM PST but > i am guessing that most people have forgotten that it was today :-) > > apologies for not sending out a reminder mail yesterday. i meant to but did > not get around to it. > > i propose we move the call this week to thursday 10 AM PST. > > please respond if that does that not work for you. > > cheers, > > -- > tyt > > phone: 408 203 9736 (c) > - > > > -- tyt phone: 408 203 9736 (c) - From owner-zfs-devel@FreeBSD.ORG Thu Jul 22 17:52:34 2010 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C914A1065670 for ; Thu, 22 Jul 2010 17:52:34 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (chello089077043238.chello.pl [89.77.43.238]) by mx1.freebsd.org (Postfix) with ESMTP id 1F4F48FC16 for ; Thu, 22 Jul 2010 17:52:24 +0000 (UTC) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id 121EB45E87; Thu, 22 Jul 2010 19:52:22 +0200 (CEST) Received: from localhost (chello089077043238.chello.pl [89.77.43.238]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id 8E74245E82 for ; Thu, 22 Jul 2010 19:52:16 +0200 (CEST) Date: Thu, 22 Jul 2010 19:51:45 +0200 From: Pawel Jakub Dawidek To: zfs-devel@FreeBSD.org Message-ID: <20100722175145.GA2409@garage.freebsd.pl> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="u3/rZRmxL6MmkK24" Content-Disposition: inline User-Agent: Mutt/1.4.2.3i X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 9.0-CURRENT amd64 X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-0.6 required=4.5 tests=BAYES_00,RCVD_IN_SORBS_DUL autolearn=no version=3.0.4 Cc: Subject: zfs_cmd_t size issue. X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Jul 2010 17:52:34 -0000 --u3/rZRmxL6MmkK24 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Samir, could you take a look at this change: http://svn.freebsd.org/changeset/base/206051 I think it should fix ioctl structure size limti for good. I'll investigate if the change can be merged to stable/8. --=20 Pawel Jakub Dawidek http://www.wheelsystems.com pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --u3/rZRmxL6MmkK24 Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (FreeBSD) iEYEARECAAYFAkxIhTEACgkQForvXbEpPzRI4wCfQoBXbiwEswb0UdgM5iQ8Qt7c CfgAoIanJ7YnYw8eegFHK7fEgoEWVYx+ =wDdW -----END PGP SIGNATURE----- --u3/rZRmxL6MmkK24-- From owner-zfs-devel@FreeBSD.ORG Thu Jul 22 17:57:21 2010 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BA1A51065678; Thu, 22 Jul 2010 17:57:21 +0000 (UTC) (envelope-from delphij@delphij.net) Received: from tarsier.geekcn.org (tarsier.geekcn.org [IPv6:2001:470:a803::1]) by mx1.freebsd.org (Postfix) with ESMTP id 63AE18FC18; Thu, 22 Jul 2010 17:57:21 +0000 (UTC) Received: from mail.geekcn.org (tarsier.geekcn.org [211.166.10.233]) by tarsier.geekcn.org (Postfix) with ESMTP id 356DCA59F49; Fri, 23 Jul 2010 01:57:20 +0800 (CST) X-Virus-Scanned: amavisd-new at geekcn.org Received: from tarsier.geekcn.org ([211.166.10.233]) by mail.geekcn.org (mail.geekcn.org [211.166.10.233]) (amavisd-new, port 10024) with LMTP id 2cFm2Mp2BNKv; Fri, 23 Jul 2010 01:57:13 +0800 (CST) Received: from delta.delphij.net (drawbridge.ixsystems.com [206.40.55.65]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by tarsier.geekcn.org (Postfix) with ESMTPSA id 7A4D2A59F86; Fri, 23 Jul 2010 01:57:10 +0800 (CST) DomainKey-Signature: a=rsa-sha1; s=default; d=delphij.net; c=nofws; q=dns; h=message-id:date:from:reply-to:organization:user-agent: mime-version:to:cc:subject:references:in-reply-to: x-enigmail-version:openpgp:content-type:content-transfer-encoding; b=aCPbuxxQyGtew3W5BnL1WhW8MTi/PI4AFi8SHTy0psoFV6JsGoLc/fJcIg31/XUBV JWrmN8r9MXawbMRmGvxFw== Message-ID: <4C488671.7040908@delphij.net> Date: Thu, 22 Jul 2010 10:57:05 -0700 From: Xin LI Organization: The Geek China Organization User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.1.11) Gecko/20100721 Thunderbird/3.0.6 ThunderBrowse/3.3 MIME-Version: 1.0 To: Pawel Jakub Dawidek References: <20100722175145.GA2409@garage.freebsd.pl> In-Reply-To: <20100722175145.GA2409@garage.freebsd.pl> X-Enigmail-Version: 1.0.1 OpenPGP: id=3FCA37C1; url=http://www.delphij.net/delphij.asc Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Cc: zfs-devel@FreeBSD.org Subject: Re: zfs_cmd_t size issue. X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: d@delphij.net List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Jul 2010 17:57:21 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 On 2010/07/22 10:51, Pawel Jakub Dawidek wrote: > Samir, could you take a look at this change: > > http://svn.freebsd.org/changeset/base/206051 > > I think it should fix ioctl structure size limti for good. > I'll investigate if the change can be merged to stable/8. I think this one is Ok (note that we may want to disable the sign-extension warning for non-verbose or even non-DIAGNOSTIC kernels as well). Cheers, - -- Xin LI http://www.delphij.net/ FreeBSD - The Power to Serve! Live free or die -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.16 (FreeBSD) iQEcBAEBCAAGBQJMSIZxAAoJEATO+BI/yjfB/AMH/A3NUkmxSOfHFoRJe51LOlj8 GXRs3emfdRQDs63Q+pJz591Ndhbhyg/0+SA2prSWyorBcZN2oVl9+GEEx+YaWPCE DERmaI8wndw1eyKe0Uj8vvMKUU9CSoz6g+qpLUKt7yU1q3yxWRfI52u11lywYufb umJGIWMdXCWa5DtC+k2DH/8RK1gDC/e4bsu3M5KyN8GEXskVanXwRZ5SVr9vqBRR vx6TZEbgycjzd0/xPeTTqj2YpXyKg3R69Dd240KOyTrOsaieyS4bYLkRZcdMi+NX 0Zonik5+6fVtah3eWBSmWe3cW4W81/G02hZZlGdiuT2Z8T2KQmBAAcKfK5ZQS6Y= =b7We -----END PGP SIGNATURE----- From owner-zfs-devel@FreeBSD.ORG Thu Jul 22 18:10:30 2010 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B3358106566C for ; Thu, 22 Jul 2010 18:10:30 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (chello089077043238.chello.pl [89.77.43.238]) by mx1.freebsd.org (Postfix) with ESMTP id 099D88FC1C for ; Thu, 22 Jul 2010 18:10:29 +0000 (UTC) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id 9470E45E86; Thu, 22 Jul 2010 20:10:28 +0200 (CEST) Received: from localhost (chello089077043238.chello.pl [89.77.43.238]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id A4C7245C99 for ; Thu, 22 Jul 2010 20:10:23 +0200 (CEST) Date: Thu, 22 Jul 2010 20:09:52 +0200 From: Pawel Jakub Dawidek To: zfs-devel@FreeBSD.org Message-ID: <20100722180952.GB2409@garage.freebsd.pl> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="kXdP64Ggrk/fb43R" Content-Disposition: inline User-Agent: Mutt/1.4.2.3i X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 9.0-CURRENT amd64 X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-0.6 required=4.5 tests=BAYES_00,RCVD_IN_SORBS_DUL autolearn=no version=3.0.4 Cc: Subject: NetApp/ZFS threats. X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Jul 2010 18:10:30 -0000 --kXdP64Ggrk/fb43R Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable There was discussion on zfs-discuss@opensolaris mailing list about this. Not sure if there was anything interesting, but: http://mail.opensolaris.org/pipermail/zfs-discuss/2010-July/042931.html --=20 Pawel Jakub Dawidek http://www.wheelsystems.com pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --kXdP64Ggrk/fb43R Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (FreeBSD) iEYEARECAAYFAkxIiW8ACgkQForvXbEpPzQsAQCgy2mrhrZ3azyA4xjGAQOaJklD p6sAoJAsqEx0SQ/bcNcLl5oyo0v9kXlg =d4qn -----END PGP SIGNATURE----- --kXdP64Ggrk/fb43R-- From owner-zfs-devel@FreeBSD.ORG Thu Jul 22 18:43:12 2010 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 25057106566B for ; Thu, 22 Jul 2010 18:43:12 +0000 (UTC) (envelope-from dewcopper@gmail.com) Received: from mail-qw0-f54.google.com (mail-qw0-f54.google.com [209.85.216.54]) by mx1.freebsd.org (Postfix) with ESMTP id ADCFB8FC16 for ; Thu, 22 Jul 2010 18:43:10 +0000 (UTC) Received: by qwg5 with SMTP id 5so73483qwg.13 for ; Thu, 22 Jul 2010 11:43:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:sender:received:date :x-google-sender-auth:message-id:subject:from:to:content-type; bh=mWfukmRifgPiMqAnEbp3wMuusqRwE0li7gjlZT2T6go=; b=w1VFfIa4Qzm4v3YgPN103JkBuRb7SJSKiUjjh/iruOIqFLX9gsLttqE3TH5y3aP7Nj 1Eylh1pcGB5YSSjyLItRRwvEL0mob+42S2yZzxsAaMCHlp2kS5cJSWnfheG7MWVdjDHz HuSJqF3xHTSVWiFjjJPZ0veET14blIeHYu4Zs= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:date:x-google-sender-auth:message-id:subject :from:to:content-type; b=ie2MWmRgxL6wOG9Tv+Bd19dgSO8aRrNdC2/cvltbUTQuneyipmbM2ZeqCfnF776Kw9 rL5fBu5iMoCk83rEil+86980nEC+ghyawighfYVsPo5d+WDdz8OsAiocMz0AyX+rKJBh XJ+0jr6BPRIL7rxjhbarRWfPP5rgZVU90+mwo= MIME-Version: 1.0 Received: by 10.224.66.8 with SMTP id l8mr1579470qai.306.1279824168826; Thu, 22 Jul 2010 11:42:48 -0700 (PDT) Sender: dewcopper@gmail.com Received: by 10.229.14.97 with HTTP; Thu, 22 Jul 2010 11:42:48 -0700 (PDT) Date: Thu, 22 Jul 2010 11:42:48 -0700 X-Google-Sender-Auth: 27XpPfEbsndETpxbd_Zd3RFitnQ Message-ID: From: Tushar Tambay To: zfs-devel@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: zfs/bsd conf call minutes X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Jul 2010 18:43:12 -0000 date: 7/22/10 attendees : samir, ganesh, xin, pawel, tushar - martin updated head to zfs v15 and enabled use py-zfs commands - commands that are now enabled (in head) include :- - zfs-allow, zfs-unallow, zfs-groupspace, zfs-userspace - plan (pawel, martin) - bsd head zfs version will match stable solaris (not opensolaris) zfs version. - on perforce we will keep up with latest & greatest version v26 (soon v27) - re. netapp's receent threatened suit against coraid - (pawel) netapp cannot go after freebsd foundation since the foundation does not own freebsd code (this is unlike the situation that exists in netbsd where the netbsd foundation has legal rights over the code) - perhaps justin can share his thoughts on this issue with the group ? action items - samir / ganesh - investigate usefulness of py-zfs enabled commands on v26. then decide whether to port py-zfs - ganesh - send out details of panic in zpool create (using files as devices) - samir - checkin changes to common tree. -- tyt phone: 408 203 9736 (c) - From owner-zfs-devel@FreeBSD.ORG Thu Jul 22 21:11:54 2010 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0452F1065672 for ; Thu, 22 Jul 2010 21:11:54 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from mail.vx.sk (core.vx.sk [188.40.32.143]) by mx1.freebsd.org (Postfix) with ESMTP id 4E4218FC15 for ; Thu, 22 Jul 2010 21:11:53 +0000 (UTC) Received: from core.vx.sk (localhost [127.0.0.1]) by mail.vx.sk (Postfix) with ESMTP id 527B71117FE for ; Thu, 22 Jul 2010 23:11:52 +0200 (CEST) X-Virus-Scanned: amavisd-new at mail.vx.sk Received: from mail.vx.sk ([127.0.0.1]) by core.vx.sk (mail.vx.sk [127.0.0.1]) (amavisd-new, port 10024) with LMTP id ZQnCppujyZjK for ; Thu, 22 Jul 2010 23:11:45 +0200 (CEST) Received: from [10.9.8.1] (188-167-78-139.dynamic.chello.sk [188.167.78.139]) by mail.vx.sk (Postfix) with ESMTPSA id 430EA1117E6 for ; Thu, 22 Jul 2010 23:11:45 +0200 (CEST) Message-ID: <4C48B418.4000907@FreeBSD.org> Date: Thu, 22 Jul 2010 23:11:52 +0200 From: Martin Matuska User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; sk; rv:1.8.1.23) Gecko/20090812 Lightning/0.9 Thunderbird/2.0.0.23 Mnenhy/0.7.5.0 MIME-Version: 1.0 References: <20100722180952.GB2409@garage.freebsd.pl> In-Reply-To: <20100722180952.GB2409@garage.freebsd.pl> X-Enigmail-Version: 1.1.1 Content-Type: text/plain; charset=windows-1250 Content-Transfer-Encoding: 8bit X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: zfs-devel@FreeBSD.org Subject: Re: NetApp/ZFS threats. X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Jul 2010 21:11:54 -0000 Here is a link to the ZFS lawsuit information: http://www.sun.com/lawsuit/zfs/ Under "More Information" you find links to many relevant legal documents. The patent 5,819,292 (dealing with "copy on write") and 5,761,662 ("Personalized retrieval using user-defined profile") have been already rejected by final action of the USPTO (United States Patent and Trademark Office). Personalized retrieval using user-defined profile: http://www.freepatentsonline.com/5761662.html http://www.sun.com/lawsuit/zfs/docs/US5761662FinalOfficeAction3-2009.pdf Rejected claims: 1,4-8,11-17,20-32 (rejected as not patentable, these claims cannot be legally enforced) Copy-on-write: http://www.freepatentsonline.com/5819292.html http://www.sun.com/lawsuit/zfs/docs/US5819292FinalOfficeActionJun2009.pdf Rejected claims: 4,8,11-15,20,23,24 (rejected as not patentable, these claims cannot be legally enforced) Confirmed claims: 1, 21, 22 (these claims can be sued and enforced) Snapshots (7,174,352): http://www.freepatentsonline.com/7174352.html http://www.sun.com/lawsuit/zfs/docs/US7174352OfficeAction2-2009.pdf Clones (6,857,001): http://www.freepatentsonline.com/6857001.html Da 22. 7. 2010 20:09, Pawel Jakub Dawidek wrote / napsal(a): > There was discussion on zfs-discuss@opensolaris mailing list about this. > Not sure if there was anything interesting, but: > > http://mail.opensolaris.org/pipermail/zfs-discuss/2010-July/042931.html > From owner-zfs-devel@FreeBSD.ORG Thu Jul 22 22:27:31 2010 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 646B11065672; Thu, 22 Jul 2010 22:27:31 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from mail.vx.sk (core.vx.sk [188.40.32.143]) by mx1.freebsd.org (Postfix) with ESMTP id E033F8FC18; Thu, 22 Jul 2010 22:27:30 +0000 (UTC) Received: from core.vx.sk (localhost [127.0.0.1]) by mail.vx.sk (Postfix) with ESMTP id EFD62BD11A; Fri, 23 Jul 2010 00:27:29 +0200 (CEST) X-Virus-Scanned: amavisd-new at mail.vx.sk Received: from mail.vx.sk ([127.0.0.1]) by core.vx.sk (mail.vx.sk [127.0.0.1]) (amavisd-new, port 10024) with LMTP id qY9O4iMZlDXP; Fri, 23 Jul 2010 00:27:27 +0200 (CEST) Received: from [10.9.8.1] (188-167-78-139.dynamic.chello.sk [188.167.78.139]) by mail.vx.sk (Postfix) with ESMTPSA id 7C8F6BD10E; Fri, 23 Jul 2010 00:27:27 +0200 (CEST) Message-ID: <4C48C5D7.7070509@FreeBSD.org> Date: Fri, 23 Jul 2010 00:27:35 +0200 From: Martin Matuska User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; sk; rv:1.8.1.23) Gecko/20090812 Lightning/0.9 Thunderbird/2.0.0.23 Mnenhy/0.7.5.0 MIME-Version: 1.0 To: zfs-devel@FreeBSD.org X-Enigmail-Version: 1.1.1 Content-Type: multipart/mixed; boundary="------------040408050608060809080606" Cc: Xin LI Subject: ZFS patch proposal: zfs_fuid.c fix X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Jul 2010 22:27:31 -0000 This is a multi-part message in MIME format. --------------040408050608060809080606 Content-Type: text/plain; charset=windows-1250 Content-Transfer-Encoding: 7bit I have examined the opensolaris SMB code and would like to propose the attached patch. It fixes zfs_fuid.c and acts like OpenSolaris with an unresolved IDMAP user: uses nulldomain (no domain) and unknown SMB user (UID_NOBODY). In addition, I would like to allow users to set/unsed the sharesmb property even if it does nothing. It is good for imported pools that have datasets with this property enabled, so that it can be disabled. It should fix kern/145778 and kern/148709. --------------040408050608060809080606 Content-Type: text/plain; name="head-sharesmb.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="head-sharesmb.patch" Index: cddl/contrib/opensolaris/lib/libzfs/common/libzfs_dataset.c =================================================================== --- cddl/contrib/opensolaris/lib/libzfs/common/libzfs_dataset.c (revision 210319) +++ cddl/contrib/opensolaris/lib/libzfs/common/libzfs_dataset.c (working copy) @@ -1265,7 +1265,6 @@ case ZFS_PROP_XATTR: case ZFS_PROP_VSCAN: case ZFS_PROP_NBMAND: - case ZFS_PROP_SHARESMB: (void) snprintf(errbuf, sizeof (errbuf), "property '%s' not supported on FreeBSD", propname); ret = zfs_error(hdl, EZFS_PERM, errbuf); Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_fuid.c =================================================================== --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_fuid.c (revision 210319) +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_fuid.c (working copy) @@ -410,7 +410,7 @@ domain = zfs_fuid_find_by_idx(zfsvfs, index); ASSERT(domain != NULL); -#ifdef TODO +#ifdef sun if (type == ZFS_OWNER || type == ZFS_ACE_USER) { (void) kidmap_getuidbysid(crgetzone(cr), domain, FUID_RID(fuid), &id); @@ -418,9 +418,9 @@ (void) kidmap_getgidbysid(crgetzone(cr), domain, FUID_RID(fuid), &id); } -#else - panic(__func__); -#endif +#else /* sun */ + id = UID_NOBODY; +#endif /* sun */ return (id); } @@ -514,21 +514,21 @@ if (!zfsvfs->z_use_fuids || !IS_EPHEMERAL(id)) return ((uint64_t)id); -#ifdef TODO +#ifdef sun ksid = crgetsid(cr, (type == ZFS_OWNER) ? KSID_OWNER : KSID_GROUP); VERIFY(ksid != NULL); rid = ksid_getrid(ksid); domain = ksid_getdomain(ksid); - +#else /* sun */ + rid = UID_NOBODY; + domain = nulldomain; +#endif /* sun */ idx = zfs_fuid_find_by_domain(zfsvfs, domain, &kdomain, B_TRUE); zfs_fuid_node_add(fuidp, kdomain, rid, idx, id, type); return (FUID_ENCODE(idx, rid)); -#else - panic(__func__); -#endif } /* @@ -597,7 +597,7 @@ }; domain = fuidp->z_domain_table[idx -1]; } else { -#ifdef TODO +#ifdef sun if (type == ZFS_OWNER || type == ZFS_ACE_USER) status = kidmap_getsidbyuid(crgetzone(cr), id, &domain, &rid); @@ -606,6 +606,7 @@ &domain, &rid); if (status != 0) { +#endif /* sun */ /* * When returning nobody we will need to * make a dummy fuid table entry for logging @@ -613,10 +614,9 @@ */ rid = UID_NOBODY; domain = nulldomain; +#ifdef sun } -#else - panic(__func__); -#endif +#endif /* sun */ } idx = zfs_fuid_find_by_domain(zfsvfs, domain, &kdomain, B_TRUE); --------------040408050608060809080606-- From owner-zfs-devel@FreeBSD.ORG Thu Jul 22 22:47:31 2010 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5F26E1065670; Thu, 22 Jul 2010 22:47:31 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (chello089077043238.chello.pl [89.77.43.238]) by mx1.freebsd.org (Postfix) with ESMTP id 6D00F8FC12; Thu, 22 Jul 2010 22:47:19 +0000 (UTC) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id 8AA1245C9B; Fri, 23 Jul 2010 00:47:17 +0200 (CEST) Received: from localhost (chello089077043238.chello.pl [89.77.43.238]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id 204D345C8C; Fri, 23 Jul 2010 00:47:12 +0200 (CEST) Date: Fri, 23 Jul 2010 00:46:41 +0200 From: Pawel Jakub Dawidek To: Martin Matuska Message-ID: <20100722224641.GE2409@garage.freebsd.pl> References: <4C48C5D7.7070509@FreeBSD.org> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="udcq9yAoWb9A4FsZ" Content-Disposition: inline In-Reply-To: <4C48C5D7.7070509@FreeBSD.org> User-Agent: Mutt/1.4.2.3i X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 9.0-CURRENT amd64 X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-0.6 required=4.5 tests=BAYES_00,RCVD_IN_SORBS_DUL autolearn=no version=3.0.4 Cc: zfs-devel@FreeBSD.org, Xin LI Subject: Re: ZFS patch proposal: zfs_fuid.c fix X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Jul 2010 22:47:31 -0000 --udcq9yAoWb9A4FsZ Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, Jul 23, 2010 at 12:27:35AM +0200, Martin Matuska wrote: > I have examined the opensolaris SMB code and would like to propose the > attached patch. It fixes zfs_fuid.c and acts like OpenSolaris with an > unresolved IDMAP user: uses nulldomain (no domain) and unknown SMB user > (UID_NOBODY). >=20 > In addition, I would like to allow users to set/unsed the sharesmb > property even if it does nothing. It is good for imported pools that > have datasets with this property enabled, so that it can be disabled. >=20 > It should fix kern/145778 and kern/148709. The patch looks good to me. > Index: cddl/contrib/opensolaris/lib/libzfs/common/libzfs_dataset.c > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > --- cddl/contrib/opensolaris/lib/libzfs/common/libzfs_dataset.c (revision= 210319) > +++ cddl/contrib/opensolaris/lib/libzfs/common/libzfs_dataset.c (working = copy) > @@ -1265,7 +1265,6 @@ > case ZFS_PROP_XATTR: > case ZFS_PROP_VSCAN: > case ZFS_PROP_NBMAND: > - case ZFS_PROP_SHARESMB: > (void) snprintf(errbuf, sizeof (errbuf), > "property '%s' not supported on FreeBSD", propname); > ret =3D zfs_error(hdl, EZFS_PERM, errbuf); > Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_fuid.c > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_fuid.c (revision 2= 10319) > +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_fuid.c (working co= py) > @@ -410,7 +410,7 @@ > domain =3D zfs_fuid_find_by_idx(zfsvfs, index); > ASSERT(domain !=3D NULL); > =20 > -#ifdef TODO > +#ifdef sun > if (type =3D=3D ZFS_OWNER || type =3D=3D ZFS_ACE_USER) { > (void) kidmap_getuidbysid(crgetzone(cr), domain, > FUID_RID(fuid), &id); > @@ -418,9 +418,9 @@ > (void) kidmap_getgidbysid(crgetzone(cr), domain, > FUID_RID(fuid), &id); > } > -#else > - panic(__func__); > -#endif > +#else /* sun */ > + id =3D UID_NOBODY; > +#endif /* sun */ > return (id); > } > =20 > @@ -514,21 +514,21 @@ > if (!zfsvfs->z_use_fuids || !IS_EPHEMERAL(id)) > return ((uint64_t)id); > =20 > -#ifdef TODO > +#ifdef sun > ksid =3D crgetsid(cr, (type =3D=3D ZFS_OWNER) ? KSID_OWNER : KSID_GROUP= ); > =20 > VERIFY(ksid !=3D NULL); > rid =3D ksid_getrid(ksid); > domain =3D ksid_getdomain(ksid); > - > +#else /* sun */ > + rid =3D UID_NOBODY; > + domain =3D nulldomain; > +#endif /* sun */ > idx =3D zfs_fuid_find_by_domain(zfsvfs, domain, &kdomain, B_TRUE); > =20 > zfs_fuid_node_add(fuidp, kdomain, rid, idx, id, type); > =20 > return (FUID_ENCODE(idx, rid)); > -#else > - panic(__func__); > -#endif > } > =20 > /* > @@ -597,7 +597,7 @@ > }; > domain =3D fuidp->z_domain_table[idx -1]; > } else { > -#ifdef TODO > +#ifdef sun > if (type =3D=3D ZFS_OWNER || type =3D=3D ZFS_ACE_USER) > status =3D kidmap_getsidbyuid(crgetzone(cr), id, > &domain, &rid); > @@ -606,6 +606,7 @@ > &domain, &rid); > =20 > if (status !=3D 0) { > +#endif /* sun */ > /* > * When returning nobody we will need to > * make a dummy fuid table entry for logging > @@ -613,10 +614,9 @@ > */ > rid =3D UID_NOBODY; > domain =3D nulldomain; > +#ifdef sun > } > -#else > - panic(__func__); > -#endif > +#endif /* sun */ > } > =20 > idx =3D zfs_fuid_find_by_domain(zfsvfs, domain, &kdomain, B_TRUE); --=20 Pawel Jakub Dawidek http://www.wheelsystems.com pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --udcq9yAoWb9A4FsZ Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (FreeBSD) iEYEARECAAYFAkxIylAACgkQForvXbEpPzTVzgCg2vhW98aOD+ahLlpOB/b4AhYK fl8AoIJ8YmvBZ8OeWMBYXmn1b7LaIrTJ =Ghn6 -----END PGP SIGNATURE----- --udcq9yAoWb9A4FsZ-- From owner-zfs-devel@FreeBSD.ORG Thu Jul 22 23:13:27 2010 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 967BC1065678; Thu, 22 Jul 2010 23:13:27 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from mail.vx.sk (core.vx.sk [188.40.32.143]) by mx1.freebsd.org (Postfix) with ESMTP id 51A638FC1D; Thu, 22 Jul 2010 23:13:26 +0000 (UTC) Received: from core.vx.sk (localhost [127.0.0.1]) by mail.vx.sk (Postfix) with ESMTP id 37CF6113834; Fri, 23 Jul 2010 01:13:26 +0200 (CEST) X-Virus-Scanned: amavisd-new at mail.vx.sk Received: from mail.vx.sk ([127.0.0.1]) by core.vx.sk (mail.vx.sk [127.0.0.1]) (amavisd-new, port 10024) with LMTP id Dbq6dK4peSQy; Fri, 23 Jul 2010 01:13:24 +0200 (CEST) Received: from [10.9.8.1] (188-167-78-139.dynamic.chello.sk [188.167.78.139]) by mail.vx.sk (Postfix) with ESMTPSA id 0AA0F11382B; Fri, 23 Jul 2010 01:13:24 +0200 (CEST) Message-ID: <4C48D09C.8040704@FreeBSD.org> Date: Fri, 23 Jul 2010 01:13:32 +0200 From: Martin Matuska User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; sk; rv:1.8.1.23) Gecko/20090812 Lightning/0.9 Thunderbird/2.0.0.23 Mnenhy/0.7.5.0 MIME-Version: 1.0 To: zfs-devel@FreeBSD.org X-Enigmail-Version: 1.1.1 Content-Type: text/plain; charset=windows-1250 Content-Transfer-Encoding: 7bit Cc: Xin LI Subject: More bugfixes from OpenSolaris X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Jul 2010 23:13:27 -0000 Hi, I have the following changesets from OpenSolaris upstream that I would like to propose to import to head and after v15 merge to stable/8. The code resulting from these bugfixes has not been changed in OpenSolaris by today (matches pjd p4 tree). 1.) revision 9788:f660bc44f2e8 (possible memory corruption on zfs umount) zfs_znode_move() does not ensure valid file system pointer http://bugs.opensolaris.org/bugdatabase/view_bug.do?bug_id=6843700 http://mfsbsd.vx.sk/zfs/head-9788.patch 2.) revision 9909:aa280f585a3e (locking bugfix - affects zfs rollback) zfs related assertion failure: vp->v_filocks == 0L, file: ../../common/fs/vnode.c, line: 2344 http://bugs.opensolaris.org/bugdatabase/view_bug.do?bug_id=6790232 http://mfsbsd.vx.sk/zfs/head-9909.patch 3.) revision 9847:2f3ba86e857a (feature) snapshots should be considered as descendants via zfs allow -d http://bugs.opensolaris.org/bugdatabase/view_bug.do?bug_id=6809340 http://mfsbsd.vx.sk/zfs/head-9847.patch Patches are direct OpenSolaris imports and apply cleanly. From owner-zfs-devel@FreeBSD.ORG Fri Jul 23 07:34:57 2010 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A25B31065678; Fri, 23 Jul 2010 07:34:57 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from mail.vx.sk (core.vx.sk [188.40.32.143]) by mx1.freebsd.org (Postfix) with ESMTP id 5C9748FC22; Fri, 23 Jul 2010 07:34:56 +0000 (UTC) Received: from core.vx.sk (localhost [127.0.0.1]) by mail.vx.sk (Postfix) with ESMTP id 28BD211236B; Fri, 23 Jul 2010 09:34:56 +0200 (CEST) X-Virus-Scanned: amavisd-new at mail.vx.sk Received: from mail.vx.sk ([127.0.0.1]) by core.vx.sk (mail.vx.sk [127.0.0.1]) (amavisd-new, port 10024) with LMTP id 2GapBKHA88r6; Fri, 23 Jul 2010 09:34:53 +0200 (CEST) Received: from [10.9.8.1] (188-167-78-139.dynamic.chello.sk [188.167.78.139]) by mail.vx.sk (Postfix) with ESMTPSA id 580AF1122F7; Fri, 23 Jul 2010 09:34:53 +0200 (CEST) Message-ID: <4C494624.5000208@FreeBSD.org> Date: Fri, 23 Jul 2010 09:35:00 +0200 From: Martin Matuska User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; sk; rv:1.8.1.23) Gecko/20090812 Lightning/0.9 Thunderbird/2.0.0.23 Mnenhy/0.7.5.0 MIME-Version: 1.0 To: Pawel Jakub Dawidek , Xin LI X-Enigmail-Version: 1.1.1 Content-Type: text/plain; charset=windows-1250 Content-Transfer-Encoding: 7bit Cc: zfs-devel@FreeBSD.org Subject: ZFS v15 slowness X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 Jul 2010 07:34:57 -0000 Hi Pawel and Xin, I have a user report and I can confirm myself of ZFS v15 acting slower than before the import on several systems. I would therefore like to propose the combined stat() and rrwlock speedup patch as this has noticeable impact on this issue in my tests and the overall impression of system responsiveness. http://mfsbsd.vx.sk/zfs/head-9981-10143-10232-10250-10269.patch The patch code remained in OpenSolaris like that until today and it was part of the original zfs v16 patch I posted so it has already been tested by the users. A small part of the 10143 patch is not imported, see notes below. The patch covers the following opensolaris revisions and bug IDs: 9981:b4907297e740 ----------------- stat() performance on files on zfs should be improved http://bugs.opensolaris.org/bugdatabase/view_bug.do?bug_id=6775100 rrwlock is overly protective of its counters http://bugs.opensolaris.org/bugdatabase/view_bug.do?bug_id=6827779 10143:d2d432dfe597 (small part not included, see below) ----------------- memory leaks found at: zfs_acl_alloc/zfs_acl_node_alloc http://bugs.opensolaris.org/bugdatabase/view_bug.do?bug_id=6857433 truncate() on zfsroot succeeds when file has a component of its path set without access permission http://bugs.opensolaris.org/bugdatabase/view_bug.do?bug_id=6860318 10232:f37b85f7e03e ----------------- zfs sometimes incorrectly giving search access to a dir http://bugs.opensolaris.org/bugdatabase/view_bug.do?bug_id=6865875 10250:b179ceb34b62 ----------------- zpool_upgrade_007_pos testcase panic'd with BAD TRAP: type=e (#pf Page fault) http://bugs.opensolaris.org/bugdatabase/view_bug.do?bug_id=6867395 10269:2788675568fd ----------------- zfs_rezget() can be hazardous when znode has a cached ACL http://bugs.opensolaris.org/bugdatabase/view_bug.do?bug_id=6868276 ---------------------------------- If the import of a small part of 10143 (http://mfsbsd.vx.sk/zfs/head-10143-addon.patch) is possible only after importing ABE: http://mfsbsd.vx.sk/zfs/head-9749-9866.patch 9749:105f407a2680 ----------------- Support for Access Based Enumeration http://bugs.opensolaris.org/bugdatabase/view_bug.do?bug_id=6802734 A test driver for some ZFS features would be beneficial (this is the yet unused zlook program) http://bugs.opensolaris.org/bugdatabase/view_bug.do?bug_id=6826760 inconsistent xattr readdir behavior with too-small buffer http://bugs.opensolaris.org/bugdatabase/view_bug.do?bug_id=6844861 9866:ddc5f1d8eb4e ----------------- zfs with rstchown=0 or file_chown_self privilege allows user to "take" ownership http://bugs.opensolaris.org/bugdatabase/view_bug.do?bug_id=6848431 From owner-zfs-devel@FreeBSD.ORG Fri Jul 23 09:18:26 2010 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9266C1065677; Fri, 23 Jul 2010 09:18:26 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (chello089077043238.chello.pl [89.77.43.238]) by mx1.freebsd.org (Postfix) with ESMTP id 2B6218FC15; Fri, 23 Jul 2010 09:18:18 +0000 (UTC) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id 3FAC645CDC; Fri, 23 Jul 2010 11:18:17 +0200 (CEST) Received: from localhost (pdawidek.wheel.pl [10.0.1.1]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id 527AD45C89; Fri, 23 Jul 2010 11:18:12 +0200 (CEST) Date: Fri, 23 Jul 2010 11:17:42 +0200 From: Pawel Jakub Dawidek To: Martin Matuska Message-ID: <20100723091742.GB1756@garage.freebsd.pl> References: <4C494624.5000208@FreeBSD.org> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="Y7xTucakfITjPcLV" Content-Disposition: inline In-Reply-To: <4C494624.5000208@FreeBSD.org> User-Agent: Mutt/1.4.2.3i X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 9.0-CURRENT amd64 X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-5.9 required=4.5 tests=ALL_TRUSTED,BAYES_00 autolearn=ham version=3.0.4 Cc: zfs-devel@FreeBSD.org, Xin LI Subject: Re: ZFS v15 slowness X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 Jul 2010 09:18:26 -0000 --Y7xTucakfITjPcLV Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, Jul 23, 2010 at 09:35:00AM +0200, Martin Matuska wrote: > Hi Pawel and Xin, >=20 > I have a user report and I can confirm myself of ZFS v15 acting slower > than before the import on several systems. > I would therefore like to propose the combined stat() and rrwlock > speedup patch as this has noticeable impact on this issue in my tests > and the overall impression of system responsiveness. Of course speed up is very welcome, but as I read it v15 is slower than v14 for some unknown reason. So of course getting more optimizations is desirable, but not really related to why v15 is slower than v14. Is my understanding correct? What kind of performance problems do you observe? --=20 Pawel Jakub Dawidek http://www.wheelsystems.com pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --Y7xTucakfITjPcLV Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (FreeBSD) iEYEARECAAYFAkxJXjUACgkQForvXbEpPzRezACgqdX6YdrTfKG5Fh3v4UqUsNem fl8AoJES/M5vxF2viDxzf7/TXlfc7EaJ =hOwW -----END PGP SIGNATURE----- --Y7xTucakfITjPcLV-- From owner-zfs-devel@FreeBSD.ORG Fri Jul 23 10:47:21 2010 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 08431106564A; Fri, 23 Jul 2010 10:47:21 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from mail.vx.sk (core.vx.sk [188.40.32.143]) by mx1.freebsd.org (Postfix) with ESMTP id B72AB8FC14; Fri, 23 Jul 2010 10:47:20 +0000 (UTC) Received: from core.vx.sk (localhost [127.0.0.1]) by mail.vx.sk (Postfix) with ESMTP id 54987114A71; Fri, 23 Jul 2010 12:47:19 +0200 (CEST) X-Virus-Scanned: amavisd-new at mail.vx.sk Received: from mail.vx.sk ([127.0.0.1]) by core.vx.sk (mail.vx.sk [127.0.0.1]) (amavisd-new, port 10024) with LMTP id 8g7wKn-0S20g; Fri, 23 Jul 2010 12:47:17 +0200 (CEST) Received: from [10.0.3.171] (188-167-71-217.dynamic.chello.sk [188.167.71.217]) by mail.vx.sk (Postfix) with ESMTPSA id 563F6114A66; Fri, 23 Jul 2010 12:47:17 +0200 (CEST) Message-ID: <4C497331.5040304@FreeBSD.org> Date: Fri, 23 Jul 2010 12:47:13 +0200 From: Martin Matuska User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; sk; rv:1.8.1.23) Gecko/20090812 Lightning/0.9 Thunderbird/2.0.0.23 Mnenhy/0.7.5.0 MIME-Version: 1.0 To: Pawel Jakub Dawidek References: <4C494624.5000208@FreeBSD.org> <20100723091742.GB1756@garage.freebsd.pl> In-Reply-To: <20100723091742.GB1756@garage.freebsd.pl> X-Enigmail-Version: 1.1.1 Content-Type: text/plain; charset=windows-1250 Content-Transfer-Encoding: 8bit Cc: zfs-devel@FreeBSD.org, Xin LI Subject: Re: ZFS v15 slowness X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 Jul 2010 10:47:21 -0000 To my experience, processes spend more time in zio (as of top). When there is a lot of FS activity the system responsiveness goes down (filesystem reacts slowly or gets stallish). A good example is my tinderbox system - when tinderbox does a lot of small ports (heavy ZFS activity, untarring of many files and compiling, searching for changed files, etc.) the system shows a stallish behaviour in disk reads and my shell reacts slowly. With the stat and rrwlock patch it responds slightly better as it did before v15. The question is, why. The affected system is core i7, amd64, 8GB ram, 2x750GB sata2 drives in a ZFS mirror. vm.kmem_size="6G" vfs.zfs.arc_max="4G" Anyway, I would vote for these patches, maybe including ABE (it doesn't hurt). Da 23. 7. 2010 11:17, Pawel Jakub Dawidek wrote / napsal(a): > On Fri, Jul 23, 2010 at 09:35:00AM +0200, Martin Matuska wrote: >> Hi Pawel and Xin, >> >> I have a user report and I can confirm myself of ZFS v15 acting slower >> than before the import on several systems. >> I would therefore like to propose the combined stat() and rrwlock >> speedup patch as this has noticeable impact on this issue in my tests >> and the overall impression of system responsiveness. > Of course speed up is very welcome, but as I read it v15 is slower than > v14 for some unknown reason. So of course getting more optimizations is > desirable, but not really related to why v15 is slower than v14. > Is my understanding correct? What kind of performance problems do you > observe? > From owner-zfs-devel@FreeBSD.ORG Fri Jul 23 14:01:18 2010 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 89B421065674; Fri, 23 Jul 2010 14:01:18 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (chello089077043238.chello.pl [89.77.43.238]) by mx1.freebsd.org (Postfix) with ESMTP id CC3B58FC16; Fri, 23 Jul 2010 14:01:17 +0000 (UTC) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id 042B245E87; Fri, 23 Jul 2010 16:01:16 +0200 (CEST) Received: from localhost (pdawidek.wheel.pl [10.0.1.1]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id E64BF45C9F; Fri, 23 Jul 2010 16:01:10 +0200 (CEST) Date: Fri, 23 Jul 2010 16:00:40 +0200 From: Pawel Jakub Dawidek To: Martin Matuska Message-ID: <20100723140040.GD1756@garage.freebsd.pl> References: <4C48D09C.8040704@FreeBSD.org> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="+JUInw4efm7IfTNU" Content-Disposition: inline In-Reply-To: <4C48D09C.8040704@FreeBSD.org> User-Agent: Mutt/1.4.2.3i X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 9.0-CURRENT amd64 X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-5.9 required=4.5 tests=ALL_TRUSTED,BAYES_00 autolearn=ham version=3.0.4 Cc: zfs-devel@FreeBSD.org, Xin LI Subject: Re: More bugfixes from OpenSolaris X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 Jul 2010 14:01:18 -0000 --+JUInw4efm7IfTNU Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, Jul 23, 2010 at 01:13:32AM +0200, Martin Matuska wrote: > Hi, I have the following changesets from OpenSolaris upstream that I > would like to propose to import to head and after v15 merge to stable/8. > The code resulting from these bugfixes has not been changed in > OpenSolaris by today (matches pjd p4 tree). >=20 > 1.) revision 9788:f660bc44f2e8 (possible memory corruption on zfs umount) > zfs_znode_move() does not ensure valid file system pointer > http://bugs.opensolaris.org/bugdatabase/view_bug.do?bug_id=3D6843700 > http://mfsbsd.vx.sk/zfs/head-9788.patch I believe that for us this is no-op, because zfs_znode_move() is unused and commented out. > 2.) revision 9909:aa280f585a3e (locking bugfix - affects zfs rollback) > zfs related assertion failure: vp->v_filocks =3D=3D 0L, file: > ../../common/fs/vnode.c, line: 2344 > http://bugs.opensolaris.org/bugdatabase/view_bug.do?bug_id=3D6790232 > http://mfsbsd.vx.sk/zfs/head-9909.patch This is also no-op, as we define both cleanlocks() and cleanshares() as 'do { } while (0)'. > 3.) revision 9847:2f3ba86e857a (feature) > snapshots should be considered as descendants via zfs allow -d > http://bugs.opensolaris.org/bugdatabase/view_bug.do?bug_id=3D6809340 > http://mfsbsd.vx.sk/zfs/head-9847.patch This looks fine. --=20 Pawel Jakub Dawidek http://www.wheelsystems.com pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --+JUInw4efm7IfTNU Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (FreeBSD) iEYEARECAAYFAkxJoIcACgkQForvXbEpPzQASgCglUNPo4nP3S5LC0RkBvz9IjuV it8An2JbSGC2IHIN0oIvks2hC2UFVY16 =PIqT -----END PGP SIGNATURE----- --+JUInw4efm7IfTNU-- From owner-zfs-devel@FreeBSD.ORG Sat Jul 24 21:11:45 2010 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 56E8C1065670; Sat, 24 Jul 2010 21:11:45 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (chello089077043238.chello.pl [89.77.43.238]) by mx1.freebsd.org (Postfix) with ESMTP id AD2DC8FC16; Sat, 24 Jul 2010 21:11:44 +0000 (UTC) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id 2FBD545E86; Sat, 24 Jul 2010 23:11:42 +0200 (CEST) Received: from localhost (chello089077043238.chello.pl [89.77.43.238]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id 0F28845685; Sat, 24 Jul 2010 23:11:36 +0200 (CEST) Date: Sat, 24 Jul 2010 23:11:05 +0200 From: Pawel Jakub Dawidek To: Martin Matuska Message-ID: <20100724211105.GC2123@garage.freebsd.pl> References: <4C494624.5000208@FreeBSD.org> <20100723091742.GB1756@garage.freebsd.pl> <4C497331.5040304@FreeBSD.org> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="kfjH4zxOES6UT95V" Content-Disposition: inline In-Reply-To: <4C497331.5040304@FreeBSD.org> User-Agent: Mutt/1.4.2.3i X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 9.0-CURRENT amd64 X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-0.6 required=4.5 tests=BAYES_00,RCVD_IN_SORBS_DUL autolearn=no version=3.0.4 Cc: zfs-devel@FreeBSD.org, Xin LI Subject: Re: ZFS v15 slowness X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 24 Jul 2010 21:11:45 -0000 --kfjH4zxOES6UT95V Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, Jul 23, 2010 at 12:47:13PM +0200, Martin Matuska wrote: > To my experience, processes spend more time in zio (as of top). >=20 > When there is a lot of FS activity the system responsiveness goes down > (filesystem reacts slowly or gets stallish). > A good example is my tinderbox system - when tinderbox does a lot of > small ports (heavy ZFS activity, untarring of many files and compiling, > searching for changed files, etc.) the system shows a stallish behaviour > in disk reads and my shell reacts slowly. [...] This looks like priority problem. What kernel threads consume most CPU time when if feel slowdown? Do you have compression enabled on loaded datasets? I'd try lowering priority of the guilty thread and see what happen. > [...] With the stat and rrwlock > patch it responds slightly better as it did before v15. The question is, > why. >=20 > The affected system is core i7, amd64, 8GB ram, 2x750GB sata2 drives in > a ZFS mirror. > vm.kmem_size=3D"6G" > vfs.zfs.arc_max=3D"4G" >=20 > Anyway, I would vote for these patches, maybe including ABE (it doesn't > hurt). I'd prefer to address performance problem with v15 first and then merge stat(2) speed up changes, especially that with stat(2) patch it is harder to reproduce. --=20 Pawel Jakub Dawidek http://www.wheelsystems.com pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --kfjH4zxOES6UT95V Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (FreeBSD) iEYEARECAAYFAkxLVugACgkQForvXbEpPzQ8wQCfdDTHW8WD+uqFexkvilWJfD8r WooAnRvHiojoMMycowrRI3xsNnfT8qJO =BK3M -----END PGP SIGNATURE----- --kfjH4zxOES6UT95V-- From owner-zfs-devel@FreeBSD.ORG Sun Jul 25 18:31:43 2010 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id EBAEF106566B; Sun, 25 Jul 2010 18:31:43 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from mail.vx.sk (core.vx.sk [188.40.32.143]) by mx1.freebsd.org (Postfix) with ESMTP id D31B88FC0A; Sun, 25 Jul 2010 18:31:43 +0000 (UTC) Received: from core.vx.sk (localhost [127.0.0.1]) by mail.vx.sk (Postfix) with ESMTP id AD33C31136; Sun, 25 Jul 2010 20:31:42 +0200 (CEST) X-Virus-Scanned: amavisd-new at mail.vx.sk Received: from mail.vx.sk ([127.0.0.1]) by core.vx.sk (mail.vx.sk [127.0.0.1]) (amavisd-new, port 10024) with LMTP id 05gRRWIs616h; Sun, 25 Jul 2010 20:31:40 +0200 (CEST) Received: from [10.9.8.1] (188-167-78-139.dynamic.chello.sk [188.167.78.139]) by mail.vx.sk (Postfix) with ESMTPSA id 3AF083112D; Sun, 25 Jul 2010 20:31:40 +0200 (CEST) Message-ID: <4C4C830D.7040101@FreeBSD.org> Date: Sun, 25 Jul 2010 20:31:41 +0200 From: Martin Matuska User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; sk; rv:1.8.1.23) Gecko/20090812 Lightning/0.9 Thunderbird/2.0.0.23 Mnenhy/0.7.5.0 MIME-Version: 1.0 To: Pawel Jakub Dawidek References: <4C494624.5000208@FreeBSD.org> <20100723091742.GB1756@garage.freebsd.pl> <4C497331.5040304@FreeBSD.org> <20100724211105.GC2123@garage.freebsd.pl> In-Reply-To: <20100724211105.GC2123@garage.freebsd.pl> X-Enigmail-Version: 1.1.1 Content-Type: text/plain; charset=windows-1250 Content-Transfer-Encoding: 8bit Cc: zfs-devel@FreeBSD.org, Xin LI Subject: Re: ZFS v15 slowness X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 25 Jul 2010 18:31:44 -0000 Ok, after running for a longer time, I can confirm that the stat() speedup does not make the problem go away. It is priority-related. If I have the problematic tasks running with a nice >0, the system responds quickly. It looks like more that maybe rc.subr was changed in a way that my tinderbox daemon does not start with a lower priority anymore, so the behaviour will be probably the same on pre-v15 and this makes this issue probably not ZFS-v15-related or not even ZFS related at all. The system slowness is overall, not only ZFS - also reaction of the shell, too. Da 24. 7. 2010 23:11, Pawel Jakub Dawidek wrote / napsal(a): > On Fri, Jul 23, 2010 at 12:47:13PM +0200, Martin Matuska wrote: >> To my experience, processes spend more time in zio (as of top). >> >> When there is a lot of FS activity the system responsiveness goes down >> (filesystem reacts slowly or gets stallish). >> A good example is my tinderbox system - when tinderbox does a lot of >> small ports (heavy ZFS activity, untarring of many files and compiling, >> searching for changed files, etc.) the system shows a stallish behaviour >> in disk reads and my shell reacts slowly. [...] > This looks like priority problem. What kernel threads consume most CPU > time when if feel slowdown? Do you have compression enabled on loaded > datasets? I'd try lowering priority of the guilty thread and see what > happen. > >> [...] With the stat and rrwlock >> patch it responds slightly better as it did before v15. The question is, >> why. >> >> The affected system is core i7, amd64, 8GB ram, 2x750GB sata2 drives in >> a ZFS mirror. >> vm.kmem_size="6G" >> vfs.zfs.arc_max="4G" >> >> Anyway, I would vote for these patches, maybe including ABE (it doesn't >> hurt). > I'd prefer to address performance problem with v15 first and then merge > stat(2) speed up changes, especially that with stat(2) patch it is > harder to reproduce. > From owner-zfs-devel@FreeBSD.ORG Mon Jul 26 07:55:37 2010 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A3F23106566B for ; Mon, 26 Jul 2010 07:55:37 +0000 (UTC) (envelope-from hellosamirdesai@gmail.com) Received: from mail-vw0-f54.google.com (mail-vw0-f54.google.com [209.85.212.54]) by mx1.freebsd.org (Postfix) with ESMTP id 6279E8FC1D for ; Mon, 26 Jul 2010 07:55:37 +0000 (UTC) Received: by vws7 with SMTP id 7so2635712vws.13 for ; Mon, 26 Jul 2010 00:55:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:in-reply-to :references:date:message-id:subject:from:to:cc:content-type; bh=OU4OgEFnqx7kidB5RGHN6ZyFPTY32XAoxtd7tTOIvKk=; b=lbBD35p1I47ye2RgjuQug/DYthZEW4z0N35UEmiGBAOyxPbF/lzByq/mAfTqQklpfk VJxqLfzKEttomsj1+ZNadlnbdoIg1C7MfaNfipzandXrCmqOToAcUXfVIsic/kF2dv/R VaUx4YvBvtmnR0hkr+MCrZvUQvSqAcT5tsBZs= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; b=LNojSOayKn8tbqfqnbhdD4r3CPcyjCTRvZF8zEfi1PqtHoXSkeAM4fAudNaG6jVS07 Uojwrphdsohlx1CoGb5Zl7w5nLN7cVF11oRVNAzjdAvNw63J+lzYt8GV5Js1Kpc129iN eopsno2FILujCyRT0z0tJysuIToNfV3IRkmr0= MIME-Version: 1.0 Received: by 10.220.49.28 with SMTP id t28mr3902099vcf.93.1280129145272; Mon, 26 Jul 2010 00:25:45 -0700 (PDT) Received: by 10.220.185.201 with HTTP; Mon, 26 Jul 2010 00:25:45 -0700 (PDT) In-Reply-To: <20100722175145.GA2409@garage.freebsd.pl> References: <20100722175145.GA2409@garage.freebsd.pl> Date: Mon, 26 Jul 2010 12:55:45 +0530 Message-ID: From: Samir Desai To: Pawel Jakub Dawidek Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: zfs-devel@freebsd.org Subject: Re: zfs_cmd_t size issue. X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 26 Jul 2010 07:55:37 -0000 hi Pawel Shouldn't the MAX ioctl size be defined to #define IOCPARM_MAX ((1 << IOCPARM_SHIFT) - 1 ) Strictly speaking size 8192 can not be encoded. regards Samir On Thu, Jul 22, 2010 at 11:21 PM, Pawel Jakub Dawidek wrote: > Samir, could you take a look at this change: > > http://svn.freebsd.org/changeset/base/206051 > > I think it should fix ioctl structure size limti for good. > I'll investigate if the change can be merged to stable/8. > > -- > Pawel Jakub Dawidek http://www.wheelsystems.com > pjd@FreeBSD.org http://www.FreeBSD.org > FreeBSD committer Am I Evil? Yes, I Am! > From owner-zfs-devel@FreeBSD.ORG Wed Jul 28 17:14:22 2010 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 61C861065686 for ; Wed, 28 Jul 2010 17:14:22 +0000 (UTC) (envelope-from avg@icyb.net.ua) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id B35758FC1A for ; Wed, 28 Jul 2010 17:14:21 +0000 (UTC) Received: from odyssey.starpoint.kiev.ua (alpha-e.starpoint.kiev.ua [212.40.38.101]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id UAA08430 for ; Wed, 28 Jul 2010 20:02:32 +0300 (EEST) (envelope-from avg@icyb.net.ua) Message-ID: <4C5062A7.5030200@icyb.net.ua> Date: Wed, 28 Jul 2010 20:02:31 +0300 From: Andriy Gapon User-Agent: Thunderbird 2.0.0.24 (X11/20100517) MIME-Version: 1.0 To: zfs-devel@freebsd.org X-Enigmail-Version: 0.95.7 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Subject: zfs threads X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 28 Jul 2010 17:14:22 -0000 Not sure if this is something that happened recently or if it was always that way - I see a lot of zfs-something-named kernel threads running under 'kernel' process (pid 0). I thought the idea was to group them under that special kernel process 'zfskern'. Example: $ procstat -t 0 | fgrep zio 0 100079 kernel spa_zio_null_iss 0 68 sleep - 0 100080 kernel spa_zio_null_int 0 68 sleep - 0 100081 kernel spa_zio_read_iss 1 68 sleep - 0 100082 kernel spa_zio_read_iss 1 68 sleep - 0 100083 kernel spa_zio_read_iss 1 68 sleep - 0 100084 kernel spa_zio_read_iss 0 68 sleep - 0 100085 kernel spa_zio_read_iss 0 68 sleep - 0 100086 kernel spa_zio_read_iss 1 68 sleep - 0 100087 kernel spa_zio_read_iss 0 68 sleep - 0 100088 kernel spa_zio_read_iss 1 68 sleep - 0 100089 kernel spa_zio_read_int 0 68 sleep - 0 100090 kernel spa_zio_write_is 1 68 sleep - 0 100091 kernel spa_zio_write_in 0 68 sleep - 0 100092 kernel spa_zio_write_in 1 68 sleep - 0 100093 kernel spa_zio_write_in 1 68 sleep - 0 100094 kernel spa_zio_write_in 0 68 sleep - 0 100095 kernel spa_zio_write_in 1 68 sleep - 0 100096 kernel spa_zio_write_in 0 68 sleep - 0 100097 kernel spa_zio_write_in 0 68 sleep - 0 100098 kernel spa_zio_write_in 0 68 sleep - 0 100099 kernel spa_zio_free_iss 1 68 sleep - 0 100100 kernel spa_zio_free_int 1 68 sleep - 0 100101 kernel spa_zio_claim_is 1 68 sleep - 0 100102 kernel spa_zio_claim_in 1 68 sleep - 0 100103 kernel spa_zio_ioctl_is 1 68 sleep - 0 100104 kernel spa_zio_ioctl_in 1 68 sleep - -- Andriy Gapon From owner-zfs-devel@FreeBSD.ORG Wed Jul 28 21:00:39 2010 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6762C1065673 for ; Wed, 28 Jul 2010 21:00:39 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (chello089077043238.chello.pl [89.77.43.238]) by mx1.freebsd.org (Postfix) with ESMTP id 00C6E8FC14 for ; Wed, 28 Jul 2010 21:00:38 +0000 (UTC) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id ACD2145C98; Wed, 28 Jul 2010 23:00:36 +0200 (CEST) Received: from localhost (chello089077043238.chello.pl [89.77.43.238]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id B547345C89; Wed, 28 Jul 2010 23:00:31 +0200 (CEST) Date: Wed, 28 Jul 2010 23:00:29 +0200 From: Pawel Jakub Dawidek To: Samir Desai Message-ID: <20100728210029.GA2390@garage.freebsd.pl> References: <20100722175145.GA2409@garage.freebsd.pl> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="ReaqsoxgOBHFXBhH" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.3i X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 9.0-CURRENT amd64 X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-0.6 required=4.5 tests=BAYES_00,RCVD_IN_SORBS_DUL autolearn=no version=3.0.4 Cc: zfs-devel@freebsd.org Subject: Re: zfs_cmd_t size issue. X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 28 Jul 2010 21:00:39 -0000 --ReaqsoxgOBHFXBhH Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Mon, Jul 26, 2010 at 12:55:45PM +0530, Samir Desai wrote: > hi Pawel >=20 > Shouldn't the MAX ioctl size be defined to >=20 > #define IOCPARM_MAX ((1 << IOCPARM_SHIFT) - 1 ) >=20 > Strictly speaking size 8192 can not be encoded. You are right. Actually, now that we can use all the bits, this check in sys/kern/sys_generic.c:ioctl() is pointless: size =3D IOCPARM_LEN(com); if ((size > IOCPARM_MAX) || [...] I think I'll just remove IOCPARM_MAX entirely. --=20 Pawel Jakub Dawidek http://www.wheelsystems.com pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --ReaqsoxgOBHFXBhH Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (FreeBSD) iEYEARECAAYFAkxQmmwACgkQForvXbEpPzTrvACg1n2O22iXM8Nm26Cm7yEkTTFx jWAAnRb2zwDj1P+o85+a/ovEF4jTZ0Zk =34n4 -----END PGP SIGNATURE----- --ReaqsoxgOBHFXBhH-- From owner-zfs-devel@FreeBSD.ORG Tue Aug 10 20:04:10 2010 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id DE3EE1065670 for ; Tue, 10 Aug 2010 20:04:10 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (chello089077043238.chello.pl [89.77.43.238]) by mx1.freebsd.org (Postfix) with ESMTP id D3BA98FC08 for ; Tue, 10 Aug 2010 20:04:09 +0000 (UTC) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id 79A4545C98; Tue, 10 Aug 2010 22:04:07 +0200 (CEST) Received: from localhost (chello089077043238.chello.pl [89.77.43.238]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id 64EAE4569A for ; Tue, 10 Aug 2010 22:04:00 +0200 (CEST) Date: Tue, 10 Aug 2010 22:03:47 +0200 From: Pawel Jakub Dawidek To: zfs-devel@FreeBSD.org Message-ID: <20100810200347.GB1766@garage.freebsd.pl> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="OwLcNYc0lM97+oe1" Content-Disposition: inline User-Agent: Mutt/1.4.2.3i X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 9.0-CURRENT amd64 X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-0.6 required=4.5 tests=BAYES_00,RCVD_IN_SORBS_DUL autolearn=no version=3.0.4 Cc: Subject: Updates on ZFS. X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Aug 2010 20:04:10 -0000 --OwLcNYc0lM97+oe1 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Hi. For the past weeks I was working on getting recent ZFS to work. This is what I accomplished: 1. Fixed (implemented) onexit functionality that is now used by zfs send/recv. This is based on device cloning in OpenSolaris, but I manage to avoid using device cloning on FreeBSD and ended up using cdevpriv, which is much less problematic. This was the main reason ZFS in my tree didn't compile for quite a while - I needed to find a way and time to implement it properly. 2. I manage to get ZVOLs to work again. Commit log for this change should explain everything: OpenSolaris switched to lazy creation of /dev/ entires for ZVOLs. It creates /dev/ entries on VOP_LOOKUP() or VOP_READDIR(). This of course can't work this way on FreeBSD with GEOM, so we need to create ZVOL providers where appropriate. I found the following cases: 1. Pool first open (pool is loaded based on zpool/cache configuration and is then opened for a first time on eg. zfs mount). 2. Pool import. It's not the same as 1. 3. ZVOL creation: zfs create -V . 4. Creation of ZVOL snapshot, this includes recursive snapshot creation. To make it work I had to fix LOR between the zfsdev_state_lock, the GEOM topology lock and the spa_namespace_lock. They are now always obtained in the following order: 1. zfsdev_state_lock 2. g_topology_lock 3. spa_namespace_lock Also, we can't use taskqueue to scan for VDEVs as this introduces deadlock (because there is no way to honour the order above). This also allows to simplify vdev_geom.c quite a bit as it is no longer a problem to taste ZVOL or ZVOL-based provider. Update /etc/rc.d/zvol as there are no longer volinit and volfini subcommands to zfs(8). 3. I integrated all the recent changes (including zfs diff and readonly pools functionality). We are at v28 now and in sync with today's OpenSolaris. 4. I fixed a handful of bugs. All in all, it compiles and at least basic functionality should work. --=20 Pawel Jakub Dawidek http://www.wheelsystems.com pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --OwLcNYc0lM97+oe1 Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (FreeBSD) iEYEARECAAYFAkxhsKMACgkQForvXbEpPzQIdACg3AiixNILSMND+/XEdHosX/KS gK0AoN11LX1t7RE681OlmWgqYwRd5qLU =yCTx -----END PGP SIGNATURE----- --OwLcNYc0lM97+oe1-- From owner-zfs-devel@FreeBSD.ORG Tue Aug 10 20:31:00 2010 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5B6B3106564A for ; Tue, 10 Aug 2010 20:31:00 +0000 (UTC) (envelope-from mattjeet@gmail.com) Received: from mail-ww0-f50.google.com (mail-ww0-f50.google.com [74.125.82.50]) by mx1.freebsd.org (Postfix) with ESMTP id E29FD8FC1F for ; Tue, 10 Aug 2010 20:30:59 +0000 (UTC) Received: by wwb13 with SMTP id 13so1807201wwb.31 for ; Tue, 10 Aug 2010 13:30:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:sender:reply-to:received :in-reply-to:references:date:x-google-sender-auth:message-id:subject :from:to:cc:content-type:content-transfer-encoding; bh=NJxjxbGrrvRprmQFDvHO6ldOg9tXJ8ew8KKvkyyxJ7w=; b=EG/oi0Uh9WSrbgblkgRKG5g3gTLzGCx2OdS6BVcagbNwY66Wm2nlPMV9dd9NH/sc1M SEKd1MwAZRFb0canO30Hm9GZbzph/ZT+di/17RQ/yzMKT7vLHbV4X8vdHz0hnnFyY8Pu l9QqGve2nELe9HVPXvQzwfMtdJTz1UHcYN5cc= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:reply-to:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; b=SONkdCOLmSkvOTP9p/IN5t22ctGtFNa7akOh30ks00LnC82YBGdxVvOtERxvBzv8bV gsaifB7La4SfE/KTci6scmn1hwxYBBZW6X751Hvir/lrGHBIcScw04OOtNlwcYzNuaov n5PjZay5bHEbD0Kfhi6ckoISklABQqRvBxJmU= MIME-Version: 1.0 Received: by 10.227.145.145 with SMTP id d17mr15773419wbv.148.1281470733163; Tue, 10 Aug 2010 13:05:33 -0700 (PDT) Sender: mattjeet@gmail.com Received: by 10.227.136.21 with HTTP; Tue, 10 Aug 2010 13:05:33 -0700 (PDT) In-Reply-To: <20100810200347.GB1766@garage.freebsd.pl> References: <20100810200347.GB1766@garage.freebsd.pl> Date: Tue, 10 Aug 2010 13:05:33 -0700 X-Google-Sender-Auth: GwcQFxPFdRX0I6EHYUzr7qgTujE Message-ID: From: Matt Olander To: Pawel Jakub Dawidek Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: zfs-devel@freebsd.org Subject: Re: Updates on ZFS. X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: matt@ixsystems.com List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Aug 2010 20:31:00 -0000 Wow, that's impressive, Pawel. Nice work! We'll get to work testing :D -matt On Tue, Aug 10, 2010 at 1:03 PM, Pawel Jakub Dawidek wrot= e: > Hi. > > For the past weeks I was working on getting recent ZFS to work. This is > what I accomplished: > > 1. Fixed (implemented) onexit functionality that is now used by zfs > =A0 send/recv. This is based on device cloning in OpenSolaris, but I > =A0 manage to avoid using device cloning on FreeBSD and ended up using > =A0 cdevpriv, which is much less problematic. > =A0 This was the main reason ZFS in my tree didn't compile for quite a > =A0 while - I needed to find a way and time to implement it properly. > > 2. I manage to get ZVOLs to work again. Commit log for this change > =A0 should explain everything: > > =A0 OpenSolaris switched to lazy creation of /dev/ entires for ZVOLs. > =A0 It creates /dev/ entries on VOP_LOOKUP() or VOP_READDIR(). > > =A0 This of course can't work this way on FreeBSD with GEOM, so we need > =A0 to create ZVOL providers where appropriate. I found the following > =A0 cases: > =A0 1. Pool first open (pool is loaded based on zpool/cache configuration > =A0 =A0 =A0and is then opened for a first time on eg. zfs mount). > =A0 2. Pool import. It's not the same as 1. > =A0 3. ZVOL creation: zfs create -V . > =A0 4. Creation of ZVOL snapshot, this includes recursive snapshot > =A0 =A0 =A0creation. > > =A0 To make it work I had to fix LOR between the zfsdev_state_lock, the > =A0 GEOM topology lock and the spa_namespace_lock. They are now always > =A0 obtained in the following order: > =A0 1. zfsdev_state_lock > =A0 2. g_topology_lock > =A0 3. spa_namespace_lock > =A0 Also, we can't use taskqueue to scan for VDEVs as this introduces > =A0 deadlock (because there is no way to honour the order above). > > =A0 This also allows to simplify vdev_geom.c quite a bit as it is no > =A0 longer a problem to taste ZVOL or ZVOL-based provider. > > =A0 Update /etc/rc.d/zvol as there are no longer volinit and volfini > =A0 subcommands to zfs(8). > > 3. I integrated all the recent changes (including zfs diff and readonly > =A0 pools functionality). We are at v28 now and in sync with today's > =A0 OpenSolaris. > > 4. I fixed a handful of bugs. > > All in all, it compiles and at least basic functionality should work. > > -- > Pawel Jakub Dawidek =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 http://ww= w.wheelsystems.com > pjd@FreeBSD.org =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 http:= //www.FreeBSD.org > FreeBSD committer =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 Am I Ev= il? Yes, I Am! > From owner-zfs-devel@FreeBSD.ORG Sat Aug 21 14:11:01 2010 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 226C3106567A; Sat, 21 Aug 2010 14:11:01 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from mail.vx.sk (core.vx.sk [IPv6:2a01:4f8:100:1043::2]) by mx1.freebsd.org (Postfix) with ESMTP id B1B4C8FC0A; Sat, 21 Aug 2010 14:11:00 +0000 (UTC) Received: from core.vx.sk (localhost [127.0.0.1]) by mail.vx.sk (Postfix) with ESMTP id E5CB09D6E2; Sat, 21 Aug 2010 16:10:59 +0200 (CEST) X-Virus-Scanned: amavisd-new at mail.vx.sk Received: from mail.vx.sk ([127.0.0.1]) by core.vx.sk (mail.vx.sk [127.0.0.1]) (amavisd-new, port 10024) with LMTP id 19QOMQcmJhi8; Sat, 21 Aug 2010 16:10:57 +0200 (CEST) Received: from [10.9.8.1] (188-167-78-139.dynamic.chello.sk [188.167.78.139]) by mail.vx.sk (Postfix) with ESMTPSA id BBC5D9D6D7; Sat, 21 Aug 2010 16:10:57 +0200 (CEST) Message-ID: <4C6FDE77.5070607@FreeBSD.org> Date: Sat, 21 Aug 2010 16:11:03 +0200 From: Martin Matuska User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; sk; rv:1.8.1.23) Gecko/20090812 Lightning/0.9 Thunderbird/2.0.0.23 Mnenhy/0.7.5.0 MIME-Version: 1.0 To: Pawel Jakub Dawidek , Xin LI X-Enigmail-Version: 1.1.1 Content-Type: text/plain; charset=windows-1250 Content-Transfer-Encoding: 7bit Cc: zfs-devel@FreeBSD.org Subject: Patch proposal: Speeding up ZFS writes X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 21 Aug 2010 14:11:01 -0000 Hi Pawel, Xin and members of zfs-devel@ ! Many of our users today are complaining about slow ZFS writes. One of the causes for these writes is the allocation method for new blocks used [1]. Solaris 10 and OpenSolaris up to November 2009 used the following scenario: - pool has more than 30% free space: use first fit method [2] - pool has less than 70% free space: use best fit method [2] This causes a major slowdown and, let's say, unpleasant "fragmentation" of the writes if we go below 30% of free space. OpenSolaris has changed this and the Oracle Storage Appliances also included the new code in Q1/2010 [2]. The source [2] states, that with this change they archieved a speedup of: "50% Improved OLTP Performance, 70% Reduced Variability, 200% Improvement on MS Exchange" So what can we do to improve this situation? a) on the very short term, as a workaround soulution [3], we could just make metaslab_df_free_pct from metaslab.c tunable so users can test lower settings (I personally tend more to b)) b) on the mid-term, I suggest this patch for head with MFC to stable/8 after some reasonable time (1-2 months): http://people.freebsd.org/~mm/patches/zfs/zfs_metaslab.patch c) on the long term, updating to current ZFS code (v28) that integrates this patch The patch in b) includes the following OpenSolaris onnv revisions: 10921 (very small part, metaslab.c) 11146 (main patch, applies almost cleanly) 11728 (fix for zdb.c) 12047 (improvement to metaslab.c) OpenSolaris Bug IDs: 6826241 Sync write IOPS drops dramatically during TXG sync 6869229 zfs should switch to shiny new metaslabs more frequently 6917066 zfs block picking can be improved 6918420 zdb -m has issues printing metaslab statistics References: [1] http://blogs.sun.com/bonwick/entry/zfs_block_allocation [2] http://sun.systemnews.com/articles/147/2/OpenStorage/22963 [3] http://blogs.everycity.co.uk/alasdair/2010/07/zfs-runs-really-slowly-when-free-disk-usage-goes-above-80/ From owner-zfs-devel@FreeBSD.ORG Sun Aug 22 22:05:40 2010 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 49D001065670 for ; Sun, 22 Aug 2010 22:05:40 +0000 (UTC) (envelope-from avg@freebsd.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id A480D8FC13 for ; Sun, 22 Aug 2010 22:05:39 +0000 (UTC) Received: from porto.topspin.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id AAA05762; Mon, 23 Aug 2010 00:46:35 +0300 (EEST) (envelope-from avg@freebsd.org) Received: from localhost.topspin.kiev.ua ([127.0.0.1]) by porto.topspin.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1OnIN9-000B06-02; Mon, 23 Aug 2010 00:46:35 +0300 Message-ID: <4C719AB9.9020006@freebsd.org> Date: Mon, 23 Aug 2010 00:46:33 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.2.8) Gecko/20100822 Lightning/1.0b2 Thunderbird/3.1.2 MIME-Version: 1.0 To: zfs-devel@freebsd.org, freebsd-hackers@freebsd.org X-Enigmail-Version: 1.1.2 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Cc: Subject: ZFS arc_reclaim_needed: better cooperation with pagedaemon X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 22 Aug 2010 22:05:40 -0000 I propose that the following code in arc_reclaim_needed (sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c) /* * If pages are needed or we're within 2048 pages * of needing to page need to reclaim */ if (vm_pages_needed || (vm_paging_target() > -2048)) be changed to if (vm_paging_needed()) Rationale. 1. Why not current checks. ARC sizing should cooperate with pagedaemon in freeing pages. If ARC starts shrinking "prematurely", before pagedaemon is waked up then no potentially eligible inactive pages will be recycled and no potentially eligible active pages will be inactive (subject to v_inactive_target). This would lead to ARC size going to its minimum value (which could hurt ZFS performance). Only after that there is a chance that pagedaemon would be waked up to do its cleaning. And conversely, if ARC doesn't shrink in time, then pagedaemon would have to recycle pages with data that could be needed again soon and that would lead to excessive swapping and disk I/O. vm_paging_target() is used only by pagedaemon internally, it effectively sets _upper_ limit on how many pages pagedaemon would free when it's activated. It is no indication of whether pagedaemon should be scanning/freeing pages. Thus check of vm_paging_target() leads to premature ARC shrinking. I believe that many people observe this behavior on sufficiently active systems (not dedicated file servers) with few GB of RAM (1-8). vm_pages_needed check is redundant, because this is a flag that is used to wake up pagedaemon. So when it is set vm_paging_needed() is true and vm_paging_target() is "way" above zero. And this flag is reset to zero when vm_page_count_min() becomes false, which corresponds to even fewer free pages than when vm_paging_needed() is true. 2. Why the new check. vm_paging_needed() is the (earliest) condition that wakes up pagedaemon (see vm_page_alloc). pagedaemon would first of all run vm_lowmem event for which ARC already has a handler and which would cause ARC size to shrink. It would seems like having vm_paging_needed() check would be redundant then. Almost - if memory pressure is significant, then vm_paging_needed() may stay true for a while and that would cause additional ARC reduction by arc_reclaim_thread. Final notes. I think that vm_paging_target() > -2048 check was modeled after the check in the original OpenSolaris code: freemem < lotsfree + needfree + extra The issue is that in my understanding OpenSolaris pagedaemon works differently from FreeBSD pagedaemon. OpenSolaris pagedaemon is activated when freemem (equivalent of our free + cache) falls down to a certain higher mark (lotsfree). Initially it scans pages at a slow rate. If freemem falls further the rate linearly increases until it reaches its maximum when freemem goes to or below certain lower mark. Our pagedaemon is activated when free + cache falls down to a value when vm_paging_needed() is true (see definition of this function). When it is activated it makes a scan pass though inactive and active pages setting a certain target for free+cache, but that target is "soft" and actually is an upper limit of how many pages could be freed during the pass. pagedaemon would make the second (or subsequent) pass only if free+cache falls to value that is even lower than the threshold in vm_paging_needed(), which means significant (severe even) memory pressure/shortage. So on sufficiently active system free+cache would typically oscillate between v_free_reserved+v_cache_min at the bottom and some semi-random values "near" v_free_target+v_cache_min at the tops. That's when excluding ARC from the picture. And about pictures :-) Behavior of free+cache with current arc_reclaim_needed code: http://people.freebsd.org/~avg/avail-mem-before.png and its behavior after the patch: http://people.freebsd.org/~avg/avail-mem-after.png The legends on the pictures are incorrect, sorry, my mastery of drraw is not good yet. Correct legends: "aqua" color - v_free_target+v_cache_min (vm_paging_target() == 0) "fuchsia" color - v_free_reserved+v_cache_min (vm_paging_needed() threshold) "lime" color - v_free_count+v_cache_count indeed :) Y axis - % of total page count. I think the graphs speak for themselves. -- Andriy Gapon From owner-zfs-devel@FreeBSD.ORG Mon Aug 23 00:14:26 2010 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 02FDF106564A; Mon, 23 Aug 2010 00:14:26 +0000 (UTC) (envelope-from artemb@gmail.com) Received: from mail-vw0-f54.google.com (mail-vw0-f54.google.com [209.85.212.54]) by mx1.freebsd.org (Postfix) with ESMTP id 82BE48FC08; Mon, 23 Aug 2010 00:14:25 +0000 (UTC) Received: by vws7 with SMTP id 7so5554391vws.13 for ; Sun, 22 Aug 2010 17:14:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:sender:received :in-reply-to:references:date:x-google-sender-auth:message-id:subject :from:to:cc:content-type:content-transfer-encoding; bh=r9FCry+Eg1eohNrqtsr3DynwTWe0jWjx7Vt0CEW3rtM=; b=tahhLFARIOcsPuGUqF2+WVbxIvOKEvNSYnueVaMLVf9SHe2ZahUuH9Ng77/VW7HCVF 6GEBNj/D1FbQLaTj8c0SmdXw7Tzw9LBonpnB7P0zFiw1HeitrtaAjK5SI5RyS4uyrZxD N8tRMl0hAhOwAgOhRwR6AKi6ZuKxU1cTsM0eE= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; b=jT/RK5jBCFGn6ftRc9YPLpbB2iGax8UlLmw8LxaQQ+DrY+SqvaM8KOOFM98lYmViqz ilQqKyNFVEH+V/4B3ywsW/HZp/lltHuGqGPRibGzhepiAOOIo0U++4A43V2vU9BZz94m /kXNdMv70WAsawhYAYQk09aG9rOKnXdeEAJ9A= MIME-Version: 1.0 Received: by 10.220.88.167 with SMTP id a39mr2792109vcm.73.1282521120809; Sun, 22 Aug 2010 16:52:00 -0700 (PDT) Sender: artemb@gmail.com Received: by 10.220.49.70 with HTTP; Sun, 22 Aug 2010 16:52:00 -0700 (PDT) In-Reply-To: <4C719AB9.9020006@freebsd.org> References: <4C719AB9.9020006@freebsd.org> Date: Sun, 22 Aug 2010 16:52:00 -0700 X-Google-Sender-Auth: 5-hwxrj3geu8D5AWbwn-u5kaWJ8 Message-ID: From: Artem Belevich To: Andriy Gapon Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-hackers@freebsd.org, zfs-devel@freebsd.org Subject: Re: ZFS arc_reclaim_needed: better cooperation with pagedaemon X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Aug 2010 00:14:26 -0000 Do you by any chance have a graph showing kstat.zfs.misc.arcstats.size behavior in addition to the stuff included on your graphs now? All I can tell from your graphs is that v_free_count+v_cache_count shifted a bit lower relative to v_free_target+v_cache_min. It would be interesting to see what effect your patch has on ARC itself, especially when ARC will start giving up memory and when does it stop shrinking. --Artem On Sun, Aug 22, 2010 at 2:46 PM, Andriy Gapon wrote: > > I propose that the following code in arc_reclaim_needed > (sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c) > /* > =A0* If pages are needed or we're within 2048 pages > =A0* of needing to page need to reclaim > =A0*/ > if (vm_pages_needed || (vm_paging_target() > -2048)) > > be changed to > > if (vm_paging_needed()) > > Rationale. > > 1. Why not current checks. > > ARC sizing should cooperate with pagedaemon in freeing pages. > If ARC starts shrinking "prematurely", before pagedaemon is waked up then= no > potentially eligible inactive pages will be recycled and no potentially e= ligible > active pages will be inactive (subject to v_inactive_target). > This would lead to ARC size going to its minimum value (which could hurt = ZFS > performance). =A0Only after that there is a chance that pagedaemon would = be waked > up to do its cleaning. > And conversely, if ARC doesn't shrink in time, then pagedaemon would have= to > recycle pages with data that could be needed again soon and that would le= ad to > excessive swapping and disk I/O. > > vm_paging_target() is used only by pagedaemon internally, it effectively = sets > _upper_ limit on how many pages pagedaemon would free when it's activated= . > It is no indication of whether pagedaemon should be scanning/freeing page= s. > Thus check of vm_paging_target() leads to premature ARC shrinking. > I believe that many people observe this behavior on sufficiently active s= ystems > (not dedicated file servers) with few GB of RAM (1-8). > > vm_pages_needed check is redundant, because this is a flag that is used t= o wake > up pagedaemon. =A0So when it is set vm_paging_needed() is true and > vm_paging_target() is "way" above zero. =A0And this flag is reset to zero= when > vm_page_count_min() becomes false, which corresponds to even fewer free p= ages > than when vm_paging_needed() is true. > > > 2. Why the new check. > > vm_paging_needed() is the (earliest) condition that wakes up pagedaemon (= see > vm_page_alloc). =A0pagedaemon would first of all run vm_lowmem event for = which ARC > already has a handler and which would cause ARC size to shrink. > It would seems like having vm_paging_needed() check would be redundant th= en. > Almost - if memory pressure is significant, then vm_paging_needed() may s= tay > true for a while and that would cause additional ARC reduction by > arc_reclaim_thread. > > > Final notes. > > I think that > vm_paging_target() > -2048 > check was modeled after the check in the original OpenSolaris code: > freemem < lotsfree + needfree + extra > > The issue is that in my understanding OpenSolaris pagedaemon works differ= ently > from FreeBSD pagedaemon. > > OpenSolaris pagedaemon is activated when freemem (equivalent of our free = + > cache) falls down to a certain higher mark (lotsfree). =A0Initially it sc= ans pages > at a slow rate. =A0If freemem falls further the rate linearly increases u= ntil it > reaches its maximum when freemem goes to or below certain lower mark. > > Our pagedaemon is activated when free + cache falls down to a value when > vm_paging_needed() is true (see definition of this function). =A0When it = is > activated it makes a scan pass though inactive and active pages setting a > certain target for free+cache, but that target is "soft" and actually is = an > upper limit of how many pages could be freed during the pass. pagedaemon = would > make the second (or subsequent) pass only if free+cache falls to value th= at is > even lower than the threshold in vm_paging_needed(), which means signific= ant > (severe even) memory pressure/shortage. > So on sufficiently active system free+cache would typically oscillate bet= ween > v_free_reserved+v_cache_min at the bottom and some semi-random values "ne= ar" > v_free_target+v_cache_min at the tops. =A0That's when excluding ARC from = the picture. > > And about pictures :-) > Behavior of free+cache with current arc_reclaim_needed code: > http://people.freebsd.org/~avg/avail-mem-before.png > and its behavior after the patch: > http://people.freebsd.org/~avg/avail-mem-after.png > > The legends on the pictures are incorrect, sorry, my mastery of drraw is = not > good yet. > Correct legends: > "aqua" color - v_free_target+v_cache_min (vm_paging_target() =3D=3D 0) > "fuchsia" color - v_free_reserved+v_cache_min (vm_paging_needed() thresho= ld) > "lime" color - v_free_count+v_cache_count indeed :) > Y axis - % of total page count. > > I think the graphs speak for themselves. > > -- > Andriy Gapon > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org= " > From owner-zfs-devel@FreeBSD.ORG Mon Aug 23 06:12:58 2010 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3F26E1065673; Mon, 23 Aug 2010 06:12:58 +0000 (UTC) (envelope-from avg@freebsd.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id CC7988FC17; Mon, 23 Aug 2010 06:12:54 +0000 (UTC) Received: from porto.topspin.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id JAA11269; Mon, 23 Aug 2010 09:12:51 +0300 (EEST) (envelope-from avg@freebsd.org) Received: from localhost.topspin.kiev.ua ([127.0.0.1]) by porto.topspin.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1OnQH5-000Dm3-56; Mon, 23 Aug 2010 09:12:51 +0300 Message-ID: <4C721161.40403@freebsd.org> Date: Mon, 23 Aug 2010 09:12:49 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.2.8) Gecko/20100822 Lightning/1.0b2 Thunderbird/3.1.2 MIME-Version: 1.0 To: Artem Belevich References: <4C719AB9.9020006@freebsd.org> In-Reply-To: X-Enigmail-Version: 1.1.2 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: freebsd-hackers@freebsd.org, zfs-devel@freebsd.org Subject: Re: ZFS arc_reclaim_needed: better cooperation with pagedaemon X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Aug 2010 06:12:58 -0000 on 23/08/2010 02:52 Artem Belevich said the following: > Do you by any chance have a graph showing kstat.zfs.misc.arcstats.size > behavior in addition to the stuff included on your graphs now? Yes, I do and not by a chance :-) > All I > can tell from your graphs is that v_free_count+v_cache_count shifted a > bit lower relative to v_free_target+v_cache_min. Don't belittle those graphs :-) Remember that the "fuchsia" line is when pagedaemon is woken up. > It would be > interesting to see what effect your patch has on ARC itself, > especially when ARC will start giving up memory and when does it stop > shrinking. In an extreme case it stops at arc_c_min as expected. An extreme case is when userland application(s) demand a lot of memory fast. Now the graphs: http://people.freebsd.org/~avg/arc1.png http://people.freebsd.org/~avg/arc2.png http://people.freebsd.org/~avg/pages.png http://people.freebsd.org/~avg/arc3.png What do you see? What do you think? -- Andriy Gapon From owner-zfs-devel@FreeBSD.ORG Mon Aug 23 07:28:09 2010 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0ACE51065694; Mon, 23 Aug 2010 07:28:09 +0000 (UTC) (envelope-from artemb@gmail.com) Received: from mail-vw0-f54.google.com (mail-vw0-f54.google.com [209.85.212.54]) by mx1.freebsd.org (Postfix) with ESMTP id 83C798FC18; Mon, 23 Aug 2010 07:28:08 +0000 (UTC) Received: by vws7 with SMTP id 7so5779518vws.13 for ; Mon, 23 Aug 2010 00:28:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:sender:received :in-reply-to:references:date:x-google-sender-auth:message-id:subject :from:to:cc:content-type:content-transfer-encoding; bh=Jgtv7TWfJwnpmhOm6b4jsBomOPI9yv3rowE3OKMkCZo=; b=pzN3epxiogxs+7ZvR6Wxm8GLOPWhd/jJ9CqqhTEL2UGvNErmfTms30kAKRJXCuX4y/ 1THI3jhqLTA1kB1Y9wD3/iBSFXPF2JmvVP3Ypz1tkQOWszqL1rEJl/P9Gc4oNkowneDn eIWevlPBz9syLluiCV8LL85jGVQs0O+eVQOjk= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; b=II9i4RHi031WiwmDntjWMTNywCUCIWVJ+XW2EdC+kYlk/H6eEKQOW6e338ONYbEGWl ukwRvxBAhcQuajAXAVgqrmMoKmW4kfiNqjdpjGg2TEmb45PrVa4EkZ+q9h0ti9RBLPpQ GpKzR4n8lJiyaV/huaLyg127B/GgnOKDzTvwE= MIME-Version: 1.0 Received: by 10.220.124.74 with SMTP id t10mr2850507vcr.179.1282548487609; Mon, 23 Aug 2010 00:28:07 -0700 (PDT) Sender: artemb@gmail.com Received: by 10.220.49.70 with HTTP; Mon, 23 Aug 2010 00:28:07 -0700 (PDT) In-Reply-To: <4C721161.40403@freebsd.org> References: <4C719AB9.9020006@freebsd.org> <4C721161.40403@freebsd.org> Date: Mon, 23 Aug 2010 00:28:07 -0700 X-Google-Sender-Auth: 8X7QKvbEAmQydQw8TDx4sxgnh9c Message-ID: From: Artem Belevich To: Andriy Gapon Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-hackers@freebsd.org, zfs-devel@freebsd.org Subject: Re: ZFS arc_reclaim_needed: better cooperation with pagedaemon X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Aug 2010 07:28:09 -0000 Ah! After re-reading your first email and I think I've finally got what you're saying -- with your change ARC would only start giving up memory when pagedaemon is awake. Presumably once it's awake it will also run through inactive list pushing some of it to cache. On the other hand existing code voluntarily frees up memory even before pagedaemon wakes up and does such a good job that pagedaemon practically never wakes up and thus does not get to clean inactive list. Can anyone test the patch on a system with mix of UFS/ZFS filesystems and see if the change mitigates or solves the issue with inactive memory excessively backpressuring ARC. Overall I think that your proposed change seems to make sense to me. If we could also deal with zone fragmentation issue you've written in another thread, that should bring ZFS even closer to being usable without shaman-style (the one with lots of muttering, swearing and dancing around) tuning. Actually, it may be worth trying your test with re-enabled UMA allocator for ARC. Now that pagedaemon will be running, it would also invoke UMA's low memory handlers and those should be able to give some memory back to the system. --Artem On Sun, Aug 22, 2010 at 11:12 PM, Andriy Gapon wrote: > on 23/08/2010 02:52 Artem Belevich said the following: >> Do you by any chance have a graph showing kstat.zfs.misc.arcstats.size >> behavior in addition to the stuff included on your graphs now? > > Yes, I do and not by a chance :-) > >> All I >> can tell from your graphs is that v_free_count+v_cache_count shifted a >> bit lower relative to v_free_target+v_cache_min. > > Don't belittle those graphs :-) > Remember that the "fuchsia" line is when pagedaemon is woken up. > >> It would be >> interesting to see what effect your patch has on ARC itself, >> especially when ARC will start giving up memory and when does it stop >> shrinking. > > In an extreme case it stops at arc_c_min as expected. =A0An extreme case = is when > userland application(s) demand a lot of memory fast. > > Now the graphs: > http://people.freebsd.org/~avg/arc1.png > http://people.freebsd.org/~avg/arc2.png > http://people.freebsd.org/~avg/pages.png > http://people.freebsd.org/~avg/arc3.png > > What do you see? =A0What do you think? > > -- > Andriy Gapon > From owner-zfs-devel@FreeBSD.ORG Mon Aug 23 07:32:02 2010 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id F3E1D10656AB; Mon, 23 Aug 2010 07:32:01 +0000 (UTC) (envelope-from avg@freebsd.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 16F978FC24; Mon, 23 Aug 2010 07:32:00 +0000 (UTC) Received: from porto.topspin.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id KAA12358; Mon, 23 Aug 2010 10:31:58 +0300 (EEST) (envelope-from avg@freebsd.org) Received: from localhost.topspin.kiev.ua ([127.0.0.1]) by porto.topspin.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1OnRVe-000DtM-7q; Mon, 23 Aug 2010 10:31:58 +0300 Message-ID: <4C7223E9.8070801@freebsd.org> Date: Mon, 23 Aug 2010 10:31:53 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.2.8) Gecko/20100822 Lightning/1.0b2 Thunderbird/3.1.2 MIME-Version: 1.0 To: Artem Belevich References: <4C719AB9.9020006@freebsd.org> <4C721161.40403@freebsd.org> In-Reply-To: X-Enigmail-Version: 1.1.2 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: freebsd-hackers@freebsd.org, zfs-devel@freebsd.org Subject: Re: ZFS arc_reclaim_needed: better cooperation with pagedaemon X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Aug 2010 07:32:02 -0000 on 23/08/2010 10:28 Artem Belevich said the following: > If we could also deal with zone fragmentation issue you've written in > another thread, that should bring ZFS even closer to being usable > without shaman-style (the one with lots of muttering, swearing and > dancing around) tuning. > > Actually, it may be worth trying your test with re-enabled UMA > allocator for ARC. Now that pagedaemon will be running, it would also > invoke UMA's low memory handlers and those should be able to give some > memory back to the system. I tried, but still no go (for my taste). The fragmentation is too high and very significant portion of memory is effectively lost to it. E.g. ARC may think that it uses only 1GB but another 1GB is used by free items in ZFS zones (of 4GB total). -- Andriy Gapon From owner-zfs-devel@FreeBSD.ORG Mon Aug 23 20:52:55 2010 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7242D10656A9; Mon, 23 Aug 2010 20:52:55 +0000 (UTC) (envelope-from jhellenthal@gmail.com) Received: from mail-ww0-f50.google.com (mail-ww0-f50.google.com [74.125.82.50]) by mx1.freebsd.org (Postfix) with ESMTP id CB4BE8FC0A; Mon, 23 Aug 2010 20:52:54 +0000 (UTC) Received: by wwf26 with SMTP id 26so1530016wwf.31 for ; Mon, 23 Aug 2010 13:52:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:sender:message-id:date:from :user-agent:mime-version:to:cc:subject:references:in-reply-to :x-enigmail-version:content-type:content-transfer-encoding; bh=FT7hMW6eazNziYd4vKtoM2YRJ8pwKkGhB7GK/+FgnpM=; b=MosJ5LqcETAT7bwEzn9CKjqe7pMqS4F8crts5rLP/XLM771PkDJOLOK0i6WK38TYBT Ci6UKJLhmNAqsxbvamRb0FB9hS9L8tpcjBf3v1f7NQghqjNOCnW00eYFK70dj/7vIbDA 8JLEYV0OLHEQEExHBNyO9vxf1piGPJoCZcGqA= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=sender:message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:x-enigmail-version:content-type :content-transfer-encoding; b=wGQKRGsmYpqyS4i+Q5wf+3qDobxP/pIgYMiNbNTL9m+FuUuLDj+gRsfqnqbwQoGAC2 DzzK5yAAheh6K4SFhzPyICxuDm2xtCQtZUnQ0+vkTFOZROWkxatLETRFn/7F7C7cNUCG gxJ/D5m8RirG9MddumVVHFh3vRsWWDnIloq+Y= Received: by 10.227.127.193 with SMTP id h1mr5000926wbs.139.1282596294803; Mon, 23 Aug 2010 13:44:54 -0700 (PDT) Received: from centel.dataix.local (adsl-99-190-84-182.dsl.klmzmi.sbcglobal.net [99.190.84.182]) by mx.google.com with ESMTPS id i14sm3860128wbe.0.2010.08.23.13.44.53 (version=SSLv3 cipher=RC4-MD5); Mon, 23 Aug 2010 13:44:54 -0700 (PDT) Sender: "J. Hellenthal" Message-ID: <4C72DDC3.1000006@DataIX.net> Date: Mon, 23 Aug 2010 16:44:51 -0400 From: jhell User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.9.2.8) Gecko/20100806 Lightning/1.0b1 Thunderbird MIME-Version: 1.0 To: Artem Belevich References: <4C719AB9.9020006@freebsd.org> <4C721161.40403@freebsd.org> <4C72DD1A.9070204@DataIX.net> In-Reply-To: <4C72DD1A.9070204@DataIX.net> X-Enigmail-Version: 1.1.2 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Mailman-Approved-At: Mon, 23 Aug 2010 21:12:04 +0000 Cc: freebsd-hackers@freebsd.org, zfs-devel@freebsd.org, Andriy Gapon Subject: Re: ZFS arc_reclaim_needed: better cooperation with pagedaemon X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Aug 2010 20:52:55 -0000 On 08/23/2010 16:42, jhell wrote: > On 08/23/2010 03:28, Artem Belevich wrote: >> Can anyone test the patch on a system with mix of UFS/ZFS filesystems >> and see if the change mitigates or solves the issue with inactive >> memory excessively backpressuring ARC. > > I have a system currently patched up to ZFSv15 and mm@'s metaslab patch > running that I can test this on. Throw me a patch and some specific > tests and I can have the results back to you in the next 2 days. > Forget the patch 1 line change I can hand type that in. As for specific tests, let me know... -- jhell,v From owner-zfs-devel@FreeBSD.ORG Mon Aug 23 21:41:34 2010 Return-Path: Delivered-To: zfs-devel@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 57EA81065670; Mon, 23 Aug 2010 21:41:34 +0000 (UTC) (envelope-from delphij@delphij.net) Received: from tarsier.geekcn.org (tarsier.geekcn.org [IPv6:2001:470:a803::1]) by mx1.freebsd.org (Postfix) with ESMTP id 00C578FC19; Mon, 23 Aug 2010 21:41:34 +0000 (UTC) Received: from mail.geekcn.org (tarsier.geekcn.org [211.166.10.233]) by tarsier.geekcn.org (Postfix) with ESMTP id 15D29A6047C; Tue, 24 Aug 2010 05:41:33 +0800 (CST) X-Virus-Scanned: amavisd-new at geekcn.org Received: from tarsier.geekcn.org ([211.166.10.233]) by mail.geekcn.org (mail.geekcn.org [211.166.10.233]) (amavisd-new, port 10024) with LMTP id UP1Bi1WzAqJZ; Tue, 24 Aug 2010 05:41:27 +0800 (CST) Received: from delta.delphij.net (drawbridge.ixsystems.com [206.40.55.65]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by tarsier.geekcn.org (Postfix) with ESMTPSA id B33C3A6045F; Tue, 24 Aug 2010 05:41:24 +0800 (CST) DomainKey-Signature: a=rsa-sha1; s=default; d=delphij.net; c=nofws; q=dns; h=message-id:date:from:reply-to:organization:user-agent: mime-version:to:cc:subject:references:in-reply-to: x-enigmail-version:openpgp:content-type:content-transfer-encoding; b=dUoIu/evJhPKs5ojRXkSDwzOq+GGHKrQv1Vixe7Uc/mlUjgabV7LCtbEI3etvw+1M Oqm0pa5rMXkZpyPWtX7CQ== Message-ID: <4C72EAFF.6060800@delphij.net> Date: Mon, 23 Aug 2010 14:41:19 -0700 From: Xin LI Organization: The Geek China Organization User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.1.11) Gecko/20100721 Thunderbird/3.0.6 ThunderBrowse/3.3.2 MIME-Version: 1.0 To: Martin Matuska References: <4C6FDE77.5070607@FreeBSD.org> In-Reply-To: <4C6FDE77.5070607@FreeBSD.org> X-Enigmail-Version: 1.0.1 OpenPGP: id=3FCA37C1; url=http://www.delphij.net/delphij.asc Content-Type: text/plain; charset=windows-1250 Content-Transfer-Encoding: 7bit Cc: Xin LI , zfs-devel@FreeBSD.ORG, Pawel Jakub Dawidek Subject: Re: Patch proposal: Speeding up ZFS writes X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: d@delphij.net List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Aug 2010 21:41:34 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 Hi, Martin, On 2010/08/21 07:11, Martin Matuska wrote: > b) on the mid-term, I suggest this patch for head with MFC to stable/8 > after some reasonable time (1-2 months): > http://people.freebsd.org/~mm/patches/zfs/zfs_metaslab.patch [...] > The patch in b) includes the following OpenSolaris onnv revisions: > 10921 (very small part, metaslab.c) > 11146 (main patch, applies almost cleanly) > 11728 (fix for zdb.c) > 12047 (improvement to metaslab.c) > > OpenSolaris Bug IDs: > 6826241 Sync write IOPS drops dramatically during TXG sync > 6869229 zfs should switch to shiny new metaslabs more frequently > 6917066 zfs block picking can be improved > 6918420 zdb -m has issues printing metaslab statistics I think we should integrate this. The only thing I feel confusing about the change is that the part imported from OpenSolaris onnv revision 10921 on metaslab.c in metaslab_activate(), which adds a space_map_load_wait(sm), does not seem to be really necessary (the later 11146 change just added a blank space there) as the corresponding space_map_load() change that removed the wait() call was not merged. Or, did I missed something here? Cheers, - -- Xin LI http://www.delphij.net/ FreeBSD - The Power to Serve! Live free or die -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.16 (FreeBSD) iQEcBAEBCAAGBQJMcur+AAoJEATO+BI/yjfBoQkH/18thCEm++1vNXrvWChRjLSF fDrd6yoU7JXjSHkM7t/rE+D0/TEbL2Na2Fu0K8Ex4+qU3OBy14AcAxU2a+c+BV0T mf3PK6ne8lv2NijQ6fhBQ6fnsHqCnGnbjNfSTmNTQq3PJzisBe4pajSqSxqXJDUU pxbXVzssoDQfTuaoGmUXVkqMY2Tkn0YmLJ9yMvZCq0xVSjHd8cLGy71gCh8RGFTs dLqItQiaLWIniO5iFAeJtx9b7SmqUU2pu/WpDDg9h2zpchn8b/hoTkpR6zonclke yrYAhp+pX29HxhUheIkqxyvZJxFQ/JUekaGMZisYGH3JUaU1Vb+4CGimH0u7dUc= =g+3i -----END PGP SIGNATURE----- From owner-zfs-devel@FreeBSD.ORG Mon Aug 23 21:05:15 2010 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id DD63A10656A8; Mon, 23 Aug 2010 21:05:14 +0000 (UTC) (envelope-from jhellenthal@gmail.com) Received: from mail-bw0-f54.google.com (mail-bw0-f54.google.com [209.85.214.54]) by mx1.freebsd.org (Postfix) with ESMTP id 34A6C8FC22; Mon, 23 Aug 2010 21:05:13 +0000 (UTC) Received: by bwz20 with SMTP id 20so502957bwz.13 for ; Mon, 23 Aug 2010 14:05:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:sender:message-id:date:from :user-agent:mime-version:to:cc:subject:references:in-reply-to :x-enigmail-version:content-type:content-transfer-encoding; bh=jHbvOGVv5XG6OltEtbCGp0MzipA3KSYzOjAW7p8hngk=; b=UjTgf9uVkYYeyJN1LXC/EwdHff6K7RkJVa7OsGrqGpgFg6rpt0VIRGM2avsmjoMom8 o8HtYV6gEn05d9lWZb/jvg9Wdh3WI2OKn1b7x74qnQrhUYHajU2mtaTRfMCOdONc+tjd tJLEGsDaf8WapZ0nqXNFNCgLQEtnHC8ygPvLc= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=sender:message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:x-enigmail-version:content-type :content-transfer-encoding; b=n84hGOdbZ2ERFOfaBBYPDNwUnBvyFBccJenXvVsU4201jjKR/2UtVaut3NLm09ogQE W6sricUb6UxRIUs+VOjbWWJNkB6eTu9iYjtyagNHvfp/KY4l2wAAqHNNjo8CeTf9M+y8 R/B5rzQqKfEM+6dXWfLW4C4ssvUBbbSR72ObM= Received: by 10.213.17.203 with SMTP id t11mr4067082eba.66.1282596126483; Mon, 23 Aug 2010 13:42:06 -0700 (PDT) Received: from centel.dataix.local (adsl-99-190-84-182.dsl.klmzmi.sbcglobal.net [99.190.84.182]) by mx.google.com with ESMTPS id v8sm11442799eeh.8.2010.08.23.13.42.04 (version=SSLv3 cipher=RC4-MD5); Mon, 23 Aug 2010 13:42:05 -0700 (PDT) Sender: "J. Hellenthal" Message-ID: <4C72DD1A.9070204@DataIX.net> Date: Mon, 23 Aug 2010 16:42:02 -0400 From: jhell User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.9.2.8) Gecko/20100806 Lightning/1.0b1 Thunderbird MIME-Version: 1.0 To: Artem Belevich References: <4C719AB9.9020006@freebsd.org> <4C721161.40403@freebsd.org> In-Reply-To: X-Enigmail-Version: 1.1.2 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Mailman-Approved-At: Mon, 23 Aug 2010 21:45:24 +0000 Cc: freebsd-hackers@freebsd.org, zfs-devel@freebsd.org, Andriy Gapon Subject: Re: ZFS arc_reclaim_needed: better cooperation with pagedaemon X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Aug 2010 21:05:15 -0000 On 08/23/2010 03:28, Artem Belevich wrote: > Can anyone test the patch on a system with mix of UFS/ZFS filesystems > and see if the change mitigates or solves the issue with inactive > memory excessively backpressuring ARC. I have a system currently patched up to ZFSv15 and mm@'s metaslab patch running that I can test this on. Throw me a patch and some specific tests and I can have the results back to you in the next 2 days. Regards, -- jhell,v From owner-zfs-devel@FreeBSD.ORG Tue Aug 24 02:11:01 2010 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8BB4C106566B; Tue, 24 Aug 2010 02:11:01 +0000 (UTC) (envelope-from artemb@gmail.com) Received: from mail-vw0-f54.google.com (mail-vw0-f54.google.com [209.85.212.54]) by mx1.freebsd.org (Postfix) with ESMTP id 067AE8FC0C; Tue, 24 Aug 2010 02:11:00 +0000 (UTC) Received: by vws7 with SMTP id 7so26409vws.13 for ; Mon, 23 Aug 2010 19:11:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:sender:received :in-reply-to:references:date:x-google-sender-auth:message-id:subject :from:to:cc:content-type:content-transfer-encoding; bh=TH0jI61hCMLXRzy4CdFHziS2srG+iybtfFG5uSHpIlQ=; b=VtMn8GDDH6kpofmw5rw1EUyy4YrH+NWdkYQ42N4Hq5BnEOxxeyjWcD3+AgWNSRqZPv h8JQ41aNCOBd8WJs8ZfInNFv/lxJ+D3aLaj2UOpfCz0OnRDnzFbx19zeNvNVvqv/W4lH ix4zKVJPY5RRC3vPEFRNDDuP/xz9HDrwR4pzQ= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; b=HU+fiejTt5yHKodzNqWts6SwDcIY7ZWZGgmsPQMXlBShBebey5wwKt2YTxF7mWg/TO /l4aH4Wyuiw+q+y3e//A27NEGp+6EFfcFH2ni2ksoR+LuEOC4jMT5v8iGW/mqs46CA8Z BlCyN2AaTKMH5AQEs2+mPO1AUD62M48wrwrxU= MIME-Version: 1.0 Received: by 10.220.85.196 with SMTP id p4mr3667193vcl.271.1282615859967; Mon, 23 Aug 2010 19:10:59 -0700 (PDT) Sender: artemb@gmail.com Received: by 10.220.49.70 with HTTP; Mon, 23 Aug 2010 19:10:59 -0700 (PDT) In-Reply-To: <4C72DDC3.1000006@DataIX.net> References: <4C719AB9.9020006@freebsd.org> <4C721161.40403@freebsd.org> <4C72DD1A.9070204@DataIX.net> <4C72DDC3.1000006@DataIX.net> Date: Mon, 23 Aug 2010 19:10:59 -0700 X-Google-Sender-Auth: 185f49DgvbAQMAeXOWU8l1Vx1WQ Message-ID: From: Artem Belevich To: jhell Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-hackers@freebsd.org, zfs-devel@freebsd.org, Andriy Gapon Subject: Re: ZFS arc_reclaim_needed: better cooperation with pagedaemon X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Aug 2010 02:11:01 -0000 Could you try following experiments before and after the patch while monitoring kstat.zfs.misc.arcstats.size and vm.stats.vm.v_inactive_count. First prepare the data. * You'll need some files totalling around the amount of physical memory on your box. Multiple copies of /usr/src should do the trick. * Place one copy on UFS filesystem and another on ZFS Experiment #1: * Prime ARC by tarring dataset on ZFS into /dev/null. * Now tar both datasets in parallel with output to /dev/null Previously you would end up with ARC size shrinking down to arc_min. What I hope to see after the patch is that inactive memory and ARC reach some sort of equilibrium with neither monopolizing all available memory. #Experiment #2: If equilibrium is reached, try running some application that would allocate and use about 1/2 of your physical memory. Something like that perl one-liner used to cause memory shortage, only a bit less drastic. perl -e '$x=3D"x"x1_000_000_000'; # this should allocate about 2GB. Tune the number to suit your system. Again, in the past ARC would be the one feeing up the memory. Let's see if inactive list gives up some, too. --Artem On Mon, Aug 23, 2010 at 1:44 PM, jhell wrote: > On 08/23/2010 16:42, jhell wrote: >> On 08/23/2010 03:28, Artem Belevich wrote: >>> Can anyone test the patch on a system with mix of UFS/ZFS filesystems >>> and see if the change mitigates or solves the issue with inactive >>> memory excessively backpressuring ARC. >> >> I have a system currently patched up to ZFSv15 and mm@'s metaslab patch >> running that I can test this on. Throw me a patch and some specific >> tests and I can have the results back to you in the next 2 days. >> > > Forget the patch 1 line change I can hand type that in. As for specific > tests, let me know... > > -- > > =A0jhell,v > From owner-zfs-devel@FreeBSD.ORG Tue Aug 24 02:28:24 2010 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B3DAC106566B; Tue, 24 Aug 2010 02:28:24 +0000 (UTC) (envelope-from jhellenthal@gmail.com) Received: from mail-qw0-f54.google.com (mail-qw0-f54.google.com [209.85.216.54]) by mx1.freebsd.org (Postfix) with ESMTP id 2A2038FC0C; Tue, 24 Aug 2010 02:28:23 +0000 (UTC) Received: by qwg5 with SMTP id 5so6357727qwg.13 for ; Mon, 23 Aug 2010 19:28:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:sender:message-id:date:from :user-agent:mime-version:to:cc:subject:references:in-reply-to :x-enigmail-version:content-type:content-transfer-encoding; bh=4fqi2BKYIFEmvU/VaYL9GXnmLV0JV5sO/hxD8y23fXs=; b=M2Qj8kXme19LMVKALA+bjWIEeYNPKCijeayda53m1IusGhUGVRQBvU8s3v4PumuTX+ 72+f6C+h8lxI7s/yqsPaI3+NTn30gMn0YWW9fH48uTnSzOEnRAOnEDxb+waQ9pOFV3f4 1nxIonIY6FZMW6PTu5hzj325/yyWVW6sIYuAo= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=sender:message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:x-enigmail-version:content-type :content-transfer-encoding; b=CDhllPG/5p5AXC7g4Ag5NBQmenfeMYIZEVUIt66q69RkzsL2d3sMYWeHscJv9u6x68 5f9djOhJMH0rySgHs5bEBShxAdCjQ5U5tnU4sK+qdHySfJUgHyrg1uFrUCvJTb0grPy7 hPMPTIj3/pWmqq12DFOojVtP0EV3sualPRk4A= Received: by 10.229.222.6 with SMTP id ie6mr4311550qcb.28.1282616903208; Mon, 23 Aug 2010 19:28:23 -0700 (PDT) Received: from centel.dataix.local (adsl-99-190-84-182.dsl.klmzmi.sbcglobal.net [99.190.84.182]) by mx.google.com with ESMTPS id t4sm7901199qcs.40.2010.08.23.19.28.21 (version=SSLv3 cipher=RC4-MD5); Mon, 23 Aug 2010 19:28:22 -0700 (PDT) Sender: "J. Hellenthal" Message-ID: <4C732E44.50702@DataIX.net> Date: Mon, 23 Aug 2010 22:28:20 -0400 From: jhell User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.9.2.8) Gecko/20100806 Lightning/1.0b1 Thunderbird MIME-Version: 1.0 To: Artem Belevich References: <4C719AB9.9020006@freebsd.org> <4C721161.40403@freebsd.org> <4C72DD1A.9070204@DataIX.net> <4C72DDC3.1000006@DataIX.net> In-Reply-To: X-Enigmail-Version: 1.1.2 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Mailman-Approved-At: Tue, 24 Aug 2010 02:39:54 +0000 Cc: freebsd-hackers@freebsd.org, zfs-devel@freebsd.org, Andriy Gapon Subject: Re: ZFS arc_reclaim_needed: better cooperation with pagedaemon X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Aug 2010 02:28:24 -0000 On 08/23/2010 22:10, Artem Belevich wrote: > First prepare the data. > * You'll need some files totalling around the amount of physical > memory on your box. Multiple copies of /usr/src should do the trick. > * Place one copy on UFS filesystem and another on ZFS > > Experiment #1: > * Prime ARC by tarring dataset on ZFS into /dev/null. > * Now tar both datasets in parallel with output to /dev/null > > Previously you would end up with ARC size shrinking down to arc_min. > What I hope to see after the patch is that inactive memory and ARC > reach some sort of equilibrium with neither monopolizing all available > memory. > > #Experiment #2: > If equilibrium is reached, try running some application that would > allocate and use about 1/2 of your physical memory. > Something like that perl one-liner used to cause memory shortage, only > a bit less drastic. > perl -e '$x="x"x1_000_000_000'; # this should allocate about 2GB. > Tune the number to suit your system. > > Again, in the past ARC would be the one feeing up the memory. Let's > see if inactive list gives up some, too. Alright I should be able to have something together with the output of this by the 25th with my current workload being the ultimate determining factor. PBS, Regards, -- jhell,v From owner-zfs-devel@FreeBSD.ORG Tue Aug 31 22:19:01 2010 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0A7D510656A6 for ; Tue, 31 Aug 2010 22:19:01 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (60.wheelsystems.com [83.12.187.60]) by mx1.freebsd.org (Postfix) with ESMTP id 728908FC1A for ; Tue, 31 Aug 2010 22:19:00 +0000 (UTC) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id 4006745DD8; Tue, 31 Aug 2010 23:59:30 +0200 (CEST) Received: from localhost (chello089077043238.chello.pl [89.77.43.238]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id 61EF845B36; Tue, 31 Aug 2010 23:59:24 +0200 (CEST) Date: Tue, 31 Aug 2010 23:59:15 +0200 From: Pawel Jakub Dawidek To: freebsd-fs@FreeBSD.org Message-ID: <20100831215915.GE1932@garage.freebsd.pl> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="RpqchZ26BWispMcB" Content-Disposition: inline User-Agent: Mutt/1.4.2.3i X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 9.0-CURRENT amd64 X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-0.6 required=4.5 tests=BAYES_00,RCVD_IN_SORBS_DUL autolearn=no version=3.0.4 X-Mailman-Approved-At: Tue, 31 Aug 2010 22:21:22 +0000 Cc: freebsd-current@FreeBSD.org Subject: ZFS v28 is ready for wider testing. X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Aug 2010 22:19:01 -0000 --RpqchZ26BWispMcB Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Hello. I'd like to give you ZFS v28 for testing. If you are neither brave nor mad, you can stop here. The patchset is very experimental. It can eat your cookie and hurt your teddy bear, so be warned. Don't try it for anything except testing. This patchset is also a message we, as the FreeBSD project, would like to send to our users: Eventhough OpenSolaris is dead, the ZFS file system is going to stay in FreeBSD. At this point we have quite a few developers involved in ZFS on FreeBSD as well as serveral companies. We are also looking forward to work with IllumOS. So, what this new ZFS brings? - Data deduplication. Read more here: http://blogs.sun.com/bonwick/entry/zfs_dedup - Triple parity RAIDZ (RAIDZ3). Read more here: http://dtrace.org/blogs/ahl/2009/07/21/triple-parity-raid-z/ - zfs diff. Read more here: http://arc.opensolaris.org/caselog/PSARC/2010/105/20100328_tim.haley - zpool split. Read more here: http://arc.opensolaris.org/caselog/PSARC/2009/511/20090924_mark.musante - Snapshot holds. Read more here: http://arc.opensolaris.org/caselog/PSARC/2009/297/20090511_chris.kirby - zpool import -F. Allows to rewind corrupted pool to earlier transaction group. - Possibility to import pool in read-only mode. And much, much more, including plenty of preformance improvements and bug fixes. So test whatever you can and report back. Look for regressions, strange behaviour, missing features, deadlocks, livelocks, preformance degradation, etc. The boot code is not updated at all, so booting off of ZFS doesn't currently work. The patch is against today's FreeBSD HEAD. The patch enables (in sys/modules/zfs/Makefile) ZFS internal debugging, please don't turn it off. Also, compile your kernel with the following options: options KDB options DDB options INVARIANTS options INVARIANT_SUPPORT options WITNESS options WITNESS_SKIPSPIN options DEBUG_LOCKS options DEBUG_VFS_LOCKS Ignore all the LOR (Lock Order Reversal) reports from WITNESS. There will be plenty of those, and you'll desperately want to report them, but please don't. The best way to report a problem is to answer to this e-mail with as short as possible procedure of how to reproduce it and debugging info. I'd prefer textdump if possible. Below you can find quick procedure how to setup textdumps: Choose spare/swap disk/partition in your system, let's say it is /dev/ad0s1b. Add the following line to /etc/fstab: /dev/ad0s1b none swap sw 0 0 Add the following line to /etc/rc.conf: ddb_enable=3D"YES" Run the following commands: # /etc/rc.d/swap1 start # /etc/rc.d/dumpon start # /etc/rc.d/ddb start This will setup swap, mark it as dump device and setup some DDB scripts. Or you can just reboot. Now when your system panic or deadlock, enter DDB and call the following command: ddb> run kdb.enter.panic It will execute all the commands I need, dump them in text format to your swap device and reboot machine. After the reboot, you should find textdump.tar.0 file in /var/crash/ directory. This is the debug info I need. End of textdumps procedure. Ok, now that I know you read everything carefully, here is the patch: http://people.freebsd.org/~pjd/patches/zfs_20100831.patch.bz2 Good luck! >:> --=20 Pawel Jakub Dawidek http://www.wheelsystems.com pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --RpqchZ26BWispMcB Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (FreeBSD) iEYEARECAAYFAkx9ezMACgkQForvXbEpPzQ+ZwCg6EtfJjx6X1nJaj5uTkEM2fwx HkoAoJ5/L97SHbIHcyLrOqmH/t4oBmFi =ePEI -----END PGP SIGNATURE----- --RpqchZ26BWispMcB-- From owner-zfs-devel@FreeBSD.ORG Sun Sep 12 13:46:20 2010 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 60F331065672; Sun, 12 Sep 2010 13:46:20 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from mail.vx.sk (mail.vx.sk [IPv6:2a01:4f8:100:1043::3]) by mx1.freebsd.org (Postfix) with ESMTP id 8653E8FC08; Sun, 12 Sep 2010 13:46:19 +0000 (UTC) Received: from core.vx.sk (localhost [127.0.0.1]) by mail.vx.sk (Postfix) with ESMTP id AC12EEDAFA; Sun, 12 Sep 2010 15:46:18 +0200 (CEST) X-Virus-Scanned: amavisd-new at mail.vx.sk Received: from mail.vx.sk ([127.0.0.1]) by core.vx.sk (mail.vx.sk [127.0.0.1]) (amavisd-new, port 10024) with LMTP id YtO9QzL6oeBD; Sun, 12 Sep 2010 15:46:03 +0200 (CEST) Received: from [127.0.0.1] (ip-91.191.85.28.o2inet.sk [91.191.85.28]) by mail.vx.sk (Postfix) with ESMTPSA id 624EEEDAE0; Sun, 12 Sep 2010 15:46:00 +0200 (CEST) Message-ID: <4C8CD994.6060103@FreeBSD.org> Date: Sun, 12 Sep 2010 15:45:56 +0200 From: Martin Matuska User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; sk; rv:1.8.1.23) Gecko/20090812 Lightning/0.9 Thunderbird/2.0.0.23 Mnenhy/0.7.5.0 MIME-Version: 1.0 To: Andriy Gapon References: <4C719AB9.9020006@freebsd.org> In-Reply-To: <4C719AB9.9020006@freebsd.org> X-Enigmail-Version: 1.1.1 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: zfs-devel@freebsd.org Subject: Re: ZFS arc_reclaim_needed: better cooperation with pagedaemon X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 12 Sep 2010 13:46:20 -0000 I am running a really busy system with 48GB of RAM, equipped with 300GB of SSD L2 cache as well and the I have recently observed the problem. The system runs static fileserving from ZFS via sendfile() and NFS export. My vfs.zfs.arc_max is set to 40GB, vfs.zfs.arc_min is auto-set to 5GB (= metalimit / 2, metalimit = arc_max / 4) NFS generates inactive memory that is not in use and leads to a reduction of ARC up to vfs.zfs.arc_min (not below). But I actually don't care about the cache of NFS and I really need the ARC. So if I understand this correctly, this might help me - but it would lead to some kind of balance of incative memory vs ARC. Anyway, best help for me would probably be setting vfs.zfs.arc_min to a high value (20GB or more) so that the system tries to stay at this level. mm Dňa 22. 8. 2010 23:46, Andriy Gapon wrote / napísal(a): > I propose that the following code in arc_reclaim_needed > (sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c) > /* > * If pages are needed or we're within 2048 pages > * of needing to page need to reclaim > */ > if (vm_pages_needed || (vm_paging_target() > -2048)) > > be changed to > > if (vm_paging_needed()) > > Rationale. > > 1. Why not current checks. > > ARC sizing should cooperate with pagedaemon in freeing pages. > If ARC starts shrinking "prematurely", before pagedaemon is waked up then no > potentially eligible inactive pages will be recycled and no potentially eligible > active pages will be inactive (subject to v_inactive_target). > This would lead to ARC size going to its minimum value (which could hurt ZFS > performance). Only after that there is a chance that pagedaemon would be waked > up to do its cleaning. > And conversely, if ARC doesn't shrink in time, then pagedaemon would have to > recycle pages with data that could be needed again soon and that would lead to > excessive swapping and disk I/O. > > vm_paging_target() is used only by pagedaemon internally, it effectively sets > _upper_ limit on how many pages pagedaemon would free when it's activated. > It is no indication of whether pagedaemon should be scanning/freeing pages. > Thus check of vm_paging_target() leads to premature ARC shrinking. > I believe that many people observe this behavior on sufficiently active systems > (not dedicated file servers) with few GB of RAM (1-8). > > vm_pages_needed check is redundant, because this is a flag that is used to wake > up pagedaemon. So when it is set vm_paging_needed() is true and > vm_paging_target() is "way" above zero. And this flag is reset to zero when > vm_page_count_min() becomes false, which corresponds to even fewer free pages > than when vm_paging_needed() is true. > > > 2. Why the new check. > > vm_paging_needed() is the (earliest) condition that wakes up pagedaemon (see > vm_page_alloc). pagedaemon would first of all run vm_lowmem event for which ARC > already has a handler and which would cause ARC size to shrink. > It would seems like having vm_paging_needed() check would be redundant then. > Almost - if memory pressure is significant, then vm_paging_needed() may stay > true for a while and that would cause additional ARC reduction by > arc_reclaim_thread. > > > Final notes. > > I think that > vm_paging_target() > -2048 > check was modeled after the check in the original OpenSolaris code: > freemem < lotsfree + needfree + extra > > The issue is that in my understanding OpenSolaris pagedaemon works differently > from FreeBSD pagedaemon. > > OpenSolaris pagedaemon is activated when freemem (equivalent of our free + > cache) falls down to a certain higher mark (lotsfree). Initially it scans pages > at a slow rate. If freemem falls further the rate linearly increases until it > reaches its maximum when freemem goes to or below certain lower mark. > > Our pagedaemon is activated when free + cache falls down to a value when > vm_paging_needed() is true (see definition of this function). When it is > activated it makes a scan pass though inactive and active pages setting a > certain target for free+cache, but that target is "soft" and actually is an > upper limit of how many pages could be freed during the pass. pagedaemon would > make the second (or subsequent) pass only if free+cache falls to value that is > even lower than the threshold in vm_paging_needed(), which means significant > (severe even) memory pressure/shortage. > So on sufficiently active system free+cache would typically oscillate between > v_free_reserved+v_cache_min at the bottom and some semi-random values "near" > v_free_target+v_cache_min at the tops. That's when excluding ARC from the picture. > > And about pictures :-) > Behavior of free+cache with current arc_reclaim_needed code: > http://people.freebsd.org/~avg/avail-mem-before.png > and its behavior after the patch: > http://people.freebsd.org/~avg/avail-mem-after.png > > The legends on the pictures are incorrect, sorry, my mastery of drraw is not > good yet. > Correct legends: > "aqua" color - v_free_target+v_cache_min (vm_paging_target() == 0) > "fuchsia" color - v_free_reserved+v_cache_min (vm_paging_needed() threshold) > "lime" color - v_free_count+v_cache_count indeed :) > Y axis - % of total page count. > > I think the graphs speak for themselves. > From owner-zfs-devel@FreeBSD.ORG Sun Sep 12 15:25:36 2010 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 858321065673 for ; Sun, 12 Sep 2010 15:25:36 +0000 (UTC) (envelope-from avg@freebsd.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 9477D8FC08 for ; Sun, 12 Sep 2010 15:25:35 +0000 (UTC) Received: from porto.topspin.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id SAA28288; Sun, 12 Sep 2010 18:25:34 +0300 (EEST) (envelope-from avg@freebsd.org) Received: from localhost.topspin.kiev.ua ([127.0.0.1]) by porto.topspin.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1OuoQv-000FyL-TR; Sun, 12 Sep 2010 18:25:33 +0300 Message-ID: <4C8CF0ED.9030004@freebsd.org> Date: Sun, 12 Sep 2010 18:25:33 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.2.8) Gecko/20100822 Lightning/1.0b2 Thunderbird/3.1.2 MIME-Version: 1.0 To: Martin Matuska References: <4C719AB9.9020006@freebsd.org> <4C8CD994.6060103@FreeBSD.org> In-Reply-To: <4C8CD994.6060103@FreeBSD.org> X-Enigmail-Version: 1.1.2 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Cc: zfs-devel@freebsd.org Subject: Re: ZFS arc_reclaim_needed: better cooperation with pagedaemon X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 12 Sep 2010 15:25:36 -0000 on 12/09/2010 16:45 Martin Matuska said the following: > I am running a really busy system with 48GB of RAM, equipped with 300GB > of SSD L2 cache as well and the I have recently observed the problem. > The system runs static fileserving from ZFS via sendfile() and NFS export. > > My vfs.zfs.arc_max is set to 40GB, vfs.zfs.arc_min is auto-set to 5GB (= > metalimit / 2, metalimit = arc_max / 4) > > NFS generates inactive memory that is not in use and leads to a > reduction of ARC up to vfs.zfs.arc_min (not below). > But I actually don't care about the cache of NFS and I really need the ARC. > > So if I understand this correctly, this might help me - but it would > lead to some kind of balance of incative memory vs ARC. > Anyway, best help for me would probably be setting vfs.zfs.arc_min to a Well, I am not sure - until you define a criterion "best" sounds a bit subjective :-) Anyway, I will greatly appreciate if you could test the patch before changing arc_min. > high value (20GB or more) so that the system tries to stay at this level. -- Andriy Gapon From owner-zfs-devel@FreeBSD.ORG Thu Sep 16 09:38:25 2010 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 84C12106564A for ; Thu, 16 Sep 2010 09:38:25 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from mail.vx.sk (mail.vx.sk [IPv6:2a01:4f8:100:1043::3]) by mx1.freebsd.org (Postfix) with ESMTP id 1D0848FC15 for ; Thu, 16 Sep 2010 09:38:25 +0000 (UTC) Received: from core.vx.sk (localhost [127.0.0.1]) by mail.vx.sk (Postfix) with ESMTP id 4B86110DCE0 for ; Thu, 16 Sep 2010 11:38:24 +0200 (CEST) X-Virus-Scanned: amavisd-new at mail.vx.sk Received: from mail.vx.sk ([127.0.0.1]) by core.vx.sk (mail.vx.sk [127.0.0.1]) (amavisd-new, port 10024) with LMTP id gbq8ixxlsMxc for ; Thu, 16 Sep 2010 11:38:11 +0200 (CEST) Received: from [10.0.3.3] (188-167-67-67.dynamic.chello.sk [188.167.67.67]) by mail.vx.sk (Postfix) with ESMTPSA id 9279B10DCD5 for ; Thu, 16 Sep 2010 11:38:11 +0200 (CEST) Message-ID: <4C91E582.3080509@FreeBSD.org> Date: Thu, 16 Sep 2010 11:38:10 +0200 From: Martin Matuska User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; sk; rv:1.8.1.23) Gecko/20090812 Lightning/0.9 Thunderbird/2.0.0.23 Mnenhy/0.7.5.0 MIME-Version: 1.0 To: zfs-devel@FreeBSD.org X-Enigmail-Version: 1.1.1 Content-Type: text/plain; charset=windows-1250 Content-Transfer-Encoding: 7bit Cc: Subject: Oracle releases Solaris 10 U9 with ZFS v22 without dedup X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 16 Sep 2010 09:38:25 -0000 Hi everyone, Oracle has released Solaris 10 update 9 on September, 8th. The new update comes with ZFS v22 without deduplication (legal or technical issues?), but includes the new metaslab code update. > zpool upgrade -v This system is currently running ZFS pool version 22. The following versions are supported: VER DESCRIPTION --- -------------------------------------------------------- 1 Initial ZFS version 2 Ditto blocks (replicated metadata) 3 Hot spares and double parity RAID-Z 4 zpool history 5 Compression using the gzip algorithm 6 bootfs pool property 7 Separate intent log devices 8 Delegated administration 9 refquota and refreservation properties 10 Cache devices 11 Improved scrub performance 12 Snapshot properties 13 snapused property 14 passthrough-x aclinherit 15 user/group space accounting 16 stmf property support 17 Triple-parity RAID-Z 18 Snapshot user holds 19 Log device removal 20 Compression using zle (zero-length encoding) 21 Reserved 22 Received properties From owner-zfs-devel@FreeBSD.ORG Thu Sep 16 11:48:17 2010 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 585E2106564A; Thu, 16 Sep 2010 11:48:17 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from harmony.bsdimp.com (bsdimp.com [199.45.160.85]) by mx1.freebsd.org (Postfix) with ESMTP id 1BDB68FC08; Thu, 16 Sep 2010 11:48:17 +0000 (UTC) Received: from [10.0.0.3] (warner-iphone.bsdimp.com [10.0.0.3]) by harmony.bsdimp.com (8.14.3/8.14.1) with ESMTP id o8GBf6WG002084; Thu, 16 Sep 2010 05:41:07 -0600 (MDT) (envelope-from imp@bsdimp.com) From: Warner Losh To: Martin Matuska In-Reply-To: <4C91E582.3080509@FreeBSD.org> X-Mailer: iPhone Mail (7D11) References: <4C91E582.3080509@FreeBSD.org> Message-Id: <3241E021-E2AC-48EA-A76C-D923DF23102A@bsdimp.com> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (iPhone Mail 7D11) Date: Thu, 16 Sep 2010 05:40:33 -0600 Cc: "zfs-devel@freebsd.org" Subject: Re: Oracle releases Solaris 10 U9 with ZFS v22 without dedup X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 16 Sep 2010 11:48:17 -0000 With or without sources??? Warner On Sep 16, 2010, at 3:38 AM, Martin Matuska wrote: > Hi everyone, > > Oracle has released Solaris 10 update 9 on September, 8th. > > The new update comes with ZFS v22 without deduplication (legal or > technical issues?), but includes the new metaslab code update. > >> zpool upgrade -v > This system is currently running ZFS pool version 22. > > The following versions are supported: > > VER DESCRIPTION > --- -------------------------------------------------------- > 1 Initial ZFS version > 2 Ditto blocks (replicated metadata) > 3 Hot spares and double parity RAID-Z > 4 zpool history > 5 Compression using the gzip algorithm > 6 bootfs pool property > 7 Separate intent log devices > 8 Delegated administration > 9 refquota and refreservation properties > 10 Cache devices > 11 Improved scrub performance > 12 Snapshot properties > 13 snapused property > 14 passthrough-x aclinherit > 15 user/group space accounting > 16 stmf property support > 17 Triple-parity RAID-Z > 18 Snapshot user holds > 19 Log device removal > 20 Compression using zle (zero-length encoding) > 21 Reserved > 22 Received properties > _______________________________________________ > zfs-devel@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/zfs-devel > To unsubscribe, send any mail to "zfs-devel-unsubscribe@freebsd.org" > > From owner-zfs-devel@FreeBSD.ORG Thu Sep 16 12:18:09 2010 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8DA6E106566C for ; Thu, 16 Sep 2010 12:18:09 +0000 (UTC) (envelope-from avg@icyb.net.ua) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id DC2DB8FC18 for ; Thu, 16 Sep 2010 12:18:08 +0000 (UTC) Received: from odyssey.starpoint.kiev.ua (alpha-e.starpoint.kiev.ua [212.40.38.101]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id OAA00676; Thu, 16 Sep 2010 14:59:17 +0300 (EEST) (envelope-from avg@icyb.net.ua) Message-ID: <4C920695.5000400@icyb.net.ua> Date: Thu, 16 Sep 2010 14:59:17 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.2.9) Gecko/20100909 Lightning/1.0b2 Thunderbird/3.1.3 MIME-Version: 1.0 To: zfs-devel@freebsd.org, Pawel Jakub Dawidek X-Enigmail-Version: 1.1.2 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Cc: Subject: changes in arc_reclaim_needed X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 16 Sep 2010 12:18:09 -0000 Pawel, I would like to go ahead and commit a (much tested now) change to arc_reclaim_needed: Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c =================================================================== --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c (revision 212636) +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c (working copy) @@ -2164,7 +2164,7 @@ * If pages are needed or we're within 2048 pages * of needing to page need to reclaim */ - if (vm_pages_needed || (vm_paging_target() > -2048)) + if (vm_paging_needed()) return (1); #if 0 Additionally I also want to commit the following change: Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c =================================================================== --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c (revision 212636) +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c (working copy) @@ -2155,10 +2155,6 @@ #ifdef _KERNEL if (needfree) return (1); - if (arc_size > arc_c_max) - return (1); - if (arc_size <= arc_c_min) - return (0); /* * If pages are needed or we're within 2048 pages Rationale. OpenSolaris upstream code doesn't have these checks. I've been using code changed as above for more than two months and all this time ARC size always stayed within its bounds. So the range is enforced in some other, more appropriate places. Reducing diff with upstream is also not bad. -- Andriy Gapon From owner-zfs-devel@FreeBSD.ORG Thu Sep 16 19:55:04 2010 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2D9A11065675 for ; Thu, 16 Sep 2010 19:55:04 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from mail.vx.sk (mail.vx.sk [IPv6:2a01:4f8:100:1043::3]) by mx1.freebsd.org (Postfix) with ESMTP id A6A4B8FC1E for ; Thu, 16 Sep 2010 19:55:03 +0000 (UTC) Received: from core.vx.sk (localhost [127.0.0.1]) by mail.vx.sk (Postfix) with ESMTP id 8679A110354; Thu, 16 Sep 2010 21:55:02 +0200 (CEST) X-Virus-Scanned: amavisd-new at mail.vx.sk Received: from mail.vx.sk ([127.0.0.1]) by core.vx.sk (mail.vx.sk [127.0.0.1]) (amavisd-new, port 10024) with LMTP id TDJBXhjv31oi; Thu, 16 Sep 2010 21:54:48 +0200 (CEST) Received: from [10.9.8.1] (188-167-78-139.dynamic.chello.sk [188.167.78.139]) by mail.vx.sk (Postfix) with ESMTPSA id 85514110324; Thu, 16 Sep 2010 21:54:48 +0200 (CEST) Message-ID: <4C927608.8080609@FreeBSD.org> Date: Thu, 16 Sep 2010 21:54:48 +0200 From: Martin Matuska User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; sk; rv:1.8.1.23) Gecko/20090812 Lightning/0.9 Thunderbird/2.0.0.23 Mnenhy/0.7.5.0 MIME-Version: 1.0 To: Warner Losh References: <4C91E582.3080509@FreeBSD.org> <3241E021-E2AC-48EA-A76C-D923DF23102A@bsdimp.com> In-Reply-To: <3241E021-E2AC-48EA-A76C-D923DF23102A@bsdimp.com> X-Enigmail-Version: 1.1.1 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: "zfs-devel@freebsd.org" Subject: Re: Oracle releases Solaris 10 U9 with ZFS v22 without dedup X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 16 Sep 2010 19:55:04 -0000 Of course without sources. And more news, Netapp and Oracle have agreed to dismiss lawsuits. http://www.netapp.com/us/company/news/news-rel-20100909-oracle-settlement.html The question is, why is Solaris 10 U9 without deduplication? If this has something to do with the agreement, one possibility is that they agreed deduplication may only be in Oracle Storage Appliances and Oracle will honor their patent. That would mean for us: no deduplication, or anybody using or selling it without paying a royalty to netapp can go to court But that is just a speculation as there is no more information available. Anyway, it is better to be cautious than to end like Coraid. Dňa 16. 9. 2010 13:40, Warner Losh wrote / napísal(a): > With or without sources??? > > Warner > > > On Sep 16, 2010, at 3:38 AM, Martin Matuska wrote: > >> Hi everyone, >> >> Oracle has released Solaris 10 update 9 on September, 8th. >> >> The new update comes with ZFS v22 without deduplication (legal or >> technical issues?), but includes the new metaslab code update. >> >>> zpool upgrade -v >> This system is currently running ZFS pool version 22. >> >> The following versions are supported: >> >> VER DESCRIPTION >> --- -------------------------------------------------------- >> 1 Initial ZFS version >> 2 Ditto blocks (replicated metadata) >> 3 Hot spares and double parity RAID-Z >> 4 zpool history >> 5 Compression using the gzip algorithm >> 6 bootfs pool property >> 7 Separate intent log devices >> 8 Delegated administration >> 9 refquota and refreservation properties >> 10 Cache devices >> 11 Improved scrub performance >> 12 Snapshot properties >> 13 snapused property >> 14 passthrough-x aclinherit >> 15 user/group space accounting >> 16 stmf property support >> 17 Triple-parity RAID-Z >> 18 Snapshot user holds >> 19 Log device removal >> 20 Compression using zle (zero-length encoding) >> 21 Reserved >> 22 Received properties >> _______________________________________________ >> zfs-devel@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/zfs-devel >> To unsubscribe, send any mail to "zfs-devel-unsubscribe@freebsd.org" >> >> From owner-zfs-devel@FreeBSD.ORG Thu Sep 16 20:30:40 2010 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 44B25106567A; Thu, 16 Sep 2010 20:30:40 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (60.wheelsystems.com [83.12.187.60]) by mx1.freebsd.org (Postfix) with ESMTP id D77BB8FC16; Thu, 16 Sep 2010 20:30:39 +0000 (UTC) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id 9D11345D8D; Thu, 16 Sep 2010 22:30:37 +0200 (CEST) Received: from localhost (chello089077043238.chello.pl [89.77.43.238]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id 96B6945C98; Thu, 16 Sep 2010 22:30:31 +0200 (CEST) Date: Thu, 16 Sep 2010 22:30:15 +0200 From: Pawel Jakub Dawidek To: Martin Matuska Message-ID: <20100916203015.GA1967@garage.freebsd.pl> References: <4C91E582.3080509@FreeBSD.org> <3241E021-E2AC-48EA-A76C-D923DF23102A@bsdimp.com> <4C927608.8080609@FreeBSD.org> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="jI8keyz6grp/JLjh" Content-Disposition: inline In-Reply-To: <4C927608.8080609@FreeBSD.org> User-Agent: Mutt/1.4.2.3i X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 9.0-CURRENT amd64 X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-0.6 required=4.5 tests=BAYES_00,RCVD_IN_SORBS_DUL autolearn=no version=3.0.4 Cc: "zfs-devel@freebsd.org" Subject: Re: Oracle releases Solaris 10 U9 with ZFS v22 without dedup X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 16 Sep 2010 20:30:40 -0000 --jI8keyz6grp/JLjh Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Thu, Sep 16, 2010 at 09:54:48PM +0200, Martin Matuska wrote: > Of course without sources. >=20 > And more news, Netapp and Oracle have agreed to dismiss lawsuits. >=20 > http://www.netapp.com/us/company/news/news-rel-20100909-oracle-settlement= .html >=20 > The question is, why is Solaris 10 U9 without deduplication? If this has > something to do with the agreement, one possibility is that they agreed > deduplication may only be in Oracle Storage Appliances and Oracle will > honor their patent. >=20 > That would mean for us: no deduplication, or anybody using or selling it > without paying a royalty to netapp can go to court >=20 > But that is just a speculation as there is no more information available. >=20 > Anyway, it is better to be cautious than to end like Coraid. My understanding is very different. I was under impression that NetApp cannot go after people using OpenSolaris until they win in court. Before that every OpenSolaris code consumer is protected by CDDL. NetApp didn't win, right? Also: "both parties have agreed to dismiss their pending patent litigation" My English might not be enough here, but my reading is that because patent litigation was dismissed, NetApp no longer claims (in legal sense) that Oracle violates NetApp's patents via OpenSolaris. Is my reading correct or not? --=20 Pawel Jakub Dawidek http://www.wheelsystems.com pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --jI8keyz6grp/JLjh Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (FreeBSD) iEYEARECAAYFAkySflYACgkQForvXbEpPzQCEACgjC3q6U1mCGaI5SrWf2nK6+Xo MQYAniKJAC49OYfal0THimuxto07iqsT =i9Po -----END PGP SIGNATURE----- --jI8keyz6grp/JLjh-- From owner-zfs-devel@FreeBSD.ORG Thu Sep 16 21:08:04 2010 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 35FA71065672; Thu, 16 Sep 2010 21:08:04 +0000 (UTC) (envelope-from mattjeet@gmail.com) Received: from mail-wy0-f182.google.com (mail-wy0-f182.google.com [74.125.82.182]) by mx1.freebsd.org (Postfix) with ESMTP id 8F84D8FC18; Thu, 16 Sep 2010 21:08:03 +0000 (UTC) Received: by wyb33 with SMTP id 33so2384128wyb.13 for ; Thu, 16 Sep 2010 14:08:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:sender:reply-to:received :in-reply-to:references:date:x-google-sender-auth:message-id:subject :from:to:cc:content-type; bh=JuYvCJvJmqEXl1RHpPBLEAj71pH2b9VLqeFd/G2qluk=; b=FQas36J82wLE8GxUyLEK+gAh28gInWsqAZdSTOb7e6IRXTq1+tEJuiG2u4tWoSBgwU DHg023Tc0AWQzrx3UbSyJYUZUJ/L1sn2Bw9/a1vbu2PDCtHvoW7WTHhvXpCf3rT+7Kdn P/Hrt1HhlKM1tvcUnFrR5ETdrtkMa4PfP8kuI= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:reply-to:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; b=ujMFeB78uee28rRNCpbHZopSTqGOq9MTRNGE+bTSQEBLpO7ttYSPqFscF/2FAe066F T2+rnDQSXQiZ3ZSzS/K4hISB+kfaCmrv+Z9Z/R8TtKxa+JSf+hTCuKXY9qMxZOyzWiFR Ez3h6U3JAt6a3pFE6rp9NortmR0lil/JRXkZI= MIME-Version: 1.0 Received: by 10.227.142.136 with SMTP id q8mr3346262wbu.95.1284669466507; Thu, 16 Sep 2010 13:37:46 -0700 (PDT) Sender: mattjeet@gmail.com Received: by 10.227.154.147 with HTTP; Thu, 16 Sep 2010 13:37:46 -0700 (PDT) In-Reply-To: <20100916203015.GA1967@garage.freebsd.pl> References: <4C91E582.3080509@FreeBSD.org> <3241E021-E2AC-48EA-A76C-D923DF23102A@bsdimp.com> <4C927608.8080609@FreeBSD.org> <20100916203015.GA1967@garage.freebsd.pl> Date: Thu, 16 Sep 2010 13:37:46 -0700 X-Google-Sender-Auth: rtd5N9ri6027KF0nxeEulYvGLLI Message-ID: From: Matt Olander To: Pawel Jakub Dawidek Content-Type: text/plain; charset=ISO-8859-1 Cc: "zfs-devel@freebsd.org" Subject: Re: Oracle releases Solaris 10 U9 with ZFS v22 without dedup X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: matt@ixsystems.com List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 16 Sep 2010 21:08:04 -0000 On Thu, Sep 16, 2010 at 1:30 PM, Pawel Jakub Dawidek wrote: > On Thu, Sep 16, 2010 at 09:54:48PM +0200, Martin Matuska wrote: >> Of course without sources. >> >> And more news, Netapp and Oracle have agreed to dismiss lawsuits. >> >> http://www.netapp.com/us/company/news/news-rel-20100909-oracle-settlement.html >> >> The question is, why is Solaris 10 U9 without deduplication? If this has >> something to do with the agreement, one possibility is that they agreed >> deduplication may only be in Oracle Storage Appliances and Oracle will >> honor their patent. >> >> That would mean for us: no deduplication, or anybody using or selling it >> without paying a royalty to netapp can go to court >> >> But that is just a speculation as there is no more information available. >> >> Anyway, it is better to be cautious than to end like Coraid. > > My understanding is very different. I was under impression that NetApp > cannot go after people using OpenSolaris until they win in court. Before > that every OpenSolaris code consumer is protected by CDDL. NetApp didn't > win, right? > > Also: > > "both parties have agreed to dismiss their pending patent litigation" > > My English might not be enough here, but my reading is that because patent > litigation was dismissed, NetApp no longer claims (in legal sense) that > Oracle violates NetApp's patents via OpenSolaris. Is my reading correct > or not? Yes, this is also how I interpreted it. Coraid backed down selling an appliance that was not theirs (Nexenta on Supermicro) based on a simple cease & desist letter. They are a very small company and would rather back down than face any kind of possible legal action. In fact, Nexenta (the more appropriate target) never received the letter. Either way, both parties dismissed their claims. -matt From owner-zfs-devel@FreeBSD.ORG Thu Sep 16 21:22:53 2010 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0FD1D10656A3; Thu, 16 Sep 2010 21:22:53 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from harmony.bsdimp.com (bsdimp.com [199.45.160.85]) by mx1.freebsd.org (Postfix) with ESMTP id C4EC58FC16; Thu, 16 Sep 2010 21:22:52 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by harmony.bsdimp.com (8.14.3/8.14.1) with ESMTP id o8GLGVCY008143; Thu, 16 Sep 2010 15:16:31 -0600 (MDT) (envelope-from imp@bsdimp.com) Date: Thu, 16 Sep 2010 15:16:40 -0600 (MDT) Message-Id: <20100916.151640.585585711625015566.imp@bsdimp.com> To: pjd@FreeBSD.org From: "M. Warner Losh" In-Reply-To: <20100916203015.GA1967@garage.freebsd.pl> References: <3241E021-E2AC-48EA-A76C-D923DF23102A@bsdimp.com> <4C927608.8080609@FreeBSD.org> <20100916203015.GA1967@garage.freebsd.pl> X-Mailer: Mew version 6.3 on Emacs 22.3 / Mule 5.0 (SAKAKI) Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit Cc: zfs-devel@FreeBSD.org, mm@FreeBSD.org Subject: Re: Oracle releases Solaris 10 U9 with ZFS v22 without dedup X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 16 Sep 2010 21:22:53 -0000 In message: <20100916203015.GA1967@garage.freebsd.pl> Pawel Jakub Dawidek writes: : On Thu, Sep 16, 2010 at 09:54:48PM +0200, Martin Matuska wrote: : > Of course without sources. : > : > And more news, Netapp and Oracle have agreed to dismiss lawsuits. : > : > http://www.netapp.com/us/company/news/news-rel-20100909-oracle-settlement.html : > : > The question is, why is Solaris 10 U9 without deduplication? If this has : > something to do with the agreement, one possibility is that they agreed : > deduplication may only be in Oracle Storage Appliances and Oracle will : > honor their patent. : > : > That would mean for us: no deduplication, or anybody using or selling it : > without paying a royalty to netapp can go to court : > : > But that is just a speculation as there is no more information available. : > : > Anyway, it is better to be cautious than to end like Coraid. : : My understanding is very different. I was under impression that NetApp : cannot go after people using OpenSolaris until they win in court. Before : that every OpenSolaris code consumer is protected by CDDL. NetApp didn't : win, right? : : Also: : : "both parties have agreed to dismiss their pending patent litigation" : : My English might not be enough here, but my reading is that because patent : litigation was dismissed, NetApp no longer claims (in legal sense) that : Oracle violates NetApp's patents via OpenSolaris. Is my reading correct : or not? Not necessarily. My reading of it was that they had some cross licensing agreement, and agreed to dismiss their claims without prejudice (meaning the patents weren't held invalid, among other things). Warner From owner-zfs-devel@FreeBSD.ORG Fri Sep 17 07:10:26 2010 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 356821065673; Fri, 17 Sep 2010 07:10:26 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (60.wheelsystems.com [83.12.187.60]) by mx1.freebsd.org (Postfix) with ESMTP id D285F8FC23; Fri, 17 Sep 2010 07:10:25 +0000 (UTC) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id 2DB1145C9C; Fri, 17 Sep 2010 09:10:24 +0200 (CEST) Received: from localhost (chello089077043238.chello.pl [89.77.43.238]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id 2E7B245C89; Fri, 17 Sep 2010 09:10:19 +0200 (CEST) Date: Fri, 17 Sep 2010 09:10:02 +0200 From: Pawel Jakub Dawidek To: "M. Warner Losh" Message-ID: <20100917071002.GB1967@garage.freebsd.pl> References: <3241E021-E2AC-48EA-A76C-D923DF23102A@bsdimp.com> <4C927608.8080609@FreeBSD.org> <20100916203015.GA1967@garage.freebsd.pl> <20100916.151640.585585711625015566.imp@bsdimp.com> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="ZoaI/ZTpAVc4A5k6" Content-Disposition: inline In-Reply-To: <20100916.151640.585585711625015566.imp@bsdimp.com> User-Agent: Mutt/1.4.2.3i X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 9.0-CURRENT amd64 X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-0.6 required=4.5 tests=BAYES_00,RCVD_IN_SORBS_DUL autolearn=no version=3.0.4 Cc: zfs-devel@FreeBSD.org, mm@FreeBSD.org Subject: Re: Oracle releases Solaris 10 U9 with ZFS v22 without dedup X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 17 Sep 2010 07:10:26 -0000 --ZoaI/ZTpAVc4A5k6 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Thu, Sep 16, 2010 at 03:16:40PM -0600, M. Warner Losh wrote: > : My understanding is very different. I was under impression that NetApp > : cannot go after people using OpenSolaris until they win in court. Before > : that every OpenSolaris code consumer is protected by CDDL. NetApp didn't > : win, right? > :=20 > : Also: > :=20 > : "both parties have agreed to dismiss their pending patent litigation" > :=20 > : My English might not be enough here, but my reading is that because pat= ent > : litigation was dismissed, NetApp no longer claims (in legal sense) that > : Oracle violates NetApp's patents via OpenSolaris. Is my reading correct > : or not? >=20 > Not necessarily. My reading of it was that they had some cross > licensing agreement, and agreed to dismiss their claims without > prejudice (meaning the patents weren't held invalid, among other > things). The patents are still valid of course, but NetApp didn't prove in court that Oracle violates them with OpenSolaris, so CDDL still protects code consumers, no? --=20 Pawel Jakub Dawidek http://www.wheelsystems.com pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --ZoaI/ZTpAVc4A5k6 Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (FreeBSD) iEYEARECAAYFAkyTFEkACgkQForvXbEpPzTCXQCfRsNOU2ejiiD0lRoPsQdwSfxD XdgAnR1J/hfelzrx7Zp9NheIiOhNRnCI =7oUV -----END PGP SIGNATURE----- --ZoaI/ZTpAVc4A5k6-- From owner-zfs-devel@FreeBSD.ORG Fri Sep 17 10:02:16 2010 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BBDC6106566C; Fri, 17 Sep 2010 10:02:16 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from mail.vx.sk (mail.vx.sk [IPv6:2a01:4f8:100:1043::3]) by mx1.freebsd.org (Postfix) with ESMTP id 510058FC0C; Fri, 17 Sep 2010 10:02:16 +0000 (UTC) Received: from core.vx.sk (localhost [127.0.0.1]) by mail.vx.sk (Postfix) with ESMTP id 7DA1A11911B; Fri, 17 Sep 2010 12:02:15 +0200 (CEST) X-Virus-Scanned: amavisd-new at mail.vx.sk Received: from mail.vx.sk ([127.0.0.1]) by core.vx.sk (mail.vx.sk [127.0.0.1]) (amavisd-new, port 10024) with LMTP id SFqoSxjktPoc; Fri, 17 Sep 2010 12:02:00 +0200 (CEST) Received: from [10.9.8.1] (188-167-78-139.dynamic.chello.sk [188.167.78.139]) by mail.vx.sk (Postfix) with ESMTPSA id B410F119105; Fri, 17 Sep 2010 12:01:59 +0200 (CEST) Message-ID: <4C933C99.9030802@FreeBSD.org> Date: Fri, 17 Sep 2010 12:02:01 +0200 From: Martin Matuska User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; sk; rv:1.8.1.23) Gecko/20090812 Lightning/0.9 Thunderbird/2.0.0.23 Mnenhy/0.7.5.0 MIME-Version: 1.0 To: Pawel Jakub Dawidek , Xin LI X-Enigmail-Version: 1.1.1 Content-Type: text/plain; charset=windows-1250 Content-Transfer-Encoding: 7bit Cc: zfs-devel@FreeBSD.org, Andriy Gapon Subject: ZFS patch proposal: offlining log devices X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 17 Sep 2010 10:02:16 -0000 The handling of log devices prior to pool version 19 is quite problematic. They cannot be removed nor offlined. I want to propose a patch against head and later MFC that imports the following OpenSolaris revision, that enables offlining of log devices: 9701:cc5b64682e64 6803605 should be able to offline log devices 6726045 vdev_deflate_ratio is not set when offlining a log device 6599442 zpool import has faults in the display http://mfsbsd.vx.sk/zfs/head-9701.patch I did successfully test the following: - offlining a log device (ZIL is written to disk and device offlined) - importing the pool without offlined log devices (UNAVAIL with saved zpool.cache) Next I would also like to propose this additional patch: http://mfsbsd.vx.sk/zfs/head-9725.patch 9725:0bf7402e8022 6843014 ZFS B_FAILFAST handling is broken Both patches reduce diff against v28. Thank you, mm From owner-zfs-devel@FreeBSD.ORG Fri Sep 17 17:57:23 2010 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 104C41065695 for ; Fri, 17 Sep 2010 17:57:23 +0000 (UTC) (envelope-from 2010@joshcarter.com) Received: from joshcarter.com (67-207-137-80.slicehost.net [67.207.137.80]) by mx1.freebsd.org (Postfix) with ESMTP id E329C8FC08 for ; Fri, 17 Sep 2010 17:57:22 +0000 (UTC) Received: from [192.168.4.126] (207-225-98-3.dia.static.qwest.net [207.225.98.3]) by joshcarter.com (Postfix) with ESMTPSA id 315F4128040 for ; Fri, 17 Sep 2010 17:38:02 +0000 (UTC) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Apple Message framework v1081) From: Josh Carter <2010@joshcarter.com> In-Reply-To: <20100917071002.GB1967@garage.freebsd.pl> Date: Fri, 17 Sep 2010 11:37:56 -0600 Content-Transfer-Encoding: quoted-printable Message-Id: References: <3241E021-E2AC-48EA-A76C-D923DF23102A@bsdimp.com> <4C927608.8080609@FreeBSD.org> <20100916203015.GA1967@garage.freebsd.pl> <20100916.151640.585585711625015566.imp@bsdimp.com> <20100917071002.GB1967@garage.freebsd.pl> To: zfs-devel@FreeBSD.org X-Mailer: Apple Mail (2.1081) Cc: Subject: Re: Oracle releases Solaris 10 U9 with ZFS v22 without dedup X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 17 Sep 2010 17:57:23 -0000 Hi all, We've been watching the NetApp/Oracle case at Spectra Logic with great = interest, and our conclusions: - The settlement is a "good thing" simply because things are moving = along. - The terms of the settlement are confidential so far. For us, this = could mean anything. The deal could be exclusively between Oracle and = NetApp, therefore other companies are left to fend for themselves. The = deal could extend to OpenSolaris, which would be far better. We need to = wait and see. - CDDL protects us only from Oracle. It gives us license to *their* = relevant patents. It does not indemnify us from prosecution by other = companies who claim infringement. Summary: wait and see. We should hear something more about it soon. In = the meantime, Coraid and other companies are still just as exposed to = NetApp as before. Best regards, Josh From owner-zfs-devel@FreeBSD.ORG Tue Sep 21 18:41:40 2010 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0DEF1106564A for ; Tue, 21 Sep 2010 18:41:40 +0000 (UTC) (envelope-from dewcopper@gmail.com) Received: from mail-ey0-f182.google.com (mail-ey0-f182.google.com [209.85.215.182]) by mx1.freebsd.org (Postfix) with ESMTP id 93D4E8FC12 for ; Tue, 21 Sep 2010 18:41:39 +0000 (UTC) Received: by eyx24 with SMTP id 24so2580477eyx.13 for ; Tue, 21 Sep 2010 11:41:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:sender:received:date :x-google-sender-auth:message-id:subject:from:to:content-type; bh=5jCo6JrgzBAd651U2BYOl1cg4gUiWPDHnOH/DZhxs7E=; b=NUxJiqdkhvAPKApMQTDpINuOsGsoGZeD1tLr0gKtThVapm3izTqRWMWysGgwFO+Gbz kWEbvqO/B/VTi5hcHg9C2mgaRLvFWlRmCtgaLSJbYH2noGc554lz7PjiW5uJZAGZYCbd x87CfrxFyPWcX1j16Mr74qPp0G2200lzf/o2E= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:date:x-google-sender-auth:message-id:subject :from:to:content-type; b=AdCyTq+YiBexOFqHPNk4mdRCBjBGzUY8a1u+N7T/94uEFy157L9tLiraxSiDbU6QU4 yQfU1LYPGFJ8tJzzPkRuBjKG+fA4y68/FieWvro5CvV3Kq/zEg/zq42lCwuTnrZZy2sW s/xnz3sXDnWNvBzUNFKgGeiSnvcfUpOzez4HY= MIME-Version: 1.0 Received: by 10.213.36.3 with SMTP id r3mr4462997ebd.88.1285093125064; Tue, 21 Sep 2010 11:18:45 -0700 (PDT) Sender: dewcopper@gmail.com Received: by 10.14.119.78 with HTTP; Tue, 21 Sep 2010 11:18:44 -0700 (PDT) Date: Tue, 21 Sep 2010 11:18:44 -0700 X-Google-Sender-Auth: uRjrg5a_mtw1eAOSxGVInc6SnZY Message-ID: From: Tushar Tambay To: zfs-devel@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: notes from today's call X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 21 Sep 2010 18:41:40 -0000 participants : samir, ganesh, xin, steve, tushar date : 9/21/10 brief action items from today's call :- ganesh / samir will send out details of test suite location & instructions for running the same. xin will try the new test suite on iXSystems h/w ganesh / samir will send out performance report on iozone based synchronous write performance testing that they did recently. xin will share his results on zfs performance testing so far and also run the iozone tests to confirm results. -- tyt phone: 408 203 9736 (c) - From owner-zfs-devel@FreeBSD.ORG Thu Sep 23 10:43:36 2010 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D558A106564A for ; Thu, 23 Sep 2010 10:43:36 +0000 (UTC) (envelope-from ganesh.varadarajan@gmail.com) Received: from mail-qy0-f182.google.com (mail-qy0-f182.google.com [209.85.216.182]) by mx1.freebsd.org (Postfix) with ESMTP id 735468FC17 for ; Thu, 23 Sep 2010 10:43:35 +0000 (UTC) Received: by qyk4 with SMTP id 4so2328637qyk.13 for ; Thu, 23 Sep 2010 03:43:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:date:message-id :subject:from:to:content-type; bh=357MCLVcl6PZeZE5Xvan++41/5y+f8nlgfkHF5ae4MM=; b=tFAF/CPxUbNq1xpe5XbrvrFFgFnmRI/R7R6+ARDDQK9XAOWG9Lvm80Iz37FTw1v1q6 eDSDVJLbOSLVoDRbBmXr8VMOSTrFn6F1WKMdfG5ABnoXmWZwy6nn96AGmJSQNx+578o0 blpoe8iGW0Yl5IwakXrNa/NrFRtiTtRK1nm/I= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:content-type; b=FW/uKZtcv9eIlPWlikVbAxaYxJyQSxW92TPkIIT0At4wF6B97VaFAjdQ16cwojUO02 Cj0U3cnjWx6Y3cfGz75AEOoNxIm8cLwj66ft57UZofO7zWDV67wrVZ2NcSysgA5a4I3V Q3xkX0i/t1B+pckISQ/M6Z3i1YV5iz6r8Do7c= MIME-Version: 1.0 Received: by 10.224.72.2 with SMTP id k2mr1111456qaj.242.1285237103298; Thu, 23 Sep 2010 03:18:23 -0700 (PDT) Received: by 10.229.221.12 with HTTP; Thu, 23 Sep 2010 03:18:23 -0700 (PDT) Date: Thu, 23 Sep 2010 15:48:23 +0530 Message-ID: From: Ganesh Varadarajan To: zfs-devel@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: ZFS performance notes X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Sep 2010 10:43:36 -0000 [This is extracted from a series of mails Samir and I wrote while investigating ZFS performance on FreeBSD. Version doesn't matter; we used both v14 and v25 with largely similar results.] We've finally got some understanding of ZFS performance in the sync-write case (important for NFS and file server loads) and why it's as low as it is. Bottom line: It's not ZFS's fault. We need a fast NVRAM-type log device if we want performance. This is the iostat output for an 8k sync write load with iozone on a ZFS filesystem (FreeBSD 8.1, local, no NFS, pool consisting of just one 7200rpm SATA disk) iozone -s 100m -r 8 -i 0 -o -f /tp/fs/f1 (gives 935 KB/sec throughput) tty ad4 ad6 ad10 cpu tin tout KB/t tps MB/s KB/t tps MB/s KB/t tps MB/s us ni sy in id 0 39 0.00 0 0.00 12.00 118 1.39 0.00 0 0.00 0 0 0 0 100 0 39 0.00 0 0.00 12.00 117 1.38 0.00 0 0.00 0 0 0 0 100 0 39 0.00 0 0.00 69.32 220 14.88 0.00 0 0.00 0 0 0 0 99 0 39 0.00 0 0.00 12.00 118 1.38 0.00 0 0.00 0 0 1 0 99 0 39 0.00 0 0.00 12.00 118 1.39 0.00 0 0.00 0 0 0 0 100 Several interesting things here. Note that each transaction is 12K. This is as we expect, each 8k write is padded with 4k of ZIL stuff. We're able to to 118 transactions per sec (aka IOPS). The steady state throughput on disk is 1.39 MB/sec, correlating well to the 935 KB/sec observed by the application after discounting ZIL metadata overhead. The spike of 14MB/sec corresponds to a flush of log data to primary fs. Removing the sync option, we try iozone -s 1000m -r 8 -i 0 -f /tp/fs/f1 and see that ZFS can generate quite a storm of IO. tty ad4 ad6 ad10 cpu tin tout KB/t tps MB/s KB/t tps MB/s KB/t tps MB/s us ni sy in id 0 39 8.38 2 0.02 128.00 751 93.82 0.00 0 0.00 0 0 13 1 86 0 39 0.00 0 0.00 128.00 1020 127.56 0.00 0 0.00 0 0 1 1 98 0 40 0.00 0 0.00 128.00 1015 126.92 0.00 0 0.00 0 0 2 1 97 0 40 0.00 0 0.00 124.94 702 85.66 0.00 0 0.00 0 0 14 0 85 Now, the question is: if ZFS can generate 120MB/sec sustained throughput to the disk, why is the ZIL throttled so low? And the answer is... On-disk write cache. By default, FreeBSD enables on-disk cache. That's why iozone to a raw disk shows very large numbers: iozone -s 1000m -r 8 -i 0 -o -f /dev/ad10 (gives 80 MB/sec) tty ad4 ad6 ad10 cpu tin tout KB/t tps MB/s KB/t tps MB/s KB/t tps MB/s us ni sy in id 0 39 0.00 0 0.00 0.00 0 0.00 8.00 10031 78.36 0 0 9 6 86 0 40 0.00 0 0.00 0.00 0 0.00 8.00 10019 78.28 0 0 9 6 84 0 40 0.00 0 0.00 0.00 0 0.00 8.00 10040 78.43 0 0 9 6 85 Note that we're doing 8k IOs as expected, but we're completing 10,000 of them in a second (10,000 IOPS!) Clearly, this is because the disk is caching them. They're not going to the platter, and might very well disappear if there is a power failure. Default UFS on such a disk therefore exhibits good performance. At higher IOsizes (64-128K), we max out at 120MB/sec - not bad for a single 7200 RPM 500Gig SATA disk, what? Random reads of 8k on raw disk give ~1.4MB/sec (since nothing can be cached) and this is a real reflection on platter performance. Notice that this correlates well with the ZFS sync-write numbers we got first. Random writes are higher (~3MB/sec), but nowhere near the max rate, reflecting that the disk cache can't absorb these as well as writes coming in a sequence. Now if we turn FreeBSD write cache off, (add a line hw.ata.wc = 0 to /boot/loader.conf and reboot), then we see, for sync IO to a raw device, iozone -s 10m -r 12 -i 0 -o -f /dev/ad10, a throughput of 1.4MB/sec, exactly the same as the first ZFS case. tty ad4 ad6 ad10 cpu tin tout KB/t tps MB/s KB/t tps MB/s KB/t tps MB/s us ni sy in id 0 39 0.00 0 0.00 0.00 0 0.00 12.00 119 1.39 0 0 0 0 100 0 39 0.00 0 0.00 0.00 0 0.00 12.00 118 1.39 0 0 0 0 100 0 39 0.00 0 0.00 0.00 0 0.00 12.00 119 1.40 0 0 0 0 100 I chose 12k to correspond to the 8k data +4k ZIL overhead workload. We see that IOPS drops to ~120 from ~10,000 when we turn off the disk cache. UFS does *worse* than ZFS if we turn off the write-cache. iozone -s 10m -r 8 -i 0 -o 0 39 0.00 0 0.00 0.00 0 0.00 16.00 145 2.27 0 0 0 0 100 It manages to generate 2.2 MB/sec throughput, but notice that it turned every 8k app write to a 16k write. The actual throughput as seen by iozone is 495K/sec, a fourth of what's going to disk. This is because of the major inefficiencies in UFS sync writes (8k->16k, metadata updates) as discussed earlier. When we do async writes (also with write-cache off), it goes upto a throughput of ~14MB/sec. iozone -s 100m -r 8 -i 0 0 40 0.00 0 0.00 0.00 0 0.00 128.00 107 13.43 0 0 0 0 100 0 40 0.00 0 0.00 0.00 0 0.00 128.00 108 13.56 0 0 0 0 100 Note that UFS gathers writes into 128k chunks, but we're band-limited by IOPs here. The platters can't do more than 110-120 IOPS (when the cache is useless - either turned off or due to random load) With write-cache turned off, ZFS async writes give exactly the same performance as UFS - ~14MB/sec, 108 IOPS, 128k IO sizes. Sync writes are the same old 1.3MB/sec. So, ZFS is actually being pretty smart. In the normal case, when disk write-cache is turned on, it uses the cache for all regular writes, but when it comes to ZIL writes, it makes sure that the bytes hit the platter. So the ZIL, although in theory sequential, actually degenerates to close to random write performance, because we cannot gather the IO at any stage. 8k hits the platter, we go back to the application, which spits out 8k more. Meanwhile, the disk has spun so the second IO has to wait for the planets to align again. There is a ZFS tunable which says "don't force a platter write for ZIL, treat it as normal IO". Set vfs.zfs.cache_flush_disable="1" in /boot/loader.conf and reboot Now you will see the ZIL unfettered by the ~100 IOPS limit iozone -s 1000m -r 8 -i 0 -o -f /tp/fs/f1 (gives ~40MB/sec throughput, up from 1.3!) 0 39 0.00 0 0.00 12.00 5530 64.80 0.00 0 0.00 0 0 13 3 84 0 39 0.00 0 0.00 12.00 5506 64.53 0.00 0 0.00 1 0 11 2 86 Note that we go to 5k IOPS and 64MB/sec ZIL throughput. To summarize so far: 1. If we turn off the disk write-cache, we are IOPS-limited. 128k * 100-120 IOPS is the best we can hope for under any circumstances per disk of this type. 2. ZFS does the sane thing, using disk cache when possible and avoiding it when semantics demand. In our case that boils down to ZIL and platter writes every time, since our NFS-client issues sync writes. 3. If we want to honour NFS sync write semantics (and we must), without getting bad performance, 1. some kind of NVRAM based log accelerator is needed. 2. large number of spindles in mirror-stripe config (untested) All this has been for single-threaded loads. Now let's look into logbias and scaling with multi-threaded loads. Multi-threading does break through the 100-120 IOPS sound barrier for Real Sync Writes (TM) if and only if disk write cache is enabled. We can go upto 300 IOPS, delivering an aggregate of 27MB/sec for 8k, 64 threads. Per-thread throughput remains low, at around 500K/sec. 1. How does this happen? As we saw earlier, ZFS performs best when the write cache is enabled. In such cases, 1. ZIL writes are done to the log device, going to the disks write-cache 2. then a cache flush is issued 3. On successful flush (bits have hit platter), return success to the caller. 2. If another ZIL write happens between 1.1 and 1.2 above (multi-threading case) then one cache flush does for both. This is what enables us to break through the IOPS barrier without compromising correctness. 3. If we turn off the write-cache, then we simply cannot bust the 100 IOPS barrier, no matter how many threads we use. This serves to confirm the above. All tests have been with logbias=latency. If we change logbias=throughput, an FS-level option, ZFS avoids using the separate log device for this filesystem and dumps it directly to primary storage. This is used when there are multiple filesystems in a pool and a log device configured on an SSD. Instead of flooding the limited SSD with sync-writes from all filesystems, you may want to selectively do it for a few (logbias=latency) and not for others (logbias=throughput). Not much use for us. Multiple disks, no log device. Adding multiple disks to the pool does help; ZFS nicely distributes data, including sync-writes to all disks. Ramdisk (as a substitute for NVRAM) as log disk improved performance significantly, as expected, to about 40 MB/sec for a single-threaded 8k sync write. Ramdisks sometimes cause odd oscillations - the log gets full very fast, ZFS can't or doesn't drain the pent-up IO to primary fast enough, then ZFS switches to the primary disks for writing the log for several seconds. This results in sharp oscillations in throughput. Ramdisks also help a lot with RAID-Z performance. Everything gets staged through the log, thus helping to gather large-striped writes in memory. Rewrites do worse than writes when the IO size is less than the record size. Dedup has negligible effect on throughput when measured for single-threaded loads. Apparently performance falls off a cliff when the dedup table runs out of memory, but we didn't push it that far. Tentative recommendations: 1. Use a dedicated NVRAM device for the log. Size should be at least 1 GB, ideally 2-4 GB. Random write latencies for sizes from 8k-64k to NVRAM should be good. 2. Since all writes are sync, increasing RAM size beyond 2x of NVRAM size won't help much. So 8 GB RAM should be enough for a typical 2-4 GB NVRAM case. We expect reads to be cached at the client. 3. For dedup, many people recommend keeping a flash device which can do fast random-access reads to keep the dedup table. This avoids the performance cliff for typical storage and record sizes without requiring extra RAM. 4. Mirroring is preferred to RAID-Z for raw performance. RAID-Z ain't bad though, if we have an NVRAM log. 5. Set recordsize to 32k - NFS write ops don't exceed this size (this needs more study) 6. Leave logbias to its default value, 'latency'. More accurate measurements and sizing recommendations can be made if we can get our hands on a "typical" customer system, with 6 or 12 spindles and an NVRAM device. From owner-zfs-devel@FreeBSD.ORG Thu Sep 23 12:41:35 2010 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 92CBF1065674 for ; Thu, 23 Sep 2010 12:41:35 +0000 (UTC) (envelope-from hellosamirdesai@gmail.com) Received: from mail-gx0-f182.google.com (mail-gx0-f182.google.com [209.85.161.182]) by mx1.freebsd.org (Postfix) with ESMTP id 511378FC0A for ; Thu, 23 Sep 2010 12:41:35 +0000 (UTC) Received: by mail-gx0-f182.google.com with SMTP id 8so650615gxk.13 for ; Thu, 23 Sep 2010 05:41:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:date:message-id :subject:from:to:content-type; bh=leZyiLURAusLBruyhjNOKWU7QnDlWkJqkFjPwDQDCmY=; b=dmwMjYp4hzmcQxKybCe1DG7wXD4RVbL3cd3wh2ofxfgfg8cFLNWQhTQO0cDsQYh0Xn O5J25+9cZrHDlUfmttUsoTdpVkN7pcvNR+9smwHjOZsg1owPFIT2A4ltxz/oDQxJJRNr Ynw5vClFbBR8BXXCd03rNYBHu19ooCxyG8TA8= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:content-type; b=MuAPEj0NFIVQJhKMlwVqaYU+iDeGYA6SCoqO2Xiv6fuufvyDovDbU7QsfIdlFYw0N1 TTqpiRtYqugcyJFlGjlmza3eNU8e6ZxvZlPSoi837EPAFxWwN3TeWlu/U2u2iLTbi8wD lYlFH1vSBcAA110ipo09abpVifJSvzU/jYpOA= MIME-Version: 1.0 Received: by 10.151.6.5 with SMTP id j5mr2726049ybi.37.1285245206255; Thu, 23 Sep 2010 05:33:26 -0700 (PDT) Received: by 10.220.195.68 with HTTP; Thu, 23 Sep 2010 05:33:23 -0700 (PDT) Date: Thu, 23 Sep 2010 18:03:23 +0530 Message-ID: From: Samir Desai To: zfs-devel@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: zfs test suite X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Sep 2010 12:41:35 -0000 hi We have checked in all the code and test suite in to perforce the branch is //depot/user/samir/zfs, this is a branch of //depot/projects/zfs/stable/8 Once you checkout, go to the zfs directory # ls zfs_stable_8/zfs README build-test.sh build.sh patch-kernel.sh please read README here, it has all the information necessary to setup the freebsd system, building and installing zfs kernel/command and building/running zfs test suite regards Samir From owner-zfs-devel@FreeBSD.ORG Thu Sep 23 12:58:19 2010 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 22413106564A for ; Thu, 23 Sep 2010 12:58:19 +0000 (UTC) (envelope-from hellosamirdesai@gmail.com) Received: from mail-gy0-f182.google.com (mail-gy0-f182.google.com [209.85.160.182]) by mx1.freebsd.org (Postfix) with ESMTP id D1ECA8FC12 for ; Thu, 23 Sep 2010 12:58:18 +0000 (UTC) Received: by gyg4 with SMTP id 4so666953gyg.13 for ; Thu, 23 Sep 2010 05:58:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:date:message-id :subject:from:to:content-type; bh=Ol6SQ0yxMCyUaUfDJNW88PiN/ZtDNzIBcvy2ssy0X/s=; b=pTjRngsDqox1RKbw4720mJnqsqh2LDT0SsZF4zV+lJIye76tH+XU7R/ajEOzabi6T9 rx7ACEz39PFrtQXb2vvpZoxfSkJ5oZmCY6EPJ0NcUmLkTBertJHO0ZcUCnCFXWknrIak EklngIx8mlX+sQpJLuatF/Ga72eWQlf0at1L8= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:content-type; b=B/Hodl+fIwchwiffYV/3+Wb0eKoln1v0hkmtr2xmvHcpcUA5AcEBJodeOcqPiZs5Ug 0gGBz/B+40olkjdktdE2vhY6mIgFboilXvt3iH+i5IeTye7VhRqqqz3CkE15kAhx6o9y lsEGpWdQ1xomfOPH4kRi9WDYUdB03J7UPcvow= MIME-Version: 1.0 Received: by 10.220.58.129 with SMTP id g1mr705706vch.218.1285244945073; Thu, 23 Sep 2010 05:29:05 -0700 (PDT) Received: by 10.220.195.68 with HTTP; Thu, 23 Sep 2010 05:28:53 -0700 (PDT) Date: Thu, 23 Sep 2010 17:58:53 +0530 Message-ID: From: Samir Desai To: zfs-devel@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: zfs test suite X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Sep 2010 12:58:19 -0000 hi We have checked in all the code and test suite in to perforce the branch is //depot/user/samir/zfs, this is a branch of //depot/projects/zfs/stable/8 Once you checkout, go to the zfs directory # ls zfs_stable_8/zfs README build-test.sh build.sh patch-kernel.sh please read README here, it has all the information necessary to setup the freebsd system, building and installing zfs kernel/command and building/running zfs test suite regards Samir From owner-zfs-devel@FreeBSD.ORG Wed Sep 29 09:56:37 2010 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id EA6E8106566C; Wed, 29 Sep 2010 09:56:37 +0000 (UTC) (envelope-from avg@freebsd.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id D03798FC12; Wed, 29 Sep 2010 09:56:36 +0000 (UTC) Received: from porto.topspin.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id MAA23405; Wed, 29 Sep 2010 12:56:35 +0300 (EEST) (envelope-from avg@freebsd.org) Received: from localhost.topspin.kiev.ua ([127.0.0.1]) by porto.topspin.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1P0tOt-0003L1-Bz; Wed, 29 Sep 2010 12:56:35 +0300 Message-ID: <4CA30D53.5080706@freebsd.org> Date: Wed, 29 Sep 2010 12:56:35 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.2.9) Gecko/20100918 Lightning/1.0b2 Thunderbird/3.1.4 MIME-Version: 1.0 To: zfs-devel@freebsd.org, Pawel Jakub Dawidek X-Enigmail-Version: 1.1.2 Content-Type: text/plain; charset=X-VIET-VPS Content-Transfer-Encoding: 7bit Cc: Subject: adjust kmem_size for large vm_kmem_size X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 29 Sep 2010 09:56:38 -0000 We use kmem_size() function in ZFS ARC sizing code to determine values like arc_c_min and arc_c_mac. kmem_size() currently returns vm_kmem_size. OpenSolaris uses 'physmem' value for the same purpose. I guess that we use vm_kmem_size because typically it is (much) smaller than amount of physical memory. On the other hand, it is possible to set vm_kmem_size to value larger than physical memory (up to 2x is allowed). Sometimes it is even advisable to do so to work around kernel KVA space fragmentation. I think that we should use smaller of the two parameters when deciding ARC size. The patch below tries to implement that logic. What do you think? --- a/sys/cddl/compat/opensolaris/kern/opensolaris_kmem.c +++ b/sys/cddl/compat/opensolaris/kern/opensolaris_kmem.c @@ -115,8 +115,14 @@ zfs_kmem_free(void *buf, size_t size __unused) uint64_t kmem_size(void) { + static uint64_t kmem_size; - return ((uint64_t)vm_kmem_size); + if (kmem_size == 0) { + kmem_size = (uint64_t)cnt.v_page_count * PAGE_SIZE; + if (kmem_size > vm_kmem_size) + kmem_size = vm_kmem_size; + } + return (kmem_size); } uint64_t Perhaps, this should be split into two functions: one that sets kmem_size at appropriate moment and one that simply returns its value. -- Andriy Gapon From owner-zfs-devel@FreeBSD.ORG Thu Oct 7 17:30:36 2010 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id DA3BB106567A; Thu, 7 Oct 2010 17:30:36 +0000 (UTC) (envelope-from avg@freebsd.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id C587C8FC16; Thu, 7 Oct 2010 17:30:35 +0000 (UTC) Received: from odyssey.starpoint.kiev.ua (alpha-e.starpoint.kiev.ua [212.40.38.101]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id UAA18206; Thu, 07 Oct 2010 20:30:34 +0300 (EEST) (envelope-from avg@freebsd.org) Message-ID: <4CAE03B9.1060504@freebsd.org> Date: Thu, 07 Oct 2010 20:30:33 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.2.9) Gecko/20100920 Lightning/1.0b2 Thunderbird/3.1.4 MIME-Version: 1.0 To: zfs-devel@freebsd.org References: <4CA30D53.5080706@freebsd.org> In-Reply-To: <4CA30D53.5080706@freebsd.org> X-Enigmail-Version: 1.1.2 Content-Type: text/plain; charset=x-viet-vps Content-Transfer-Encoding: 7bit Cc: Subject: Re: adjust kmem_size for large vm_kmem_size X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 Oct 2010 17:30:36 -0000 on 29/09/2010 12:56 Andriy Gapon said the following: > > We use kmem_size() function in ZFS ARC sizing code to determine values like > arc_c_min and arc_c_mac. kmem_size() currently returns vm_kmem_size. > OpenSolaris uses 'physmem' value for the same purpose. > > I guess that we use vm_kmem_size because typically it is (much) smaller than > amount of physical memory. > On the other hand, it is possible to set vm_kmem_size to value larger than > physical memory (up to 2x is allowed). Sometimes it is even advisable to do so > to work around kernel KVA space fragmentation. > > I think that we should use smaller of the two parameters when deciding ARC size. > The patch below tries to implement that logic. [snip] > Perhaps, this should be split into two functions: one that sets kmem_size at > appropriate moment and one that simply returns its value. Updated patch: http://people.freebsd.org/~avg/osol-kmem_size.diff I am going to commit it if no one objects. -- Andriy Gapon From owner-zfs-devel@FreeBSD.ORG Thu Oct 7 17:33:14 2010 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7D0A9106564A for ; Thu, 7 Oct 2010 17:33:14 +0000 (UTC) (envelope-from avg@freebsd.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id AA65A8FC08 for ; Thu, 7 Oct 2010 17:33:12 +0000 (UTC) Received: from odyssey.starpoint.kiev.ua (alpha-e.starpoint.kiev.ua [212.40.38.101]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id UAA18456; Thu, 07 Oct 2010 20:33:11 +0300 (EEST) (envelope-from avg@freebsd.org) Message-ID: <4CAE0457.8040900@freebsd.org> Date: Thu, 07 Oct 2010 20:33:11 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.2.9) Gecko/20100920 Lightning/1.0b2 Thunderbird/3.1.4 MIME-Version: 1.0 To: zfs-devel@freebsd.org, hackers@freebsd.org X-Enigmail-Version: 1.1.2 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Cc: Subject: zfs: ru_inblock/ru_oublock accounting X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 Oct 2010 17:33:14 -0000 A simple, probably incomplete and perhaps incorrect patch for ru_inblock/ru_oublock accounting in zfs: http://people.freebsd.org/~avg/zfs-ru.diff Still quite better than nothing. P.S. This is about top -m io. -- Andriy Gapon From owner-zfs-devel@FreeBSD.ORG Thu Oct 7 18:05:36 2010 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B2D1D106566B for ; Thu, 7 Oct 2010 18:05:36 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (60.wheelsystems.com [83.12.187.60]) by mx1.freebsd.org (Postfix) with ESMTP id 6021E8FC15 for ; Thu, 7 Oct 2010 18:05:32 +0000 (UTC) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id 8866045CAC; Thu, 7 Oct 2010 20:05:31 +0200 (CEST) Received: from localhost (chello089073192049.chello.pl [89.73.192.49]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id 04F5945C98 for ; Thu, 7 Oct 2010 20:05:25 +0200 (CEST) Date: Thu, 7 Oct 2010 20:05:01 +0200 From: Pawel Jakub Dawidek To: zfs-devel@FreeBSD.org Message-ID: <20101007180501.GA1733@garage.freebsd.pl> References: <4CAE0457.8040900@freebsd.org> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="LZvS9be/3tNcYl/X" Content-Disposition: inline In-Reply-To: <4CAE0457.8040900@freebsd.org> User-Agent: Mutt/1.4.2.3i X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 9.0-CURRENT amd64 X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-2.6 required=4.5 tests=BAYES_00 autolearn=ham version=3.0.4 Cc: Subject: Re: zfs: ru_inblock/ru_oublock accounting X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 Oct 2010 18:05:36 -0000 --LZvS9be/3tNcYl/X Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Thu, Oct 07, 2010 at 08:33:11PM +0300, Andriy Gapon wrote: [...] Picking up random e-mail, I hope Andriy doesn't mind. Please do not cross-post to zfs-devel@ and public mailing lists, so I don't have to moderate incoming responses, as zfs-devel@ is invitation-only;) BCCing zfs-devel@ is probably the best choice. --=20 Pawel Jakub Dawidek http://www.wheelsystems.com pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --LZvS9be/3tNcYl/X Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (FreeBSD) iEYEARECAAYFAkyuC80ACgkQForvXbEpPzQo5gCgl6Ptzvq06xRlK6PNTsUrm2N/ pH8AoLIzSSx8vcCo/GBhrJT5gHM41a8Q =JtWu -----END PGP SIGNATURE----- --LZvS9be/3tNcYl/X-- From owner-zfs-devel@FreeBSD.ORG Fri Oct 8 08:23:31 2010 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C0F811065672; Fri, 8 Oct 2010 08:23:31 +0000 (UTC) (envelope-from avg@freebsd.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id BA0FA8FC2B; Fri, 8 Oct 2010 08:23:30 +0000 (UTC) Received: from porto.topspin.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id LAA01056; Fri, 08 Oct 2010 11:23:29 +0300 (EEST) (envelope-from avg@freebsd.org) Received: from localhost.topspin.kiev.ua ([127.0.0.1]) by porto.topspin.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1P48Ej-000AFi-8D; Fri, 08 Oct 2010 11:23:29 +0300 Message-ID: <4CAED500.3090001@freebsd.org> Date: Fri, 08 Oct 2010 11:23:28 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.2.9) Gecko/20100918 Lightning/1.0b2 Thunderbird/3.1.4 MIME-Version: 1.0 To: hackers@freebsd.org References: <4CAE0457.8040900@freebsd.org> In-Reply-To: <4CAE0457.8040900@freebsd.org> X-Enigmail-Version: 1.1.2 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Mailman-Approved-At: Fri, 08 Oct 2010 11:19:28 +0000 Cc: Subject: Re: zfs: ru_inblock/ru_oublock accounting X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 Oct 2010 08:23:31 -0000 on 07/10/2010 20:33 Andriy Gapon said the following: > > A simple, probably incomplete and perhaps incorrect patch for > ru_inblock/ru_oublock accounting in zfs: > http://people.freebsd.org/~avg/zfs-ru.diff I've updated the patch at the same location. Thanks to "swell.k" for pointing out that the changes are applicable in kernel only. > Still quite better than nothing. > > P.S. > This is about top -m io. -- Andriy Gapon From owner-zfs-devel@FreeBSD.ORG Sun Oct 10 08:38:18 2010 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9F31C106564A for ; Sun, 10 Oct 2010 08:38:18 +0000 (UTC) (envelope-from avg@freebsd.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id D0C898FC08 for ; Sun, 10 Oct 2010 08:38:17 +0000 (UTC) Received: from porto.topspin.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id LAA05547 for ; Sun, 10 Oct 2010 11:38:16 +0300 (EEST) (envelope-from avg@freebsd.org) Received: from localhost.topspin.kiev.ua ([127.0.0.1]) by porto.topspin.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1P4rQ8-000HTZ-5M for zfs-devel@freebsd.org; Sun, 10 Oct 2010 11:38:16 +0300 Message-ID: <4CB17B77.7070509@freebsd.org> Date: Sun, 10 Oct 2010 11:38:15 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.2.9) Gecko/20100918 Lightning/1.0b2 Thunderbird/3.1.4 MIME-Version: 1.0 To: zfs-devel@freebsd.org X-Enigmail-Version: 1.1.2 Content-Type: text/plain; charset=X-VIET-VPS Content-Transfer-Encoding: 7bit Subject: zfs_getpages X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 10 Oct 2010 08:38:18 -0000 Guys, could you please review and or test the following addition to FreeBSD ZFS? http://people.freebsd.org/~avg/zfs-getpages.diff Does it improve anything for you? Is it something worth committing? -- Andriy Gapon From owner-zfs-devel@FreeBSD.ORG Mon Oct 11 10:34:54 2010 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 149641065670; Mon, 11 Oct 2010 10:34:54 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from mail.vx.sk (mail.vx.sk [IPv6:2a01:4f8:100:1043::3]) by mx1.freebsd.org (Postfix) with ESMTP id C89FB8FC0C; Mon, 11 Oct 2010 10:34:53 +0000 (UTC) Received: from core.vx.sk (localhost [127.0.0.1]) by mail.vx.sk (Postfix) with ESMTP id 3233512A553; Mon, 11 Oct 2010 12:34:53 +0200 (CEST) X-Virus-Scanned: amavisd-new at mail.vx.sk Received: from mail.vx.sk ([127.0.0.1]) by core.vx.sk (mail.vx.sk [127.0.0.1]) (amavisd-new, port 10024) with LMTP id 9WqyumyIgZY7; Mon, 11 Oct 2010 12:34:51 +0200 (CEST) Received: from [10.0.3.3] (188-167-67-67.dynamic.chello.sk [188.167.67.67]) by mail.vx.sk (Postfix) with ESMTPSA id 67E1012A547; Mon, 11 Oct 2010 12:34:51 +0200 (CEST) Message-ID: <4CB2E84A.2000008@FreeBSD.org> Date: Mon, 11 Oct 2010 12:34:50 +0200 From: Martin Matuska User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; sk; rv:1.8.1.23) Gecko/20090812 Lightning/0.9 Thunderbird/2.0.0.23 Mnenhy/0.7.5.0 MIME-Version: 1.0 To: Andriy Gapon References: <4CB17B77.7070509@freebsd.org> In-Reply-To: <4CB17B77.7070509@freebsd.org> X-Enigmail-Version: 1.1.1 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: zfs-devel@freebsd.org Subject: Re: zfs_getpages X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 Oct 2010 10:34:54 -0000 As of this code: + for (i = 0; i < pcount; i++) { + if (i != reqpage) { + vm_page_lock(m[i]); + vm_page_free(m[i]); + vm_page_unlock(m[i]); + } + } Do you think locking a page is sufficient to protect against a race? As to our reference implementation it should be more like this: + vm_page_lock_queues(); + for (i = 0; i < pcount; i++) { + if (i != reqpage) { + vm_page_free(m[i]); + } + } + vm_page_unlock_queues(); Cheers, mm Dňa 10. 10. 2010 10:38, Andriy Gapon wrote / napísal(a): > > Guys, > > could you please review and or test the following addition to FreeBSD ZFS? > http://people.freebsd.org/~avg/zfs-getpages.diff > > Does it improve anything for you? > Is it something worth committing? > From owner-zfs-devel@FreeBSD.ORG Mon Oct 11 11:03:47 2010 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B140A10656A3; Mon, 11 Oct 2010 11:03:47 +0000 (UTC) (envelope-from avg@freebsd.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id C8B378FC25; Mon, 11 Oct 2010 11:03:46 +0000 (UTC) Received: from porto.topspin.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id OAA21822; Mon, 11 Oct 2010 14:03:45 +0300 (EEST) (envelope-from avg@freebsd.org) Received: from localhost.topspin.kiev.ua ([127.0.0.1]) by porto.topspin.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1P5GAT-000KaJ-6G; Mon, 11 Oct 2010 14:03:45 +0300 Message-ID: <4CB2EF10.50601@freebsd.org> Date: Mon, 11 Oct 2010 14:03:44 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.2.9) Gecko/20100918 Lightning/1.0b2 Thunderbird/3.1.4 MIME-Version: 1.0 To: Martin Matuska References: <4CB17B77.7070509@freebsd.org> <4CB2E84A.2000008@FreeBSD.org> In-Reply-To: <4CB2E84A.2000008@FreeBSD.org> X-Enigmail-Version: 1.1.2 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: zfs-devel@freebsd.org Subject: Re: zfs_getpages X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 Oct 2010 11:03:47 -0000 on 11/10/2010 13:34 Martin Matuska said the following: > As of this code: > + for (i = 0; i < pcount; i++) { > + if (i != reqpage) { > + vm_page_lock(m[i]); > + vm_page_free(m[i]); > + vm_page_unlock(m[i]); > + } > + } > > Do you think locking a page is sufficient to protect against a race? What race? > As to our reference implementation it should be more like this: Which reference implementation would that be? > + vm_page_lock_queues(); > + for (i = 0; i < pcount; i++) { > + if (i != reqpage) { > + vm_page_free(m[i]); > + } > + } > + vm_page_unlock_queues(); I think that in head only page lock is required for vm_page_free(). Locking queues around the loop should not be required, it is done internally only for a short duration when it's needed. > Dňa 10. 10. 2010 10:38, Andriy Gapon wrote / napísal(a): >> >> Guys, >> >> could you please review and or test the following addition to FreeBSD ZFS? >> http://people.freebsd.org/~avg/zfs-getpages.diff >> >> Does it improve anything for you? >> Is it something worth committing? >> > _______________________________________________ > zfs-devel@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/zfs-devel > To unsubscribe, send any mail to "zfs-devel-unsubscribe@freebsd.org" -- Andriy Gapon From owner-zfs-devel@FreeBSD.ORG Mon Oct 11 11:25:49 2010 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8BB3C1065674; Mon, 11 Oct 2010 11:25:49 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from mail.vx.sk (mail.vx.sk [IPv6:2a01:4f8:100:1043::3]) by mx1.freebsd.org (Postfix) with ESMTP id 4B14C8FC1C; Mon, 11 Oct 2010 11:25:49 +0000 (UTC) Received: from core.vx.sk (localhost [127.0.0.1]) by mail.vx.sk (Postfix) with ESMTP id A8AAF12B95A; Mon, 11 Oct 2010 13:25:48 +0200 (CEST) X-Virus-Scanned: amavisd-new at mail.vx.sk Received: from mail.vx.sk ([127.0.0.1]) by core.vx.sk (mail.vx.sk [127.0.0.1]) (amavisd-new, port 10024) with LMTP id QHN29E1X5moo; Mon, 11 Oct 2010 13:25:47 +0200 (CEST) Received: from [10.0.3.3] (188-167-67-67.dynamic.chello.sk [188.167.67.67]) by mail.vx.sk (Postfix) with ESMTPSA id D8A6012B94E; Mon, 11 Oct 2010 13:25:46 +0200 (CEST) Message-ID: <4CB2F43A.7050503@FreeBSD.org> Date: Mon, 11 Oct 2010 13:25:46 +0200 From: Martin Matuska User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; sk; rv:1.8.1.23) Gecko/20090812 Lightning/0.9 Thunderbird/2.0.0.23 Mnenhy/0.7.5.0 MIME-Version: 1.0 To: Andriy Gapon References: <4CB17B77.7070509@freebsd.org> <4CB2E84A.2000008@FreeBSD.org> <4CB2EF10.50601@freebsd.org> In-Reply-To: <4CB2EF10.50601@freebsd.org> X-Enigmail-Version: 1.1.1 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Cc: zfs-devel@freebsd.org Subject: Re: zfs_getpages X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 Oct 2010 11:25:49 -0000 > Which reference implementation would that be? vop_stdgetpages() kern/vfs_default.c utilizing vnode_pager_generic_getpages() in sys/vm/vnode_pager.c We have four filesystems that implement their own vop_getpages and all use the queue locking: sys/fs/nfsclient/nfs_clbio.c sys/fs/nwfs/nwfs_io.c sys/fs/tmpfs/tmpfs_vnops.c sys/fs/smbfs/smbfs_io.c From owner-zfs-devel@FreeBSD.ORG Mon Oct 11 11:33:02 2010 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0881410656A8; Mon, 11 Oct 2010 11:33:02 +0000 (UTC) (envelope-from avg@freebsd.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 240BD8FC13; Mon, 11 Oct 2010 11:33:00 +0000 (UTC) Received: from porto.topspin.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id OAA22806; Mon, 11 Oct 2010 14:32:59 +0300 (EEST) (envelope-from avg@freebsd.org) Received: from localhost.topspin.kiev.ua ([127.0.0.1]) by porto.topspin.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1P5Gcl-000Kc3-EU; Mon, 11 Oct 2010 14:32:59 +0300 Message-ID: <4CB2F5EA.2020908@freebsd.org> Date: Mon, 11 Oct 2010 14:32:58 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.2.9) Gecko/20100918 Lightning/1.0b2 Thunderbird/3.1.4 MIME-Version: 1.0 To: Martin Matuska References: <4CB17B77.7070509@freebsd.org> <4CB2E84A.2000008@FreeBSD.org> <4CB2EF10.50601@freebsd.org> <4CB2F43A.7050503@FreeBSD.org> In-Reply-To: <4CB2F43A.7050503@FreeBSD.org> X-Enigmail-Version: 1.1.2 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Cc: zfs-devel@freebsd.org Subject: Re: zfs_getpages X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 Oct 2010 11:33:02 -0000 on 11/10/2010 14:25 Martin Matuska said the following: >> Which reference implementation would that be? > vop_stdgetpages() kern/vfs_default.c > utilizing vnode_pager_generic_getpages() in sys/vm/vnode_pager.c > > We have four filesystems that implement their own vop_getpages and all > use the queue locking: I am sorry, but I don't see that in my copy of FreeBSD source tree, head branch. > sys/fs/nfsclient/nfs_clbio.c > sys/fs/nwfs/nwfs_io.c > sys/fs/tmpfs/tmpfs_vnops.c > sys/fs/smbfs/smbfs_io.c -- Andriy Gapon From owner-zfs-devel@FreeBSD.ORG Mon Oct 11 11:53:18 2010 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 40669106567A; Mon, 11 Oct 2010 11:53:18 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from mail.vx.sk (mail.vx.sk [IPv6:2a01:4f8:100:1043::3]) by mx1.freebsd.org (Postfix) with ESMTP id F26BD8FC2A; Mon, 11 Oct 2010 11:53:17 +0000 (UTC) Received: from core.vx.sk (localhost [127.0.0.1]) by mail.vx.sk (Postfix) with ESMTP id 51B9B1194E5; Mon, 11 Oct 2010 13:53:17 +0200 (CEST) X-Virus-Scanned: amavisd-new at mail.vx.sk Received: from mail.vx.sk ([127.0.0.1]) by core.vx.sk (mail.vx.sk [127.0.0.1]) (amavisd-new, port 10024) with LMTP id iN4PfNpbZci9; Mon, 11 Oct 2010 13:53:15 +0200 (CEST) Received: from [10.0.3.3] (188-167-67-67.dynamic.chello.sk [188.167.67.67]) by mail.vx.sk (Postfix) with ESMTPSA id 77CA41194D7; Mon, 11 Oct 2010 13:53:15 +0200 (CEST) Message-ID: <4CB2FAAA.6060000@FreeBSD.org> Date: Mon, 11 Oct 2010 13:53:14 +0200 From: Martin Matuska User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; sk; rv:1.8.1.23) Gecko/20090812 Lightning/0.9 Thunderbird/2.0.0.23 Mnenhy/0.7.5.0 MIME-Version: 1.0 To: Andriy Gapon References: <4CB17B77.7070509@freebsd.org> <4CB2E84A.2000008@FreeBSD.org> <4CB2EF10.50601@freebsd.org> <4CB2F43A.7050503@FreeBSD.org> <4CB2F5EA.2020908@freebsd.org> In-Reply-To: <4CB2F5EA.2020908@freebsd.org> X-Enigmail-Version: 1.1.1 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: zfs-devel@freebsd.org Subject: Re: zfs_getpages X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 Oct 2010 11:53:18 -0000 I apologize, need probably more sleep after EuroBSDcon. I had a checkout of 8-STABLE into a wrong directory. Thx. Dňa 11. 10. 2010 13:32, Andriy Gapon wrote / napísal(a): > on 11/10/2010 14:25 Martin Matuska said the following: >>> Which reference implementation would that be? >> vop_stdgetpages() kern/vfs_default.c >> utilizing vnode_pager_generic_getpages() in sys/vm/vnode_pager.c >> >> We have four filesystems that implement their own vop_getpages and all >> use the queue locking: > > I am sorry, but I don't see that in my copy of FreeBSD source tree, head branch. > >> sys/fs/nfsclient/nfs_clbio.c >> sys/fs/nwfs/nwfs_io.c >> sys/fs/tmpfs/tmpfs_vnops.c >> sys/fs/smbfs/smbfs_io.c > > From owner-zfs-devel@FreeBSD.ORG Mon Oct 11 20:29:00 2010 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 24C87106566C for ; Mon, 11 Oct 2010 20:29:00 +0000 (UTC) (envelope-from delphij@delphij.net) Received: from tarsier.geekcn.org (tarsier.geekcn.org [IPv6:2001:470:a803::1]) by mx1.freebsd.org (Postfix) with ESMTP id 3DBA98FC08 for ; Mon, 11 Oct 2010 20:28:58 +0000 (UTC) Received: from mail.geekcn.org (tarsier.geekcn.org [211.166.10.233]) by tarsier.geekcn.org (Postfix) with ESMTP id 3FAD1A6A59D; Tue, 12 Oct 2010 04:28:52 +0800 (CST) X-Virus-Scanned: amavisd-new at geekcn.org Received: from tarsier.geekcn.org ([211.166.10.233]) by mail.geekcn.org (mail.geekcn.org [211.166.10.233]) (amavisd-new, port 10024) with LMTP id aSgbba03TEHZ; Tue, 12 Oct 2010 04:28:45 +0800 (CST) Received: from delta.delphij.net (drawbridge.ixsystems.com [206.40.55.65]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by tarsier.geekcn.org (Postfix) with ESMTPSA id A14A2A6A297; Tue, 12 Oct 2010 04:28:42 +0800 (CST) DomainKey-Signature: a=rsa-sha1; s=default; d=delphij.net; c=nofws; q=dns; h=message-id:date:from:reply-to:organization:user-agent: mime-version:to:subject:x-enigmail-version:openpgp:content-type; b=AeNHZdwhGghaooYrOfaeq8TyHRFjnOE4eUjghCtAjTMwFOEChtb4s3JoB12Zv36CW sYpbjmVPKYmlwTO+RNztA== Message-ID: <4CB37377.9060202@delphij.net> Date: Mon, 11 Oct 2010 13:28:39 -0700 From: Xin LI Organization: The FreeBSD Project User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.1.12) Gecko/20100920 Thunderbird/3.0.8 ThunderBrowse/3.3.2 MIME-Version: 1.0 To: zfs-devel@FreeBSD.org X-Enigmail-Version: 1.0.1 OpenPGP: id=3FCA37C1; url=http://www.delphij.net/delphij.asc Content-Type: multipart/mixed; boundary="------------080904090108030003080008" Cc: Subject: ZFS deadlock on 9-CURRENT with NFS X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: d@delphij.net List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 Oct 2010 20:29:00 -0000 This is a multi-part message in MIME format. --------------080904090108030003080008 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 Hi, This is a user's report, a first glance shows that 'nfsd' stuck in 'zfs', which happens from time to time. He was uploading a big file to the server, which might be related. We are still trying to find out a way to provoke the issue more reliably. The ZFS pool have some data errors on it. I have grabbed the output of procstat -k output from his system when it happens, just FYI. The user is willing to help so please let me know if you need more data. Cheers, - -- Xin LI http://www.delphij.net/ FreeBSD - The Power to Serve! Live free or die -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.16 (FreeBSD) iQEcBAEBCAAGBQJMs3N2AAoJEATO+BI/yjfBOxQH/jLaRVADGIZpt3Qcp3hvgMIH 7yAbNAsIfUmVktyw4P5/o7EHnXk+o0avLgQMWYub/YoJDVeBL3h/QpjKEX6Z3fsb Zk2MvIrQhOT5ZZWBU9lJh/OSNtZRjCji8K130rriQcJLtAv00qem654i9w7tDHW5 sFVHnzcXu1e4g3cbuy0b8IFh4jBSpeOmsWRmXdM/cpDrGQqTtG/1tuqefBsAZj0g M4r3kZNflgpBIZtpFPepzVARvH9gc+k7V+ihjjcR0bXNEzP5X0eaSezbiQA6lydT BTi/coie8ODDOEkW2xO1ue//4k20xIkHADy0o0MoOZKP1JfydLUXL2sqzurxrps= =iDwb -----END PGP SIGNATURE----- --------------080904090108030003080008 Content-Type: text/plain; name="procstat.txt" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="procstat.txt" IyBwcm9jc3RhdCAtayAyNjc1IDI0MzYgMTA4MSAxOCA1IDAKICBQSUQgICAgVElEIENPTU0g ICAgICAgICAgICAgVEROQU1FICAgICAgICAgICBLU1RBQ0sgICAgICAgICAgICAgICAgICAg ICAgIAogMjY3NSAxMDAyOTIgc2Z0cC1zZXJ2ZXIgICAgICAtICAgICAgICAgICAgICAgIG1p X3N3aXRjaCBzbGVlcHFfd2FpdCBfY3Zfd2FpdCB0eGdfd2FpdF9vcGVuIHpmc19mcmVlYnNk X3dyaXRlIFZPUF9XUklURV9BUFYgdm5fd3JpdGUgZG9maWxld3JpdGUga2Vybl93cml0ZXYg d3JpdGUgc3lzY2FsbGVudGVyIHN5c2NhbGwgWGZhc3Rfc3lzY2FsbCAKIDI0MzYgMTAwMzM3 IGN2c3VwICAgICAgICAgICAgLSAgICAgICAgICAgICAgICBtaV9zd2l0Y2ggc2xlZXBxX3dh aXQgX2N2X3dhaXQgemlvX3dhaXQgZGJ1Zl9yZWFkIGRtdV9idWZfaG9sZCB6YXBfbG9ja2Rp ciB6YXBfbG9va3VwX25vcm0gemFwX2xvb2t1cCB6ZnNfZGlyZW50X2xvY2sgemZzX2Rpcmxv b2sgemZzX2xvb2t1cCB6ZnNfZnJlZWJzZF9sb29rdXAgdmZzX2NhY2hlX2xvb2t1cCBWT1Bf TE9PS1VQX0FQViBsb29rdXAgbmFtZWkga2Vybl9zdGF0YXRfdm5ob29rIAogMTA4MSAxMDAy OTkgbmZzZCAgICAgICAgICAgICBuZnNkOiBtYXN0ZXIgICAgIG1pX3N3aXRjaCBzbGVlcHFf d2FpdCBfY3Zfd2FpdCB6aW9fd2FpdCBkYnVmX3JlYWQgZGJ1Zl9maW5kYnAgZGJ1Zl9ob2xk X2ltcGwgZGJ1Zl9ob2xkIGRtdV9idWZfaG9sZCB6YXBfbG9ja2RpciB6YXBfbG9va3VwX25v cm0gemFwX2xvb2t1cCB6ZnNfZGlyZW50X2xvY2sgemZzX2Rpcmxvb2sgemZzX2xvb2t1cCB6 ZnNfZnJlZWJzZF9sb29rdXAgdmZzX2NhY2hlX2xvb2t1cCBWT1BfTE9PS1VQX0FQViAKIDEw ODEgMTAwMzE4IG5mc2QgICAgICAgICAgICAgbmZzZDogc2VydmljZSAgICBtaV9zd2l0Y2gg c2xlZXBxX3dhaXQgX19sb2NrbWdyX2FyZ3Mgdm9wX3N0ZGxvY2sgVk9QX0xPQ0sxX0FQViBf dm5fbG9jayB6ZnNfZmh0b3ZwIG5mc3Zub19maHRvdnAgbmZzZF9maHRvdnAgbmZzcnZkX2Rv cnBjIG5mc3N2Y19wcm9ncmFtIHN2Y19ydW5faW50ZXJuYWwgc3ZjX3RocmVhZF9zdGFydCBm b3JrX2V4aXQgZm9ya190cmFtcG9saW5lIAogMTA4MSAxMDAzMTkgbmZzZCAgICAgICAgICAg ICBuZnNkOiBzZXJ2aWNlICAgIG1pX3N3aXRjaCBzbGVlcHFfd2FpdCBfX2xvY2ttZ3JfYXJn cyB2b3Bfc3RkbG9jayBWT1BfTE9DSzFfQVBWIF92bl9sb2NrIHpmc19maHRvdnAgbmZzdm5v X2ZodG92cCBuZnNkX2ZodG92cCBuZnNydmRfZG9ycGMgbmZzc3ZjX3Byb2dyYW0gc3ZjX3J1 bl9pbnRlcm5hbCBzdmNfdGhyZWFkX3N0YXJ0IGZvcmtfZXhpdCBmb3JrX3RyYW1wb2xpbmUg CiAxMDgxIDEwMDMyMCBuZnNkICAgICAgICAgICAgIG5mc2Q6IHNlcnZpY2UgICAgbWlfc3dp dGNoIHNsZWVwcV93YWl0IF9fbG9ja21ncl9hcmdzIHZvcF9zdGRsb2NrIFZPUF9MT0NLMV9B UFYgX3ZuX2xvY2sgemZzX2ZodG92cCBuZnN2bm9fZmh0b3ZwIG5mc2RfZmh0b3ZwIG5mc3J2 ZF9kb3JwYyBuZnNzdmNfcHJvZ3JhbSBzdmNfcnVuX2ludGVybmFsIHN2Y190aHJlYWRfc3Rh cnQgZm9ya19leGl0IGZvcmtfdHJhbXBvbGluZSAKICAgMTggMTAwMDgxIHN5bmNlciAgICAg ICAgICAgLSAgICAgICAgICAgICAgICBtaV9zd2l0Y2ggc2xlZXBxX3dhaXQgX2N2X3dhaXQg emlvX3dhaXQgemlsX2NvbW1pdCB6ZnNfc3luYyBzeW5jX2ZzeW5jIHN5bmNfdm5vZGUgc2No ZWRfc3luYyBmb3JrX2V4aXQgZm9ya190cmFtcG9saW5lIAogICAgNSAxMDAwNzEgemZza2Vy biAgICAgICAgICBhcmNfcmVjbGFpbV90aHJlIG1pX3N3aXRjaCBzbGVlcHFfdGltZWR3YWl0 IF9jdl90aW1lZHdhaXQgYXJjX3JlY2xhaW1fdGhyZWFkIGZvcmtfZXhpdCBmb3JrX3RyYW1w b2xpbmUgCiAgICA1IDEwMDA3MiB6ZnNrZXJuICAgICAgICAgIGwyYXJjX2ZlZWRfdGhyZWEg bWlfc3dpdGNoIHNsZWVwcV90aW1lZHdhaXQgX2N2X3RpbWVkd2FpdCBsMmFyY19mZWVkX3Ro cmVhZCBmb3JrX2V4aXQgZm9ya190cmFtcG9saW5lIAogICAgNSAxMDAxMjUgemZza2VybiAg ICAgICAgICB0eGdfdGhyZWFkX2VudGVyIG1pX3N3aXRjaCBzbGVlcHFfd2FpdCBfY3Zfd2Fp dCB0eGdfdGhyZWFkX3dhaXQgdHhnX3F1aWVzY2VfdGhyZWFkIGZvcmtfZXhpdCBmb3JrX3Ry YW1wb2xpbmUgCiAgICA1IDEwMDEyNiB6ZnNrZXJuICAgICAgICAgIHR4Z190aHJlYWRfZW50 ZXIgbWlfc3dpdGNoIHNsZWVwcV90aW1lZHdhaXQgX2N2X3RpbWVkd2FpdCB0eGdfdGhyZWFk X3dhaXQgdHhnX3N5bmNfdGhyZWFkIGZvcmtfZXhpdCBmb3JrX3RyYW1wb2xpbmUgCiAgICA1 IDEwMDE4MiB6ZnNrZXJuICAgICAgICAgIHR4Z190aHJlYWRfZW50ZXIgbWlfc3dpdGNoIHNs ZWVwcV93YWl0IF9jdl93YWl0IHR4Z19xdWllc2NlX3RocmVhZCBmb3JrX2V4aXQgZm9ya190 cmFtcG9saW5lIAogICAgNSAxMDAxODMgemZza2VybiAgICAgICAgICB0eGdfdGhyZWFkX2Vu dGVyIG1pX3N3aXRjaCBzbGVlcHFfd2FpdCBfY3Zfd2FpdCB6aW9fd2FpdCBkc2xfcG9vbF9z eW5jIHNwYV9zeW5jIHR4Z19zeW5jX3RocmVhZCBmb3JrX2V4aXQgZm9ya190cmFtcG9saW5l IAogICAgMCAxMDAwMDAga2VybmVsICAgICAgICAgICBzd2FwcGVyICAgICAgICAgIG1pX3N3 aXRjaCBzbGVlcHFfdGltZWR3YWl0IF9zbGVlcCBzY2hlZHVsZXIgbWlfc3RhcnR1cCBidGV4 dCAKICAgIDAgMTAwMDE2IGtlcm5lbCAgICAgICAgICAgZmlybXdhcmUgdGFza3EgICBtaV9z d2l0Y2ggc2xlZXBxX3dhaXQgX3NsZWVwIHRhc2txdWV1ZV90aHJlYWRfbG9vcCBmb3JrX2V4 aXQgZm9ya190cmFtcG9saW5lIAogICAgMCAxMDAwMjEga2VybmVsICAgICAgICAgICBhY3Bp X3Rhc2tfMCAgICAgIG1pX3N3aXRjaCBzbGVlcHFfd2FpdCBtc2xlZXBfc3BpbiB0YXNrcXVl dWVfdGhyZWFkX2xvb3AgZm9ya19leGl0IGZvcmtfdHJhbXBvbGluZSAKICAgIDAgMTAwMDIy IGtlcm5lbCAgICAgICAgICAgYWNwaV90YXNrXzEgICAgICBtaV9zd2l0Y2ggc2xlZXBxX3dh aXQgbXNsZWVwX3NwaW4gdGFza3F1ZXVlX3RocmVhZF9sb29wIGZvcmtfZXhpdCBmb3JrX3Ry YW1wb2xpbmUgCiAgICAwIDEwMDAyMyBrZXJuZWwgICAgICAgICAgIGFjcGlfdGFza18yICAg ICAgbWlfc3dpdGNoIHNsZWVwcV93YWl0IG1zbGVlcF9zcGluIHRhc2txdWV1ZV90aHJlYWRf bG9vcCBmb3JrX2V4aXQgZm9ya190cmFtcG9saW5lIAogICAgMCAxMDAwMjQga2VybmVsICAg ICAgICAgICBrcXVldWUgdGFza3EgICAgIG1pX3N3aXRjaCBzbGVlcHFfd2FpdCBfc2xlZXAg dGFza3F1ZXVlX3RocmVhZF9sb29wIGZvcmtfZXhpdCBmb3JrX3RyYW1wb2xpbmUgCiAgICAw IDEwMDAyNiBrZXJuZWwgICAgICAgICAgIHRocmVhZCB0YXNrcSAgICAgbWlfc3dpdGNoIHNs ZWVwcV93YWl0IF9zbGVlcCB0YXNrcXVldWVfdGhyZWFkX2xvb3AgZm9ya19leGl0IGZvcmtf dHJhbXBvbGluZSAKICAgIDAgMTAwMDU5IGtlcm5lbCAgICAgICAgICAgZW0wIHRhc2txICAg ICAgICBtaV9zd2l0Y2ggc2xlZXBxX3dhaXQgbXNsZWVwX3NwaW4gdGFza3F1ZXVlX3RocmVh ZF9sb29wIGZvcmtfZXhpdCBmb3JrX3RyYW1wb2xpbmUgCiAgICAwIDEwMDA2NyBrZXJuZWwg ICAgICAgICAgIHN5c3RlbV90YXNrcV8wICAgbWlfc3dpdGNoIHNsZWVwcV93YWl0IF9zbGVl cCB0YXNrcXVldWVfdGhyZWFkX2xvb3AgZm9ya19leGl0IGZvcmtfdHJhbXBvbGluZSAKICAg IDAgMTAwMDY4IGtlcm5lbCAgICAgICAgICAgc3lzdGVtX3Rhc2txXzEgICBtaV9zd2l0Y2gg c2xlZXBxX3dhaXQgX3NsZWVwIHRhc2txdWV1ZV90aHJlYWRfbG9vcCBmb3JrX2V4aXQgZm9y a190cmFtcG9saW5lIAogICAgMCAxMDAwNjkga2VybmVsICAgICAgICAgICBzeXN0ZW1fdGFz a3FfMiAgIG1pX3N3aXRjaCBzbGVlcHFfd2FpdCBfc2xlZXAgdGFza3F1ZXVlX3RocmVhZF9s b29wIGZvcmtfZXhpdCBmb3JrX3RyYW1wb2xpbmUgCiAgICAwIDEwMDA3MCBrZXJuZWwgICAg ICAgICAgIHN5c3RlbV90YXNrcV8zICAgbWlfc3dpdGNoIHNsZWVwcV93YWl0IF9zbGVlcCB0 YXNrcXVldWVfdGhyZWFkX2xvb3AgZm9ya19leGl0IGZvcmtfdHJhbXBvbGluZSAKICAgIDAg MTAwMDczIGtlcm5lbCAgICAgICAgICAgZGVhZGxrcmVzICAgICAgICBtaV9zd2l0Y2ggc2xl ZXBxX3RpbWVkd2FpdCBfc2xlZXAgZGVhZGxrcmVzIGZvcmtfZXhpdCBmb3JrX3RyYW1wb2xp bmUgCiAgICAwIDEwMDA4NCBrZXJuZWwgICAgICAgICAgIHppb19udWxsX2lzc3VlICAgbWlf c3dpdGNoIHNsZWVwcV93YWl0IF9zbGVlcCB0YXNrcXVldWVfdGhyZWFkX2xvb3AgZm9ya19l eGl0IGZvcmtfdHJhbXBvbGluZSAKICAgIDAgMTAwMDg1IGtlcm5lbCAgICAgICAgICAgemlv X251bGxfaW50ciAgICBtaV9zd2l0Y2ggc2xlZXBxX3dhaXQgX3NsZWVwIHRhc2txdWV1ZV90 aHJlYWRfbG9vcCBmb3JrX2V4aXQgZm9ya190cmFtcG9saW5lIAogICAgMCAxMDAwODYga2Vy bmVsICAgICAgICAgICB6aW9fcmVhZF9pc3N1ZV8wIG1pX3N3aXRjaCBzbGVlcHFfd2FpdCBf c2xlZXAgdGFza3F1ZXVlX3RocmVhZF9sb29wIGZvcmtfZXhpdCBmb3JrX3RyYW1wb2xpbmUg CiAgICAwIDEwMDA4NyBrZXJuZWwgICAgICAgICAgIHppb19yZWFkX2lzc3VlXzEgbWlfc3dp dGNoIHNsZWVwcV93YWl0IF9zbGVlcCB0YXNrcXVldWVfdGhyZWFkX2xvb3AgZm9ya19leGl0 IGZvcmtfdHJhbXBvbGluZSAKICAgIDAgMTAwMDg4IGtlcm5lbCAgICAgICAgICAgemlvX3Jl YWRfaXNzdWVfMiBtaV9zd2l0Y2ggc2xlZXBxX3dhaXQgX3NsZWVwIHRhc2txdWV1ZV90aHJl YWRfbG9vcCBmb3JrX2V4aXQgZm9ya190cmFtcG9saW5lIAogICAgMCAxMDAwODkga2VybmVs ICAgICAgICAgICB6aW9fcmVhZF9pc3N1ZV8zIG1pX3N3aXRjaCBzbGVlcHFfd2FpdCBfc2xl ZXAgdGFza3F1ZXVlX3RocmVhZF9sb29wIGZvcmtfZXhpdCBmb3JrX3RyYW1wb2xpbmUgCiAg ICAwIDEwMDA5MCBrZXJuZWwgICAgICAgICAgIHppb19yZWFkX2lzc3VlXzQgbWlfc3dpdGNo IHNsZWVwcV93YWl0IF9zbGVlcCB0YXNrcXVldWVfdGhyZWFkX2xvb3AgZm9ya19leGl0IGZv cmtfdHJhbXBvbGluZSAKICAgIDAgMTAwMDkxIGtlcm5lbCAgICAgICAgICAgemlvX3JlYWRf aXNzdWVfNSBtaV9zd2l0Y2ggc2xlZXBxX3dhaXQgX3NsZWVwIHRhc2txdWV1ZV90aHJlYWRf bG9vcCBmb3JrX2V4aXQgZm9ya190cmFtcG9saW5lIAogICAgMCAxMDAwOTIga2VybmVsICAg ICAgICAgICB6aW9fcmVhZF9pc3N1ZV82IG1pX3N3aXRjaCBzbGVlcHFfd2FpdCBfc2xlZXAg dGFza3F1ZXVlX3RocmVhZF9sb29wIGZvcmtfZXhpdCBmb3JrX3RyYW1wb2xpbmUgCiAgICAw IDEwMDA5MyBrZXJuZWwgICAgICAgICAgIHppb19yZWFkX2lzc3VlXzcgbWlfc3dpdGNoIHNs ZWVwcV93YWl0IF9zbGVlcCB0YXNrcXVldWVfdGhyZWFkX2xvb3AgZm9ya19leGl0IGZvcmtf dHJhbXBvbGluZSAKICAgIDAgMTAwMDk0IGtlcm5lbCAgICAgICAgICAgemlvX3JlYWRfaW50 cl8wICBtaV9zd2l0Y2ggc2xlZXBxX3dhaXQgX3NsZWVwIHRhc2txdWV1ZV90aHJlYWRfbG9v cCBmb3JrX2V4aXQgZm9ya190cmFtcG9saW5lIAogICAgMCAxMDAwOTUga2VybmVsICAgICAg ICAgICB6aW9fcmVhZF9pbnRyXzEgIG1pX3N3aXRjaCBzbGVlcHFfd2FpdCBfc2xlZXAgdGFz a3F1ZXVlX3RocmVhZF9sb29wIGZvcmtfZXhpdCBmb3JrX3RyYW1wb2xpbmUgCiAgICAwIDEw MDA5NiBrZXJuZWwgICAgICAgICAgIHppb19yZWFkX2ludHJfMiAgbWlfc3dpdGNoIHNsZWVw cV93YWl0IF9zbGVlcCB0YXNrcXVldWVfdGhyZWFkX2xvb3AgZm9ya19leGl0IGZvcmtfdHJh bXBvbGluZSAKICAgIDAgMTAwMDk3IGtlcm5lbCAgICAgICAgICAgemlvX3dyaXRlX2lzc3Vl XyBtaV9zd2l0Y2ggc2xlZXBxX3dhaXQgX3NsZWVwIHRhc2txdWV1ZV90aHJlYWRfbG9vcCBm b3JrX2V4aXQgZm9ya190cmFtcG9saW5lIAogICAgMCAxMDAwOTgga2VybmVsICAgICAgICAg ICB6aW9fd3JpdGVfaXNzdWVfIG1pX3N3aXRjaCBzbGVlcHFfd2FpdCBfc2xlZXAgdGFza3F1 ZXVlX3RocmVhZF9sb29wIGZvcmtfZXhpdCBmb3JrX3RyYW1wb2xpbmUgCiAgICAwIDEwMDA5 OSBrZXJuZWwgICAgICAgICAgIHppb193cml0ZV9pc3N1ZV8gbWlfc3dpdGNoIHNsZWVwcV93 YWl0IF9zbGVlcCB0YXNrcXVldWVfdGhyZWFkX2xvb3AgZm9ya19leGl0IGZvcmtfdHJhbXBv bGluZSAKICAgIDAgMTAwMTAwIGtlcm5lbCAgICAgICAgICAgemlvX3dyaXRlX2lzc3VlXyBt aV9zd2l0Y2ggc2xlZXBxX3dhaXQgX3NsZWVwIHRhc2txdWV1ZV90aHJlYWRfbG9vcCBmb3Jr X2V4aXQgZm9ya190cmFtcG9saW5lIAogICAgMCAxMDAxMDEga2VybmVsICAgICAgICAgICB6 aW9fd3JpdGVfaXNzdWVfIG1pX3N3aXRjaCBzbGVlcHFfd2FpdCBfc2xlZXAgdGFza3F1ZXVl X3RocmVhZF9sb29wIGZvcmtfZXhpdCBmb3JrX3RyYW1wb2xpbmUgCiAgICAwIDEwMDEwMiBr ZXJuZWwgICAgICAgICAgIHppb193cml0ZV9pc3N1ZV8gbWlfc3dpdGNoIHNsZWVwcV93YWl0 IF9zbGVlcCB0YXNrcXVldWVfdGhyZWFkX2xvb3AgZm9ya19leGl0IGZvcmtfdHJhbXBvbGlu ZSAKICAgIDAgMTAwMTAzIGtlcm5lbCAgICAgICAgICAgemlvX3dyaXRlX2lzc3VlXyBtaV9z d2l0Y2ggc2xlZXBxX3dhaXQgX3NsZWVwIHRhc2txdWV1ZV90aHJlYWRfbG9vcCBmb3JrX2V4 aXQgZm9ya190cmFtcG9saW5lIAogICAgMCAxMDAxMDQga2VybmVsICAgICAgICAgICB6aW9f d3JpdGVfaXNzdWVfIG1pX3N3aXRjaCBzbGVlcHFfd2FpdCBfc2xlZXAgdGFza3F1ZXVlX3Ro cmVhZF9sb29wIGZvcmtfZXhpdCBmb3JrX3RyYW1wb2xpbmUgCiAgICAwIDEwMDEwNSBrZXJu ZWwgICAgICAgICAgIHppb193cml0ZV9pbnRyXzAgbWlfc3dpdGNoIHNsZWVwcV93YWl0IF9z bGVlcCB0YXNrcXVldWVfdGhyZWFkX2xvb3AgZm9ya19leGl0IGZvcmtfdHJhbXBvbGluZSAK ICAgIDAgMTAwMTA2IGtlcm5lbCAgICAgICAgICAgemlvX3dyaXRlX2ludHJfMSBtaV9zd2l0 Y2ggc2xlZXBxX3dhaXQgX3NsZWVwIHRhc2txdWV1ZV90aHJlYWRfbG9vcCBmb3JrX2V4aXQg Zm9ya190cmFtcG9saW5lIAogICAgMCAxMDAxMDcga2VybmVsICAgICAgICAgICB6aW9fd3Jp dGVfaW50cl8yIG1pX3N3aXRjaCBzbGVlcHFfd2FpdCBfc2xlZXAgdGFza3F1ZXVlX3RocmVh ZF9sb29wIGZvcmtfZXhpdCBmb3JrX3RyYW1wb2xpbmUgCiAgICAwIDEwMDEwOCBrZXJuZWwg ICAgICAgICAgIHppb193cml0ZV9pbnRyXzMgbWlfc3dpdGNoIHNsZWVwcV93YWl0IF9zbGVl cCB0YXNrcXVldWVfdGhyZWFkX2xvb3AgZm9ya19leGl0IGZvcmtfdHJhbXBvbGluZSAKICAg IDAgMTAwMTA5IGtlcm5lbCAgICAgICAgICAgemlvX3dyaXRlX2ludHJfNCBtaV9zd2l0Y2gg c2xlZXBxX3dhaXQgX3NsZWVwIHRhc2txdWV1ZV90aHJlYWRfbG9vcCBmb3JrX2V4aXQgZm9y a190cmFtcG9saW5lIAogICAgMCAxMDAxMTAga2VybmVsICAgICAgICAgICB6aW9fd3JpdGVf aW50cl81IG1pX3N3aXRjaCBzbGVlcHFfd2FpdCBfc2xlZXAgdGFza3F1ZXVlX3RocmVhZF9s b29wIGZvcmtfZXhpdCBmb3JrX3RyYW1wb2xpbmUgCiAgICAwIDEwMDExMSBrZXJuZWwgICAg ICAgICAgIHppb193cml0ZV9pbnRyXzYgbWlfc3dpdGNoIHNsZWVwcV93YWl0IF9zbGVlcCB0 YXNrcXVldWVfdGhyZWFkX2xvb3AgZm9ya19leGl0IGZvcmtfdHJhbXBvbGluZSAKICAgIDAg MTAwMTEyIGtlcm5lbCAgICAgICAgICAgemlvX3dyaXRlX2ludHJfNyBtaV9zd2l0Y2ggc2xl ZXBxX3dhaXQgX3NsZWVwIHRhc2txdWV1ZV90aHJlYWRfbG9vcCBmb3JrX2V4aXQgZm9ya190 cmFtcG9saW5lIAogICAgMCAxMDAxMTMga2VybmVsICAgICAgICAgICB6aW9fd3JpdGVfaW50 cl9oIG1pX3N3aXRjaCBzbGVlcHFfd2FpdCBfc2xlZXAgdGFza3F1ZXVlX3RocmVhZF9sb29w IGZvcmtfZXhpdCBmb3JrX3RyYW1wb2xpbmUgCiAgICAwIDEwMDExNCBrZXJuZWwgICAgICAg ICAgIHppb193cml0ZV9pbnRyX2ggbWlfc3dpdGNoIHNsZWVwcV93YWl0IF9zbGVlcCB0YXNr cXVldWVfdGhyZWFkX2xvb3AgZm9ya19leGl0IGZvcmtfdHJhbXBvbGluZSAKICAgIDAgMTAw MTE1IGtlcm5lbCAgICAgICAgICAgemlvX3dyaXRlX2ludHJfaCBtaV9zd2l0Y2ggc2xlZXBx X3dhaXQgX3NsZWVwIHRhc2txdWV1ZV90aHJlYWRfbG9vcCBmb3JrX2V4aXQgZm9ya190cmFt cG9saW5lIAogICAgMCAxMDAxMTYga2VybmVsICAgICAgICAgICB6aW9fd3JpdGVfaW50cl9o IG1pX3N3aXRjaCBzbGVlcHFfd2FpdCBfc2xlZXAgdGFza3F1ZXVlX3RocmVhZF9sb29wIGZv cmtfZXhpdCBmb3JrX3RyYW1wb2xpbmUgCiAgICAwIDEwMDExNyBrZXJuZWwgICAgICAgICAg IHppb193cml0ZV9pbnRyX2ggbWlfc3dpdGNoIHNsZWVwcV93YWl0IF9zbGVlcCB0YXNrcXVl dWVfdGhyZWFkX2xvb3AgZm9ya19leGl0IGZvcmtfdHJhbXBvbGluZSAKICAgIDAgMTAwMTE4 IGtlcm5lbCAgICAgICAgICAgemlvX2ZyZWVfaXNzdWUgICBtaV9zd2l0Y2ggc2xlZXBxX3dh aXQgX3NsZWVwIHRhc2txdWV1ZV90aHJlYWRfbG9vcCBmb3JrX2V4aXQgZm9ya190cmFtcG9s aW5lIAogICAgMCAxMDAxMTkga2VybmVsICAgICAgICAgICB6aW9fZnJlZV9pbnRyICAgIG1p X3N3aXRjaCBzbGVlcHFfd2FpdCBfc2xlZXAgdGFza3F1ZXVlX3RocmVhZF9sb29wIGZvcmtf ZXhpdCBmb3JrX3RyYW1wb2xpbmUgCiAgICAwIDEwMDEyMCBrZXJuZWwgICAgICAgICAgIHpp b19jbGFpbV9pc3N1ZSAgbWlfc3dpdGNoIHNsZWVwcV93YWl0IF9zbGVlcCB0YXNrcXVldWVf dGhyZWFkX2xvb3AgZm9ya19leGl0IGZvcmtfdHJhbXBvbGluZSAKICAgIDAgMTAwMTIxIGtl cm5lbCAgICAgICAgICAgemlvX2NsYWltX2ludHIgICBtaV9zd2l0Y2ggc2xlZXBxX3dhaXQg X3NsZWVwIHRhc2txdWV1ZV90aHJlYWRfbG9vcCBmb3JrX2V4aXQgZm9ya190cmFtcG9saW5l IAogICAgMCAxMDAxMjIga2VybmVsICAgICAgICAgICB6aW9faW9jdGxfaXNzdWUgIG1pX3N3 aXRjaCBzbGVlcHFfd2FpdCBfc2xlZXAgdGFza3F1ZXVlX3RocmVhZF9sb29wIGZvcmtfZXhp dCBmb3JrX3RyYW1wb2xpbmUgCiAgICAwIDEwMDEyMyBrZXJuZWwgICAgICAgICAgIHppb19p b2N0bF9pbnRyICAgbWlfc3dpdGNoIHNsZWVwcV93YWl0IF9zbGVlcCB0YXNrcXVldWVfdGhy ZWFkX2xvb3AgZm9ya19leGl0IGZvcmtfdHJhbXBvbGluZSAKICAgIDAgMTAwMTI0IGtlcm5l bCAgICAgICAgICAgemZzX3ZuX3JlbGVfdGFzayBtaV9zd2l0Y2ggc2xlZXBxX3dhaXQgX3Ns ZWVwIHRhc2txdWV1ZV90aHJlYWRfbG9vcCBmb3JrX2V4aXQgZm9ya190cmFtcG9saW5lIAog ICAgMCAxMDAxMjcga2VybmVsICAgICAgICAgICB6aWxfY2xlYW4gICAgICAgIG1pX3N3aXRj aCBzbGVlcHFfd2FpdCBfc2xlZXAgdGFza3F1ZXVlX3RocmVhZF9sb29wIGZvcmtfZXhpdCBm b3JrX3RyYW1wb2xpbmUgCiAgICAwIDEwMDE0MSBrZXJuZWwgICAgICAgICAgIHppb19udWxs X2lzc3VlICAgbWlfc3dpdGNoIHNsZWVwcV93YWl0IF9zbGVlcCB0YXNrcXVldWVfdGhyZWFk X2xvb3AgZm9ya19leGl0IGZvcmtfdHJhbXBvbGluZSAKICAgIDAgMTAwMTQyIGtlcm5lbCAg ICAgICAgICAgemlvX251bGxfaW50ciAgICBtaV9zd2l0Y2ggc2xlZXBxX3dhaXQgX3NsZWVw IHRhc2txdWV1ZV90aHJlYWRfbG9vcCBmb3JrX2V4aXQgZm9ya190cmFtcG9saW5lIAogICAg MCAxMDAxNDMga2VybmVsICAgICAgICAgICB6aW9fcmVhZF9pc3N1ZV8wIG1pX3N3aXRjaCBz bGVlcHFfd2FpdCBfc2xlZXAgdGFza3F1ZXVlX3RocmVhZF9sb29wIGZvcmtfZXhpdCBmb3Jr X3RyYW1wb2xpbmUgCiAgICAwIDEwMDE0NCBrZXJuZWwgICAgICAgICAgIHppb19yZWFkX2lz c3VlXzEgbWlfc3dpdGNoIHNsZWVwcV93YWl0IF9zbGVlcCB0YXNrcXVldWVfdGhyZWFkX2xv b3AgZm9ya19leGl0IGZvcmtfdHJhbXBvbGluZSAKICAgIDAgMTAwMTQ1IGtlcm5lbCAgICAg ICAgICAgemlvX3JlYWRfaXNzdWVfMiBtaV9zd2l0Y2ggc2xlZXBxX3dhaXQgX3NsZWVwIHRh c2txdWV1ZV90aHJlYWRfbG9vcCBmb3JrX2V4aXQgZm9ya190cmFtcG9saW5lIAogICAgMCAx MDAxNDYga2VybmVsICAgICAgICAgICB6aW9fcmVhZF9pc3N1ZV8zIG1pX3N3aXRjaCBzbGVl cHFfd2FpdCBfc2xlZXAgdGFza3F1ZXVlX3RocmVhZF9sb29wIGZvcmtfZXhpdCBmb3JrX3Ry YW1wb2xpbmUgCiAgICAwIDEwMDE0NyBrZXJuZWwgICAgICAgICAgIHppb19yZWFkX2lzc3Vl XzQgbWlfc3dpdGNoIHNsZWVwcV93YWl0IF9zbGVlcCB0YXNrcXVldWVfdGhyZWFkX2xvb3Ag Zm9ya19leGl0IGZvcmtfdHJhbXBvbGluZSAKICAgIDAgMTAwMTQ4IGtlcm5lbCAgICAgICAg ICAgemlvX3JlYWRfaXNzdWVfNSBtaV9zd2l0Y2ggc2xlZXBxX3dhaXQgX3NsZWVwIHRhc2tx dWV1ZV90aHJlYWRfbG9vcCBmb3JrX2V4aXQgZm9ya190cmFtcG9saW5lIAogICAgMCAxMDAx NDkga2VybmVsICAgICAgICAgICB6aW9fcmVhZF9pc3N1ZV82IG1pX3N3aXRjaCBzbGVlcHFf d2FpdCBfc2xlZXAgdGFza3F1ZXVlX3RocmVhZF9sb29wIGZvcmtfZXhpdCBmb3JrX3RyYW1w b2xpbmUgCiAgICAwIDEwMDE1MCBrZXJuZWwgICAgICAgICAgIHppb19yZWFkX2lzc3VlXzcg bWlfc3dpdGNoIHNsZWVwcV93YWl0IF9zbGVlcCB0YXNrcXVldWVfdGhyZWFkX2xvb3AgZm9y a19leGl0IGZvcmtfdHJhbXBvbGluZSAKICAgIDAgMTAwMTUxIGtlcm5lbCAgICAgICAgICAg emlvX3JlYWRfaW50cl8wICBtaV9zd2l0Y2ggc2xlZXBxX3dhaXQgX3NsZWVwIHRhc2txdWV1 ZV90aHJlYWRfbG9vcCBmb3JrX2V4aXQgZm9ya190cmFtcG9saW5lIAogICAgMCAxMDAxNTIg a2VybmVsICAgICAgICAgICB6aW9fcmVhZF9pbnRyXzEgIG1pX3N3aXRjaCBzbGVlcHFfd2Fp dCBfc2xlZXAgdGFza3F1ZXVlX3RocmVhZF9sb29wIGZvcmtfZXhpdCBmb3JrX3RyYW1wb2xp bmUgCiAgICAwIDEwMDE1MyBrZXJuZWwgICAgICAgICAgIHppb19yZWFkX2ludHJfMiAgbWlf c3dpdGNoIHNsZWVwcV93YWl0IF9zbGVlcCB0YXNrcXVldWVfdGhyZWFkX2xvb3AgZm9ya19l eGl0IGZvcmtfdHJhbXBvbGluZSAKICAgIDAgMTAwMTU0IGtlcm5lbCAgICAgICAgICAgemlv X3dyaXRlX2lzc3VlXyBtaV9zd2l0Y2ggc2xlZXBxX3dhaXQgX3NsZWVwIHRhc2txdWV1ZV90 aHJlYWRfbG9vcCBmb3JrX2V4aXQgZm9ya190cmFtcG9saW5lIAogICAgMCAxMDAxNTUga2Vy bmVsICAgICAgICAgICB6aW9fd3JpdGVfaXNzdWVfIG1pX3N3aXRjaCBzbGVlcHFfd2FpdCBf c2xlZXAgdGFza3F1ZXVlX3RocmVhZF9sb29wIGZvcmtfZXhpdCBmb3JrX3RyYW1wb2xpbmUg CiAgICAwIDEwMDE1NiBrZXJuZWwgICAgICAgICAgIHppb193cml0ZV9pc3N1ZV8gbWlfc3dp dGNoIHNsZWVwcV93YWl0IF9zbGVlcCB0YXNrcXVldWVfdGhyZWFkX2xvb3AgZm9ya19leGl0 IGZvcmtfdHJhbXBvbGluZSAKICAgIDAgMTAwMTU3IGtlcm5lbCAgICAgICAgICAgemlvX3dy aXRlX2lzc3VlXyBtaV9zd2l0Y2ggc2xlZXBxX3dhaXQgX3NsZWVwIHRhc2txdWV1ZV90aHJl YWRfbG9vcCBmb3JrX2V4aXQgZm9ya190cmFtcG9saW5lIAogICAgMCAxMDAxNTgga2VybmVs ICAgICAgICAgICB6aW9fd3JpdGVfaXNzdWVfIG1pX3N3aXRjaCBzbGVlcHFfd2FpdCBfc2xl ZXAgdGFza3F1ZXVlX3RocmVhZF9sb29wIGZvcmtfZXhpdCBmb3JrX3RyYW1wb2xpbmUgCiAg ICAwIDEwMDE1OSBrZXJuZWwgICAgICAgICAgIHppb193cml0ZV9pc3N1ZV8gbWlfc3dpdGNo IHNsZWVwcV93YWl0IF9zbGVlcCB0YXNrcXVldWVfdGhyZWFkX2xvb3AgZm9ya19leGl0IGZv cmtfdHJhbXBvbGluZSAKICAgIDAgMTAwMTYwIGtlcm5lbCAgICAgICAgICAgemlvX3dyaXRl X2lzc3VlXyBtaV9zd2l0Y2ggc2xlZXBxX3dhaXQgX3NsZWVwIHRhc2txdWV1ZV90aHJlYWRf bG9vcCBmb3JrX2V4aXQgZm9ya190cmFtcG9saW5lIAogICAgMCAxMDAxNjEga2VybmVsICAg ICAgICAgICB6aW9fd3JpdGVfaXNzdWVfIG1pX3N3aXRjaCBzbGVlcHFfd2FpdCBfc2xlZXAg dGFza3F1ZXVlX3RocmVhZF9sb29wIGZvcmtfZXhpdCBmb3JrX3RyYW1wb2xpbmUgCiAgICAw IDEwMDE2MiBrZXJuZWwgICAgICAgICAgIHppb193cml0ZV9pbnRyXzAgbWlfc3dpdGNoIHNs ZWVwcV93YWl0IF9zbGVlcCB0YXNrcXVldWVfdGhyZWFkX2xvb3AgZm9ya19leGl0IGZvcmtf dHJhbXBvbGluZSAKICAgIDAgMTAwMTYzIGtlcm5lbCAgICAgICAgICAgemlvX3dyaXRlX2lu dHJfMSBtaV9zd2l0Y2ggc2xlZXBxX3dhaXQgX3NsZWVwIHRhc2txdWV1ZV90aHJlYWRfbG9v cCBmb3JrX2V4aXQgZm9ya190cmFtcG9saW5lIAogICAgMCAxMDAxNjQga2VybmVsICAgICAg ICAgICB6aW9fd3JpdGVfaW50cl8yIG1pX3N3aXRjaCBzbGVlcHFfd2FpdCBfc2xlZXAgdGFz a3F1ZXVlX3RocmVhZF9sb29wIGZvcmtfZXhpdCBmb3JrX3RyYW1wb2xpbmUgCiAgICAwIDEw MDE2NSBrZXJuZWwgICAgICAgICAgIHppb193cml0ZV9pbnRyXzMgbWlfc3dpdGNoIHNsZWVw cV93YWl0IF9zbGVlcCB0YXNrcXVldWVfdGhyZWFkX2xvb3AgZm9ya19leGl0IGZvcmtfdHJh bXBvbGluZSAKICAgIDAgMTAwMTY2IGtlcm5lbCAgICAgICAgICAgemlvX3dyaXRlX2ludHJf NCBtaV9zd2l0Y2ggc2xlZXBxX3dhaXQgX3NsZWVwIHRhc2txdWV1ZV90aHJlYWRfbG9vcCBm b3JrX2V4aXQgZm9ya190cmFtcG9saW5lIAogICAgMCAxMDAxNjcga2VybmVsICAgICAgICAg ICB6aW9fd3JpdGVfaW50cl81IG1pX3N3aXRjaCBzbGVlcHFfd2FpdCBfc2xlZXAgdGFza3F1 ZXVlX3RocmVhZF9sb29wIGZvcmtfZXhpdCBmb3JrX3RyYW1wb2xpbmUgCiAgICAwIDEwMDE2 OCBrZXJuZWwgICAgICAgICAgIHppb193cml0ZV9pbnRyXzYgbWlfc3dpdGNoIHNsZWVwcV93 YWl0IF9zbGVlcCB0YXNrcXVldWVfdGhyZWFkX2xvb3AgZm9ya19leGl0IGZvcmtfdHJhbXBv bGluZSAKICAgIDAgMTAwMTY5IGtlcm5lbCAgICAgICAgICAgemlvX3dyaXRlX2ludHJfNyBt aV9zd2l0Y2ggc2xlZXBxX3dhaXQgX3NsZWVwIHRhc2txdWV1ZV90aHJlYWRfbG9vcCBmb3Jr X2V4aXQgZm9ya190cmFtcG9saW5lIAogICAgMCAxMDAxNzAga2VybmVsICAgICAgICAgICB6 aW9fd3JpdGVfaW50cl9oIG1pX3N3aXRjaCBzbGVlcHFfd2FpdCBfc2xlZXAgdGFza3F1ZXVl X3RocmVhZF9sb29wIGZvcmtfZXhpdCBmb3JrX3RyYW1wb2xpbmUgCiAgICAwIDEwMDE3MSBr ZXJuZWwgICAgICAgICAgIHppb193cml0ZV9pbnRyX2ggbWlfc3dpdGNoIHNsZWVwcV93YWl0 IF9zbGVlcCB0YXNrcXVldWVfdGhyZWFkX2xvb3AgZm9ya19leGl0IGZvcmtfdHJhbXBvbGlu ZSAKICAgIDAgMTAwMTcyIGtlcm5lbCAgICAgICAgICAgemlvX3dyaXRlX2ludHJfaCBtaV9z d2l0Y2ggc2xlZXBxX3dhaXQgX3NsZWVwIHRhc2txdWV1ZV90aHJlYWRfbG9vcCBmb3JrX2V4 aXQgZm9ya190cmFtcG9saW5lIAogICAgMCAxMDAxNzMga2VybmVsICAgICAgICAgICB6aW9f d3JpdGVfaW50cl9oIG1pX3N3aXRjaCBzbGVlcHFfd2FpdCBfc2xlZXAgdGFza3F1ZXVlX3Ro cmVhZF9sb29wIGZvcmtfZXhpdCBmb3JrX3RyYW1wb2xpbmUgCiAgICAwIDEwMDE3NCBrZXJu ZWwgICAgICAgICAgIHppb193cml0ZV9pbnRyX2ggbWlfc3dpdGNoIHNsZWVwcV93YWl0IF9z bGVlcCB0YXNrcXVldWVfdGhyZWFkX2xvb3AgZm9ya19leGl0IGZvcmtfdHJhbXBvbGluZSAK ICAgIDAgMTAwMTc1IGtlcm5lbCAgICAgICAgICAgemlvX2ZyZWVfaXNzdWUgICBtaV9zd2l0 Y2ggc2xlZXBxX3dhaXQgX3NsZWVwIHRhc2txdWV1ZV90aHJlYWRfbG9vcCBmb3JrX2V4aXQg Zm9ya190cmFtcG9saW5lIAogICAgMCAxMDAxNzYga2VybmVsICAgICAgICAgICB6aW9fZnJl ZV9pbnRyICAgIG1pX3N3aXRjaCBzbGVlcHFfd2FpdCBfc2xlZXAgdGFza3F1ZXVlX3RocmVh ZF9sb29wIGZvcmtfZXhpdCBmb3JrX3RyYW1wb2xpbmUgCiAgICAwIDEwMDE3NyBrZXJuZWwg ICAgICAgICAgIHppb19jbGFpbV9pc3N1ZSAgbWlfc3dpdGNoIHNsZWVwcV93YWl0IF9zbGVl cCB0YXNrcXVldWVfdGhyZWFkX2xvb3AgZm9ya19leGl0IGZvcmtfdHJhbXBvbGluZSAKICAg IDAgMTAwMTc4IGtlcm5lbCAgICAgICAgICAgemlvX2NsYWltX2ludHIgICBtaV9zd2l0Y2gg c2xlZXBxX3dhaXQgX3NsZWVwIHRhc2txdWV1ZV90aHJlYWRfbG9vcCBmb3JrX2V4aXQgZm9y a190cmFtcG9saW5lIAogICAgMCAxMDAxNzkga2VybmVsICAgICAgICAgICB6aW9faW9jdGxf aXNzdWUgIG1pX3N3aXRjaCBzbGVlcHFfd2FpdCBfc2xlZXAgdGFza3F1ZXVlX3RocmVhZF9s b29wIGZvcmtfZXhpdCBmb3JrX3RyYW1wb2xpbmUgCiAgICAwIDEwMDE4MCBrZXJuZWwgICAg ICAgICAgIHppb19pb2N0bF9pbnRyICAgbWlfc3dpdGNoIHNsZWVwcV93YWl0IF9zbGVlcCB0 YXNrcXVldWVfdGhyZWFkX2xvb3AgZm9ya19leGl0IGZvcmtfdHJhbXBvbGluZSAKICAgIDAg MTAwMTgxIGtlcm5lbCAgICAgICAgICAgemZzX3ZuX3JlbGVfdGFzayBtaV9zd2l0Y2ggc2xl ZXBxX3dhaXQgX3NsZWVwIHRhc2txdWV1ZV90aHJlYWRfbG9vcCBmb3JrX2V4aXQgZm9ya190 cmFtcG9saW5lIAogICAgMCAxMDAxOTUga2VybmVsICAgICAgICAgICB6aWxfY2xlYW4gICAg ICAgIG1pX3N3aXRjaCBzbGVlcHFfd2FpdCBfc2xlZXAgdGFza3F1ZXVlX3RocmVhZF9sb29w IGZvcmtfZXhpdCBmb3JrX3RyYW1wb2xpbmUgCiAgICAwIDEwMDE5NiBrZXJuZWwgICAgICAg ICAgIHppbF9jbGVhbiAgICAgICAgbWlfc3dpdGNoIHNsZWVwcV93YWl0IF9zbGVlcCB0YXNr cXVldWVfdGhyZWFkX2xvb3AgZm9ya19leGl0IGZvcmtfdHJhbXBvbGluZSAKICAgIDAgMTAw MTk3IGtlcm5lbCAgICAgICAgICAgemlsX2NsZWFuICAgICAgICBtaV9zd2l0Y2ggc2xlZXBx X3dhaXQgX3NsZWVwIHRhc2txdWV1ZV90aHJlYWRfbG9vcCBmb3JrX2V4aXQgZm9ya190cmFt cG9saW5lIAogICAgMCAxMDAxOTgga2VybmVsICAgICAgICAgICB6aWxfY2xlYW4gICAgICAg IG1pX3N3aXRjaCBzbGVlcHFfd2FpdCBfc2xlZXAgdGFza3F1ZXVlX3RocmVhZF9sb29wIGZv cmtfZXhpdCBmb3JrX3RyYW1wb2xpbmUgCiAgICAwIDEwMDE5OSBrZXJuZWwgICAgICAgICAg IHppbF9jbGVhbiAgICAgICAgbWlfc3dpdGNoIHNsZWVwcV93YWl0IF9zbGVlcCB0YXNrcXVl dWVfdGhyZWFkX2xvb3AgZm9ya19leGl0IGZvcmtfdHJhbXBvbGluZSAKICAgIDAgMTAwMjAw IGtlcm5lbCAgICAgICAgICAgemlsX2NsZWFuICAgICAgICBtaV9zd2l0Y2ggc2xlZXBxX3dh aXQgX3NsZWVwIHRhc2txdWV1ZV90aHJlYWRfbG9vcCBmb3JrX2V4aXQgZm9ya190cmFtcG9s aW5lIAogICAgMCAxMDAyMDEga2VybmVsICAgICAgICAgICB6aWxfY2xlYW4gICAgICAgIG1p X3N3aXRjaCBzbGVlcHFfd2FpdCBfc2xlZXAgdGFza3F1ZXVlX3RocmVhZF9sb29wIGZvcmtf ZXhpdCBmb3JrX3RyYW1wb2xpbmUgCiAgICAwIDEwMDIwMiBrZXJuZWwgICAgICAgICAgIHpp bF9jbGVhbiAgICAgICAgbWlfc3dpdGNoIHNsZWVwcV93YWl0IF9zbGVlcCB0YXNrcXVldWVf dGhyZWFkX2xvb3AgZm9ya19leGl0IGZvcmtfdHJhbXBvbGluZSAKICAgIDAgMTAwMjAzIGtl cm5lbCAgICAgICAgICAgemlsX2NsZWFuICAgICAgICBtaV9zd2l0Y2ggc2xlZXBxX3dhaXQg X3NsZWVwIHRhc2txdWV1ZV90aHJlYWRfbG9vcCBmb3JrX2V4aXQgZm9ya190cmFtcG9saW5l IAogICAgMCAxMDAyMDQga2VybmVsICAgICAgICAgICB6aWxfY2xlYW4gICAgICAgIG1pX3N3 aXRjaCBzbGVlcHFfd2FpdCBfc2xlZXAgdGFza3F1ZXVlX3RocmVhZF9sb29wIGZvcmtfZXhp dCBmb3JrX3RyYW1wb2xpbmUgCiAgICAwIDEwMDIwNSBrZXJuZWwgICAgICAgICAgIHppbF9j bGVhbiAgICAgICAgbWlfc3dpdGNoIHNsZWVwcV93YWl0IF9zbGVlcCB0YXNrcXVldWVfdGhy ZWFkX2xvb3AgZm9ya19leGl0IGZvcmtfdHJhbXBvbGluZSAKICAgIDAgMTAwMjA2IGtlcm5l bCAgICAgICAgICAgemlsX2NsZWFuICAgICAgICBtaV9zd2l0Y2ggc2xlZXBxX3dhaXQgX3Ns ZWVwIHRhc2txdWV1ZV90aHJlYWRfbG9vcCBmb3JrX2V4aXQgZm9ya190cmFtcG9saW5lIAog ICAgMCAxMDAyMDcga2VybmVsICAgICAgICAgICB6aWxfY2xlYW4gICAgICAgIG1pX3N3aXRj aCBzbGVlcHFfd2FpdCBfc2xlZXAgdGFza3F1ZXVlX3RocmVhZF9sb29wIGZvcmtfZXhpdCBm b3JrX3RyYW1wb2xpbmUgCiAgICAwIDEwMDIwOCBrZXJuZWwgICAgICAgICAgIHppbF9jbGVh biAgICAgICAgbWlfc3dpdGNoIHNsZWVwcV93YWl0IF9zbGVlcCB0YXNrcXVldWVfdGhyZWFk X2xvb3AgZm9ya19leGl0IGZvcmtfdHJhbXBvbGluZSAKICAgIDAgMTAwMjA5IGtlcm5lbCAg ICAgICAgICAgemlsX2NsZWFuICAgICAgICBtaV9zd2l0Y2ggc2xlZXBxX3dhaXQgX3NsZWVw IHRhc2txdWV1ZV90aHJlYWRfbG9vcCBmb3JrX2V4aXQgZm9ya190cmFtcG9saW5lIAogICAg MCAxMDAyMTAga2VybmVsICAgICAgICAgICB6aWxfY2xlYW4gICAgICAgIG1pX3N3aXRjaCBz bGVlcHFfd2FpdCBfc2xlZXAgdGFza3F1ZXVlX3RocmVhZF9sb29wIGZvcmtfZXhpdCBmb3Jr X3RyYW1wb2xpbmUgCiAgICAwIDEwMDIxMSBrZXJuZWwgICAgICAgICAgIHppbF9jbGVhbiAg ICAgICAgbWlfc3dpdGNoIHNsZWVwcV93YWl0IF9zbGVlcCB0YXNrcXVldWVfdGhyZWFkX2xv b3AgZm9ya19leGl0IGZvcmtfdHJhbXBvbGluZSAKICAgIDAgMTAwMjEyIGtlcm5lbCAgICAg ICAgICAgemlsX2NsZWFuICAgICAgICBtaV9zd2l0Y2ggc2xlZXBxX3dhaXQgX3NsZWVwIHRh c2txdWV1ZV90aHJlYWRfbG9vcCBmb3JrX2V4aXQgZm9ya190cmFtcG9saW5lIAogICAgMCAx MDAyMTMga2VybmVsICAgICAgICAgICB6aWxfY2xlYW4gICAgICAgIG1pX3N3aXRjaCBzbGVl cHFfd2FpdCBfc2xlZXAgdGFza3F1ZXVlX3RocmVhZF9sb29wIGZvcmtfZXhpdCBmb3JrX3Ry YW1wb2xpbmUgCiAgICAwIDEwMDIxNCBrZXJuZWwgICAgICAgICAgIHppbF9jbGVhbiAgICAg ICAgbWlfc3dpdGNoIHNsZWVwcV93YWl0IF9zbGVlcCB0YXNrcXVldWVfdGhyZWFkX2xvb3Ag Zm9ya19leGl0IGZvcmtfdHJhbXBvbGluZSAKICAgIDAgMTAwMjE1IGtlcm5lbCAgICAgICAg ICAgemlsX2NsZWFuICAgICAgICBtaV9zd2l0Y2ggc2xlZXBxX3dhaXQgX3NsZWVwIHRhc2tx dWV1ZV90aHJlYWRfbG9vcCBmb3JrX2V4aXQgZm9ya190cmFtcG9saW5lIAogICAgMCAxMDAy MTYga2VybmVsICAgICAgICAgICB6aWxfY2xlYW4gICAgICAgIG1pX3N3aXRjaCBzbGVlcHFf d2FpdCBfc2xlZXAgdGFza3F1ZXVlX3RocmVhZF9sb29wIGZvcmtfZXhpdCBmb3JrX3RyYW1w b2xpbmUgCiAgICAwIDEwMDIxNyBrZXJuZWwgICAgICAgICAgIHppbF9jbGVhbiAgICAgICAg bWlfc3dpdGNoIHNsZWVwcV93YWl0IF9zbGVlcCB0YXNrcXVldWVfdGhyZWFkX2xvb3AgZm9y a19leGl0IGZvcmtfdHJhbXBvbGluZSAKICAgIDAgMTAwMjE4IGtlcm5lbCAgICAgICAgICAg emlsX2NsZWFuICAgICAgICBtaV9zd2l0Y2ggc2xlZXBxX3dhaXQgX3NsZWVwIHRhc2txdWV1 ZV90aHJlYWRfbG9vcCBmb3JrX2V4aXQgZm9ya190cmFtcG9saW5lIAogICAgMCAxMDAyMTkg a2VybmVsICAgICAgICAgICB6aWxfY2xlYW4gICAgICAgIG1pX3N3aXRjaCBzbGVlcHFfd2Fp dCBfc2xlZXAgdGFza3F1ZXVlX3RocmVhZF9sb29wIGZvcmtfZXhpdCBmb3JrX3RyYW1wb2xp bmUgCiAgICAwIDEwMDIyMCBrZXJuZWwgICAgICAgICAgIHppbF9jbGVhbiAgICAgICAgbWlf c3dpdGNoIHNsZWVwcV93YWl0IF9zbGVlcCB0YXNrcXVldWVfdGhyZWFkX2xvb3AgZm9ya19l eGl0IGZvcmtfdHJhbXBvbGluZSAKICAgIDAgMTAwMjIxIGtlcm5lbCAgICAgICAgICAgemls X2NsZWFuICAgICAgICBtaV9zd2l0Y2ggc2xlZXBxX3dhaXQgX3NsZWVwIHRhc2txdWV1ZV90 aHJlYWRfbG9vcCBmb3JrX2V4aXQgZm9ya190cmFtcG9saW5lIAogICAgMCAxMDAyMjIga2Vy bmVsICAgICAgICAgICB6aWxfY2xlYW4gICAgICAgIG1pX3N3aXRjaCBzbGVlcHFfd2FpdCBf c2xlZXAgdGFza3F1ZXVlX3RocmVhZF9sb29wIGZvcmtfZXhpdCBmb3JrX3RyYW1wb2xpbmUg CiAgICAwIDEwMDIyMyBrZXJuZWwgICAgICAgICAgIHppbF9jbGVhbiAgICAgICAgbWlfc3dp dGNoIHNsZWVwcV93YWl0IF9zbGVlcCB0YXNrcXVldWVfdGhyZWFkX2xvb3AgZm9ya19leGl0 IGZvcmtfdHJhbXBvbGluZSAKICAgIDAgMTAwMjI0IGtlcm5lbCAgICAgICAgICAgemlsX2Ns ZWFuICAgICAgICBtaV9zd2l0Y2ggc2xlZXBxX3dhaXQgX3NsZWVwIHRhc2txdWV1ZV90aHJl YWRfbG9vcCBmb3JrX2V4aXQgZm9ya190cmFtcG9saW5lIAogICAgMCAxMDAyMjUga2VybmVs ICAgICAgICAgICB6aWxfY2xlYW4gICAgICAgIG1pX3N3aXRjaCBzbGVlcHFfd2FpdCBfc2xl ZXAgdGFza3F1ZXVlX3RocmVhZF9sb29wIGZvcmtfZXhpdCBmb3JrX3RyYW1wb2xpbmUgCiAg ICAwIDEwMDIyNiBrZXJuZWwgICAgICAgICAgIHppbF9jbGVhbiAgICAgICAgbWlfc3dpdGNo IHNsZWVwcV93YWl0IF9zbGVlcCB0YXNrcXVldWVfdGhyZWFkX2xvb3AgZm9ya19leGl0IGZv cmtfdHJhbXBvbGluZSAKICAgIDAgMTAwMjI3IGtlcm5lbCAgICAgICAgICAgemlsX2NsZWFu ICAgICAgICBtaV9zd2l0Y2ggc2xlZXBxX3dhaXQgX3NsZWVwIHRhc2txdWV1ZV90aHJlYWRf bG9vcCBmb3JrX2V4aXQgZm9ya190cmFtcG9saW5lIAogICAgMCAxMDAyMjgga2VybmVsICAg ICAgICAgICB6aWxfY2xlYW4gICAgICAgIG1pX3N3aXRjaCBzbGVlcHFfd2FpdCBfc2xlZXAg dGFza3F1ZXVlX3RocmVhZF9sb29wIGZvcmtfZXhpdCBmb3JrX3RyYW1wb2xpbmUgCiAgICAw IDEwMDIyOSBrZXJuZWwgICAgICAgICAgIHppbF9jbGVhbiAgICAgICAgbWlfc3dpdGNoIHNs ZWVwcV93YWl0IF9zbGVlcCB0YXNrcXVldWVfdGhyZWFkX2xvb3AgZm9ya19leGl0IGZvcmtf dHJhbXBvbGluZSAKICAgIDAgMTAwMjMwIGtlcm5lbCAgICAgICAgICAgemlsX2NsZWFuICAg ICAgICBtaV9zd2l0Y2ggc2xlZXBxX3dhaXQgX3NsZWVwIHRhc2txdWV1ZV90aHJlYWRfbG9v cCBmb3JrX2V4aXQgZm9ya190cmFtcG9saW5lIAogICAgMCAxMDAyMzEga2VybmVsICAgICAg ICAgICB6aWxfY2xlYW4gICAgICAgIG1pX3N3aXRjaCBzbGVlcHFfd2FpdCBfc2xlZXAgdGFz a3F1ZXVlX3RocmVhZF9sb29wIGZvcmtfZXhpdCBmb3JrX3RyYW1wb2xpbmUgCiAgICAwIDEw MDIzMiBrZXJuZWwgICAgICAgICAgIHppbF9jbGVhbiAgICAgICAgbWlfc3dpdGNoIHNsZWVw cV93YWl0IF9zbGVlcCB0YXNrcXVldWVfdGhyZWFkX2xvb3AgZm9ya19leGl0IGZvcmtfdHJh bXBvbGluZSAKICAgIDAgMTAwMjMzIGtlcm5lbCAgICAgICAgICAgemlsX2NsZWFuICAgICAg ICBtaV9zd2l0Y2ggc2xlZXBxX3dhaXQgX3NsZWVwIHRhc2txdWV1ZV90aHJlYWRfbG9vcCBm b3JrX2V4aXQgZm9ya190cmFtcG9saW5lIAogICAgMCAxMDAyMzQga2VybmVsICAgICAgICAg ICB6aWxfY2xlYW4gICAgICAgIG1pX3N3aXRjaCBzbGVlcHFfd2FpdCBfc2xlZXAgdGFza3F1 ZXVlX3RocmVhZF9sb29wIGZvcmtfZXhpdCBmb3JrX3RyYW1wb2xpbmUgCiAgICAwIDEwMDIz NSBrZXJuZWwgICAgICAgICAgIHppbF9jbGVhbiAgICAgICAgbWlfc3dpdGNoIHNsZWVwcV93 YWl0IF9zbGVlcCB0YXNrcXVldWVfdGhyZWFkX2xvb3AgZm9ya19leGl0IGZvcmtfdHJhbXBv bGluZSAKICAgIDAgMTAwMjM2IGtlcm5lbCAgICAgICAgICAgemlsX2NsZWFuICAgICAgICBt aV9zd2l0Y2ggc2xlZXBxX3dhaXQgX3NsZWVwIHRhc2txdWV1ZV90aHJlYWRfbG9vcCBmb3Jr X2V4aXQgZm9ya190cmFtcG9saW5lIAogICAgMCAxMDAyMzcga2VybmVsICAgICAgICAgICB6 aWxfY2xlYW4gICAgICAgIG1pX3N3aXRjaCBzbGVlcHFfd2FpdCBfc2xlZXAgdGFza3F1ZXVl X3RocmVhZF9sb29wIGZvcmtfZXhpdCBmb3JrX3RyYW1wb2xpbmUgCiAgICAwIDEwMDIzOCBr ZXJuZWwgICAgICAgICAgIHppbF9jbGVhbiAgICAgICAgbWlfc3dpdGNoIHNsZWVwcV93YWl0 IF9zbGVlcCB0YXNrcXVldWVfdGhyZWFkX2xvb3AgZm9ya19leGl0IGZvcmtfdHJhbXBvbGlu ZSAKICAgIDAgMTAwMjM5IGtlcm5lbCAgICAgICAgICAgemlsX2NsZWFuICAgICAgICBtaV9z d2l0Y2ggc2xlZXBxX3dhaXQgX3NsZWVwIHRhc2txdWV1ZV90aHJlYWRfbG9vcCBmb3JrX2V4 aXQgZm9ya190cmFtcG9saW5lIAogICAgMCAxMDAyNDAga2VybmVsICAgICAgICAgICB6aWxf Y2xlYW4gICAgICAgIG1pX3N3aXRjaCBzbGVlcHFfd2FpdCBfc2xlZXAgdGFza3F1ZXVlX3Ro cmVhZF9sb29wIGZvcmtfZXhpdCBmb3JrX3RyYW1wb2xpbmUgCiAgICAwIDEwMDI0MSBrZXJu ZWwgICAgICAgICAgIHppbF9jbGVhbiAgICAgICAgbWlfc3dpdGNoIHNsZWVwcV93YWl0IF9z bGVlcCB0YXNrcXVldWVfdGhyZWFkX2xvb3AgZm9ya19leGl0IGZvcmtfdHJhbXBvbGluZSAK ICAgIDAgMTAwMjQyIGtlcm5lbCAgICAgICAgICAgemlsX2NsZWFuICAgICAgICBtaV9zd2l0 Y2ggc2xlZXBxX3dhaXQgX3NsZWVwIHRhc2txdWV1ZV90aHJlYWRfbG9vcCBmb3JrX2V4aXQg Zm9ya190cmFtcG9saW5lIAogICAgMCAxMDAyNDMga2VybmVsICAgICAgICAgICB6aWxfY2xl YW4gICAgICAgIG1pX3N3aXRjaCBzbGVlcHFfd2FpdCBfc2xlZXAgdGFza3F1ZXVlX3RocmVh ZF9sb29wIGZvcmtfZXhpdCBmb3JrX3RyYW1wb2xpbmUgCiAgICAwIDEwMDI0NCBrZXJuZWwg ICAgICAgICAgIHppbF9jbGVhbiAgICAgICAgbWlfc3dpdGNoIHNsZWVwcV93YWl0IF9zbGVl cCB0YXNrcXVldWVfdGhyZWFkX2xvb3AgZm9ya19leGl0IGZvcmtfdHJhbXBvbGluZSAKICAg IDAgMTAwMjQ1IGtlcm5lbCAgICAgICAgICAgemlsX2NsZWFuICAgICAgICBtaV9zd2l0Y2gg c2xlZXBxX3dhaXQgX3NsZWVwIHRhc2txdWV1ZV90aHJlYWRfbG9vcCBmb3JrX2V4aXQgZm9y a190cmFtcG9saW5lIAogICAgMCAxMDAyNDYga2VybmVsICAgICAgICAgICB6aWxfY2xlYW4g ICAgICAgIG1pX3N3aXRjaCBzbGVlcHFfd2FpdCBfc2xlZXAgdGFza3F1ZXVlX3RocmVhZF9s b29wIGZvcmtfZXhpdCBmb3JrX3RyYW1wb2xpbmUgCiAgICAwIDEwMDI0NyBrZXJuZWwgICAg ICAgICAgIHppbF9jbGVhbiAgICAgICAgbWlfc3dpdGNoIHNsZWVwcV93YWl0IF9zbGVlcCB0 YXNrcXVldWVfdGhyZWFkX2xvb3AgZm9ya19leGl0IGZvcmtfdHJhbXBvbGluZSAKICAgIDAg MTAwMjQ4IGtlcm5lbCAgICAgICAgICAgemlsX2NsZWFuICAgICAgICBtaV9zd2l0Y2ggc2xl ZXBxX3dhaXQgX3NsZWVwIHRhc2txdWV1ZV90aHJlYWRfbG9vcCBmb3JrX2V4aXQgZm9ya190 cmFtcG9saW5lIAogICAgMCAxMDAyNDkga2VybmVsICAgICAgICAgICB6aWxfY2xlYW4gICAg ICAgIG1pX3N3aXRjaCBzbGVlcHFfd2FpdCBfc2xlZXAgdGFza3F1ZXVlX3RocmVhZF9sb29w IGZvcmtfZXhpdCBmb3JrX3RyYW1wb2xpbmUgCiAgICAwIDEwMDI1MCBrZXJuZWwgICAgICAg ICAgIHppbF9jbGVhbiAgICAgICAgbWlfc3dpdGNoIHNsZWVwcV93YWl0IF9zbGVlcCB0YXNr cXVldWVfdGhyZWFkX2xvb3AgZm9ya19leGl0IGZvcmtfdHJhbXBvbGluZSAKICAgIDAgMTAw MjUxIGtlcm5lbCAgICAgICAgICAgemlsX2NsZWFuICAgICAgICBtaV9zd2l0Y2ggc2xlZXBx X3dhaXQgX3NsZWVwIHRhc2txdWV1ZV90aHJlYWRfbG9vcCBmb3JrX2V4aXQgZm9ya190cmFt cG9saW5lIAogICAgMCAxMDAyNTIga2VybmVsICAgICAgICAgICB6aWxfY2xlYW4gICAgICAg IG1pX3N3aXRjaCBzbGVlcHFfd2FpdCBfc2xlZXAgdGFza3F1ZXVlX3RocmVhZF9sb29wIGZv cmtfZXhpdCBmb3JrX3RyYW1wb2xpbmUgCiAgICAwIDEwMDI1MyBrZXJuZWwgICAgICAgICAg IHppbF9jbGVhbiAgICAgICAgbWlfc3dpdGNoIHNsZWVwcV93YWl0IF9zbGVlcCB0YXNrcXVl dWVfdGhyZWFkX2xvb3AgZm9ya19leGl0IGZvcmtfdHJhbXBvbGluZSAKICAgIDAgMTAwMjU0 IGtlcm5lbCAgICAgICAgICAgemlsX2NsZWFuICAgICAgICBtaV9zd2l0Y2ggc2xlZXBxX3dh aXQgX3NsZWVwIHRhc2txdWV1ZV90aHJlYWRfbG9vcCBmb3JrX2V4aXQgZm9ya190cmFtcG9s aW5lIAogICAgMCAxMDAyNTUga2VybmVsICAgICAgICAgICB6aWxfY2xlYW4gICAgICAgIG1p X3N3aXRjaCBzbGVlcHFfd2FpdCBfc2xlZXAgdGFza3F1ZXVlX3RocmVhZF9sb29wIGZvcmtf ZXhpdCBmb3JrX3RyYW1wb2xpbmUgCiAgICAwIDEwMDI1NiBrZXJuZWwgICAgICAgICAgIHpp bF9jbGVhbiAgICAgICAgbWlfc3dpdGNoIHNsZWVwcV93YWl0IF9zbGVlcCB0YXNrcXVldWVf dGhyZWFkX2xvb3AgZm9ya19leGl0IGZvcmtfdHJhbXBvbGluZSAKICAgIDAgMTAwMjU3IGtl cm5lbCAgICAgICAgICAgemlsX2NsZWFuICAgICAgICBtaV9zd2l0Y2ggc2xlZXBxX3dhaXQg X3NsZWVwIHRhc2txdWV1ZV90aHJlYWRfbG9vcCBmb3JrX2V4aXQgZm9ya190cmFtcG9saW5l IAogICAgMCAxMDAyNTgga2VybmVsICAgICAgICAgICB6aWxfY2xlYW4gICAgICAgIG1pX3N3 aXRjaCBzbGVlcHFfd2FpdCBfc2xlZXAgdGFza3F1ZXVlX3RocmVhZF9sb29wIGZvcmtfZXhp dCBmb3JrX3RyYW1wb2xpbmUgCiAgICAwIDEwMDI1OSBrZXJuZWwgICAgICAgICAgIHppbF9j bGVhbiAgICAgICAgbWlfc3dpdGNoIHNsZWVwcV93YWl0IF9zbGVlcCB0YXNrcXVldWVfdGhy ZWFkX2xvb3AgZm9ya19leGl0IGZvcmtfdHJhbXBvbGluZSAKICAgIDAgMTAwMjYwIGtlcm5l bCAgICAgICAgICAgemlsX2NsZWFuICAgICAgICBtaV9zd2l0Y2ggc2xlZXBxX3dhaXQgX3Ns ZWVwIHRhc2txdWV1ZV90aHJlYWRfbG9vcCBmb3JrX2V4aXQgZm9ya190cmFtcG9saW5lIAog ICAgMCAxMDAyNjEga2VybmVsICAgICAgICAgICB6aWxfY2xlYW4gICAgICAgIG1pX3N3aXRj aCBzbGVlcHFfd2FpdCBfc2xlZXAgdGFza3F1ZXVlX3RocmVhZF9sb29wIGZvcmtfZXhpdCBm b3JrX3RyYW1wb2xpbmUgCiAgICAwIDEwMDI2MiBrZXJuZWwgICAgICAgICAgIHppbF9jbGVh biAgICAgICAgbWlfc3dpdGNoIHNsZWVwcV93YWl0IF9zbGVlcCB0YXNrcXVldWVfdGhyZWFk X2xvb3AgZm9ya19leGl0IGZvcmtfdHJhbXBvbGluZSAKCg== --------------080904090108030003080008-- From owner-zfs-devel@FreeBSD.ORG Mon Oct 25 10:46:23 2010 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 397DE106566C; Mon, 25 Oct 2010 10:46:23 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from mail.vx.sk (mail.vx.sk [IPv6:2a01:4f8:100:1043::3]) by mx1.freebsd.org (Postfix) with ESMTP id C39BF8FC16; Mon, 25 Oct 2010 10:46:22 +0000 (UTC) Received: from core.vx.sk (localhost [127.0.0.1]) by mail.vx.sk (Postfix) with ESMTP id ED7E7119B71; Mon, 25 Oct 2010 12:46:21 +0200 (CEST) X-Virus-Scanned: amavisd-new at mail.vx.sk Received: from mail.vx.sk ([127.0.0.1]) by core.vx.sk (mail.vx.sk [127.0.0.1]) (amavisd-new, port 10024) with LMTP id eYgPoc+zv+HL; Mon, 25 Oct 2010 12:46:20 +0200 (CEST) Received: from [10.0.3.3] (188-167-67-67.dynamic.chello.sk [188.167.67.67]) by mail.vx.sk (Postfix) with ESMTPSA id 1BE6B119B60; Mon, 25 Oct 2010 12:46:20 +0200 (CEST) Message-ID: <4CC55FFB.1010303@FreeBSD.org> Date: Mon, 25 Oct 2010 12:46:19 +0200 From: Martin Matuska User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; sk; rv:1.8.1.23) Gecko/20090812 Lightning/0.9 Thunderbird/2.0.0.23 Mnenhy/0.7.5.0 MIME-Version: 1.0 To: Pawel Jakub Dawidek , Xin LI X-Enigmail-Version: 1.1.1 Content-Type: text/plain; charset=windows-1250 Content-Transfer-Encoding: 7bit Cc: zfs-devel@FreeBSD.org Subject: ZFS patch for review: osol rev. 10204, 10209 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 25 Oct 2010 10:46:23 -0000 Hi, I would like to propose the following two upstream changesets for import: OpenSolaris revision 10204: (fixes for zfs receive) Bug ID 6760420: zfs unmount -f causes recv to fail Bug ID 6759975: zplprops unavailable to examiner of in-progress receive http://people.freebsd.org/~mm/patches/zfs/head-10204.patch OpenSolaris revision 10209: Bug ID 6830541: zfs_get_data_trips on a verify Bug ID 6696242: multiple zfs_fillpage() zfs: accessing past end of object panics Bug ID 6785914: zfs fails to drop dn_struct_rwlock in recovery code path http://people.freebsd.org/~mm/patches/zfs/head-10209.patch Next, I am currently also working to import the corresponding fix for Bug ID 6860996 (%temporary clones are not automatically destroyed on error) that fixes in revision 10298 a problem that got introduced in 9396. From owner-zfs-devel@FreeBSD.ORG Tue Oct 26 08:25:04 2010 Return-Path: Delivered-To: zfs-devel@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 402C8106564A; Tue, 26 Oct 2010 08:25:04 +0000 (UTC) (envelope-from delphij@delphij.net) Received: from tarsier.geekcn.org (tarsier.geekcn.org [IPv6:2001:470:a803::1]) by mx1.freebsd.org (Postfix) with ESMTP id 81D888FC12; Tue, 26 Oct 2010 08:25:03 +0000 (UTC) Received: from mail.geekcn.org (tarsier.geekcn.org [211.166.10.233]) by tarsier.geekcn.org (Postfix) with ESMTP id 6F827A68C2D; Tue, 26 Oct 2010 16:25:01 +0800 (CST) X-Virus-Scanned: amavisd-new at geekcn.org Received: from tarsier.geekcn.org ([211.166.10.233]) by mail.geekcn.org (mail.geekcn.org [211.166.10.233]) (amavisd-new, port 10024) with LMTP id k+QsEb73O1cq; Tue, 26 Oct 2010 16:24:52 +0800 (CST) Received: from delta.delphij.net (c-76-102-48-203.hsd1.ca.comcast.net [76.102.48.203]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by tarsier.geekcn.org (Postfix) with ESMTPSA id 1C0A6A68C1E; Tue, 26 Oct 2010 16:24:49 +0800 (CST) DomainKey-Signature: a=rsa-sha1; s=default; d=delphij.net; c=nofws; q=dns; h=message-id:date:from:reply-to:organization:user-agent: mime-version:to:cc:subject:references:in-reply-to: x-enigmail-version:openpgp:content-type:content-transfer-encoding; b=IWHHuVDDpccqmQZBz5OGZoQBR4KC3n2MyFj1/4nsddjUjO5EfGse5W54uvhSqjqE4 D928dsKsoOkztINXqCymw== Message-ID: <4CC6904B.4090905@delphij.net> Date: Tue, 26 Oct 2010 01:24:43 -0700 From: Xin LI Organization: The FreeBSD Project User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.1.14) Gecko/20101020 Thunderbird/3.0.9 ThunderBrowse/3.3.2 MIME-Version: 1.0 To: Martin Matuska References: <4CC55FFB.1010303@FreeBSD.org> In-Reply-To: <4CC55FFB.1010303@FreeBSD.org> X-Enigmail-Version: 1.0.1 OpenPGP: id=3FCA37C1; url=http://www.delphij.net/delphij.asc Content-Type: text/plain; charset=windows-1250 Content-Transfer-Encoding: 7bit Cc: Xin LI , zfs-devel@FreeBSD.ORG, Pawel Jakub Dawidek Subject: Re: ZFS patch for review: osol rev. 10204, 10209 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: d@delphij.net List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 26 Oct 2010 08:25:04 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 Hi, On 10/25/10 03:46, Martin Matuska wrote: > Hi, I would like to propose the following two upstream changesets for > import: > > OpenSolaris revision 10204: (fixes for zfs receive) > Bug ID 6760420: zfs unmount -f causes recv to fail > Bug ID 6759975: zplprops unavailable to examiner of in-progress receive > > http://people.freebsd.org/~mm/patches/zfs/head-10204.patch > > OpenSolaris revision 10209: > Bug ID 6830541: zfs_get_data_trips on a verify > Bug ID 6696242: multiple zfs_fillpage() zfs: accessing past end of > object panics > Bug ID 6785914: zfs fails to drop dn_struct_rwlock in recovery code path > > http://people.freebsd.org/~mm/patches/zfs/head-10209.patch I have ran these patches on my laptop and checked with OpenSolaris code, I believe both are good for merge. Cheers, - -- Xin LI http://www.delphij.net/ FreeBSD - The Power to Serve! Live free or die -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.16 (FreeBSD) iQEcBAEBCAAGBQJMxpBKAAoJEATO+BI/yjfBF6QIAIGzbmx14mN2E2qYRaLuVF3r OCfThQpI5bUxALMRyYwTs2s5ybWsfyc0KguYffmoHZrv4f0KPJAwuxUzik4rFDDw FtJiblJTFfFXYkaSZk04nI/fS694xcnWkp2Jrn9h6Y1x1rn/d/liU2MTGWeIgOAo KvR3hF9cqvb5wWuJijy+BwiNp3x2Feb2HRQ7+391++9Yj7tOx1LaInmWlq8NdAaQ YhQ6LtusYHl1/CsvbitNWMMMQdx5kjnMqnTIwl6leVq/P5CmwWSG71TP68xQJP3k WarnguSDP0/m3SYrz0cga5iVb7YEDGYwi/IqQijsHe9y1LpKLPa9tHxV6GgfgwA= =P0dR -----END PGP SIGNATURE----- From owner-zfs-devel@FreeBSD.ORG Thu Oct 28 08:20:46 2010 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 454B5106564A; Thu, 28 Oct 2010 08:20:46 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from mail.vx.sk (mail.vx.sk [IPv6:2a01:4f8:100:1043::3]) by mx1.freebsd.org (Postfix) with ESMTP id CCE538FC08; Thu, 28 Oct 2010 08:20:45 +0000 (UTC) Received: from core.vx.sk (localhost [127.0.0.1]) by mail.vx.sk (Postfix) with ESMTP id 1DF8D126AC5; Thu, 28 Oct 2010 10:20:44 +0200 (CEST) X-Virus-Scanned: amavisd-new at mail.vx.sk Received: from mail.vx.sk ([127.0.0.1]) by core.vx.sk (mail.vx.sk [127.0.0.1]) (amavisd-new, port 10024) with LMTP id B3+vfEmPCQYZ; Thu, 28 Oct 2010 10:20:42 +0200 (CEST) Received: from [127.0.0.1] (unknown [217.67.16.66]) by mail.vx.sk (Postfix) with ESMTPSA id 2CF5A126AB6; Thu, 28 Oct 2010 10:20:42 +0200 (CEST) Message-ID: <4CC9325F.2030006@FreeBSD.org> Date: Thu, 28 Oct 2010 10:20:47 +0200 From: Martin Matuska User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; sk; rv:1.8.1.23) Gecko/20090812 Lightning/0.9 Thunderbird/2.0.0.23 Mnenhy/0.7.5.0 MIME-Version: 1.0 To: Pawel Jakub Dawidek , Xin Li X-Enigmail-Version: 1.1.1 Content-Type: text/plain; charset=windows-1250 Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: zfs-devel@freebsd.org Subject: Patch propisal: device opening and creation speedup X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Oct 2010 08:20:46 -0000 Hi, I would like to propose the following two upstream changesets for import: OpenSolaris revision 9846:6527c7b4a92e Bug ID 6566744: vdev_open() should be done in parallel OpenSolaris revision 10588:dc03f981ea18 (partial) - vdev.c change for zpools on top of zvol devices http://people.freebsd.org/~mm/patches/zfs/head-9846-10588p.patch The code is identical to state of V28 and produces a noticeable speedup of vdev_open() and vdev_create() on systems with many devices (e.g. storage servers - FreeNAS etc.). It would be good to have this in 8.2-RELEASE as well. From owner-zfs-devel@FreeBSD.ORG Thu Nov 4 16:56:17 2010 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C7B5C106566B; Thu, 4 Nov 2010 16:56:17 +0000 (UTC) (envelope-from avg@freebsd.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 70B638FC0A; Thu, 4 Nov 2010 16:56:15 +0000 (UTC) Received: from odyssey.starpoint.kiev.ua (alpha-e.starpoint.kiev.ua [212.40.38.101]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id SAA29317; Thu, 04 Nov 2010 18:56:11 +0200 (EET) (envelope-from avg@freebsd.org) Message-ID: <4CD2E5AA.1090208@freebsd.org> Date: Thu, 04 Nov 2010 18:56:10 +0200 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.2.11) Gecko/20101021 Lightning/1.0b2 Thunderbird/3.1.5 MIME-Version: 1.0 To: Kostik Belousov , Pawel Jakub Dawidek , Alan Cox References: <4C91F031.1010801@freebsd.org> <20101010134717.GA59922@deviant.kiev.zoral.com.ua> In-Reply-To: <20101010134717.GA59922@deviant.kiev.zoral.com.ua> X-Enigmail-Version: 1.1.2 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Mailman-Approved-At: Thu, 04 Nov 2010 17:25:47 +0000 Cc: freebsd-fs@freebsd.org Subject: Re: vop_getpages for zfs X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Nov 2010 16:56:18 -0000 on 10/10/2010 16:47 Kostik Belousov said the following: > I think that readahead requests should be satisfied, if possible. Esp. > in the case the readahead does not cause additional i/o. So, per Kostik's suggestion, here is patch for your review and testing: http://people.freebsd.org/~avg/zfs_getpages.2.diff The patch tries to fill as many of the passed in pages as possible as long as that can be done from a block (or even two blocks) that has to be read to fill the requested page. This means that the optimization kicks in only if a vnode has a block size greater than the page size. I mentioned "two blocks" above because I tried to optimize the case when, for example, block size is 6KB and page size is 4KB and the requested page covers parts of two adjacent blocks (e.g. range from 4KB to 8KB). Not sure if this was worth the trouble. In other words, the code tries to avoid reading more blocks than is strictly required, because additional blocks may be far apart on a disk and thus read latency could get increased. You will also notice that I added vop_bmap method to ZFS vops. This is needed to satisfy logic in vnode_pager_haspage(); otherwise, for example, vm_fault() would never request any additional pages besides the required one. zfs_bmap() implementation is really simple - it returns zeros for a_runp and a_runb (and some reasonable-ish but unused values in other fields), and vnode_pager_haspage() figures out number of eligible pages from the block size (f_iosize actually[*]) and the page size. Now, that ZFS has vop_getpages method, it is safe to provide this dummy-ish implementation of vop_bmap, because this operation would not be called by any other place but vnode_pager_haspage(). Code from vfs_bio.c and vfs_cluster.c is never called for ZFS, because it doesn't use the buffer cache layer. [*] There is a few interesting questions with respect to f_iosize usage in vnode_pager_haspage() and ZFS. For one, ZFS has a variable block size and f_iosize for ZFS has a currently configured maximum block ('record' in ZFS terms) size. This should not result in non-optimal behavior for files with smaller block sizes, because those files would have smaller sizes too (within the block size). Unless, of course, recordsize property is modified after some files are created on a ZFS filesystem. In that case there could be large files with some smaller (previously maximum) block size; or files with a block size larger than the current recordsize == f_iosize. So, such situations may lead to sub-optimal performance. Another issue related to the above is that f_iosize for ZFS can change on the fly when an administrators runs zfs set recordsize=NNN, and that may happen concurrently with vnode_pager_haspage() using f_iosize for its calculations. Not sure how to handle this correctly. I will appreciate your comment comments, test reports, reviews and suggestions. Thanks a lot! -- Andriy Gapon From owner-zfs-devel@FreeBSD.ORG Sat Nov 13 09:15:14 2010 Return-Path: Delivered-To: zfs-devel@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 150B3106564A for ; Sat, 13 Nov 2010 09:15:14 +0000 (UTC) (envelope-from delphij@delphij.net) Received: from tarsier.geekcn.org (tarsier.geekcn.org [IPv6:2001:470:a803::1]) by mx1.freebsd.org (Postfix) with ESMTP id 48E3C8FC17 for ; Sat, 13 Nov 2010 09:15:13 +0000 (UTC) Received: from mail.geekcn.org (tarsier.geekcn.org [211.166.10.233]) by tarsier.geekcn.org (Postfix) with ESMTP id 9AC2FA68247; Sat, 13 Nov 2010 17:15:10 +0800 (CST) X-Virus-Scanned: amavisd-new at geekcn.org Received: from tarsier.geekcn.org ([211.166.10.233]) by mail.geekcn.org (mail.geekcn.org [211.166.10.233]) (amavisd-new, port 10024) with LMTP id xZGweRuNh-WE; Sat, 13 Nov 2010 17:15:03 +0800 (CST) Received: from delta.delphij.net (c-76-102-26-215.hsd1.ca.comcast.net [76.102.26.215]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by tarsier.geekcn.org (Postfix) with ESMTPSA id 823A3A68219; Sat, 13 Nov 2010 17:15:02 +0800 (CST) DomainKey-Signature: a=rsa-sha1; s=default; d=delphij.net; c=nofws; q=dns; h=message-id:date:from:reply-to:organization:user-agent: mime-version:to:subject:x-enigmail-version:openpgp:content-type:content-transfer-encoding; b=Q3exD4DsTH9p+QNMOv1+M4dnNnq/y6fl7E1eT2++mycK+RhVO9EaT97Ud/DH14a4x pzBuLG9StxSElHr40uFaA== Message-ID: <4CDE5712.2040306@delphij.net> Date: Sat, 13 Nov 2010 01:14:58 -0800 From: Xin LI Organization: The FreeBSD Project User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.1.15) Gecko/20101028 Thunderbird/3.0.10 ThunderBrowse/3.3.2 MIME-Version: 1.0 To: zfs-devel@FreeBSD.ORG X-Enigmail-Version: 1.0.1 OpenPGP: id=3FCA37C1; url=http://www.delphij.net/delphij.asc Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Cc: Subject: Dead/Live lock near VOP_LOCK1_APV? X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: d@delphij.net List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 13 Nov 2010 09:15:14 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 Hi, I have observed a live/dead lock on a system serving both NFS and Samba CIFS, running 8.1-STABLE. Is this a known issue or should I investigate further? The system is not running with WITNESS but if further debugging is needed I can add it. procstat -kk shows something like: 7 100068 zfskern l2arc_feed_threa mi_switch sleepq_timedwait _cv_timedwait l2arc_feed_thread fork_exit fork_trampoline *7 100122 zfskern txg_thread_enter mi_switch sleepq_wait _cv_wait txg_thread_wait txg_quiesce_thread fork_exit fork_trampoline 7 100123 zfskern txg_thread_enter mi_switch sleepq_timedwait _cv_timedwait txg_thread_wait txg_sync_thread fork_exit fork_trampoline *7 100184 zfskern txg_thread_enter mi_switch sleepq_wait _cv_wait txg_thread_wait txg_quiesce_thread fork_exit fork_trampoline 7 100185 zfskern txg_thread_enter mi_switch sleepq_timedwait _cv_timedwait txg_thread_wait txg_sync_thread fork_exit fork_trampoline 1502 100284 nfsd nfsd: service mi_switch sleepq_wait __lockmgr_args vop_stdlock VOP_LOCK1_APV _vn_lock zfs_fhtovp nfsrv_fhtovp nfsrv_statfs nfssvc_program svc_run_internal svc_thread_start fork_exit fork_trampoline 27663 100779 smbd - mi_switch sleepq_wait __lockmgr_args vop_stdlock VOP_LOCK1_APV _vn_lock cache_lookup vfs_cache_lookup VOP_LOOK UP_APV lookup namei vn_open_cred kern_openat syscallenter syscall Xfast_syscall Full procstat -kk output can be found at: http://neptune.delphij.net/procstat.txt Cheers, - -- Xin LI http://www.delphij.net/ FreeBSD - The Power to Serve! Live free or die -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.16 (FreeBSD) iQEcBAEBCAAGBQJM3lcSAAoJEATO+BI/yjfB9NsIAKWvZ2NDaK1kNm2fQsmh+fmX jwMZAeceeySdCF3/VD5y9AQefGF4xUVRjJkN9iPnqFW2ggEhJbmftd9soVKXQlgK zO8Bky4roINImjwyz4bjfrkU/0yEXitgGpvusLTKZyDMNqqnMHZCkR8mthGGfpFU iCU3dpEI4I7Vm1sOf9tsGOsXS88cf/mbJupUI0IwUIjtfBIbZnoDQYq6oeFd3+L9 Vw2XjGCw5cqONTG6UKoixQbQ8UoD+chXUDbrf211ECe9fE8PDD5GAduQ4teLeH3G Tdsl3zLrw7plMXRR18lwx2oMLngJVnJrMJ7N5/LMnfVvwvK+h0ZzPMCyfErrNvs= =cclF -----END PGP SIGNATURE----- From owner-zfs-devel@FreeBSD.ORG Sat Nov 13 10:05:27 2010 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 380BE106566B for ; Sat, 13 Nov 2010 10:05:27 +0000 (UTC) (envelope-from avg@icyb.net.ua) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 7E17F8FC23 for ; Sat, 13 Nov 2010 10:05:26 +0000 (UTC) Received: from porto.topspin.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id LAA11724; Sat, 13 Nov 2010 11:47:33 +0200 (EET) (envelope-from avg@icyb.net.ua) Received: from localhost.topspin.kiev.ua ([127.0.0.1]) by porto.topspin.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1PHChp-000MrO-9x; Sat, 13 Nov 2010 11:47:33 +0200 Message-ID: <4CDE5E64.3090708@icyb.net.ua> Date: Sat, 13 Nov 2010 11:46:12 +0200 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.2.12) Gecko/20101029 Lightning/1.0b2 Thunderbird/3.1.6 MIME-Version: 1.0 To: d@delphij.net References: <4CDE5712.2040306@delphij.net> In-Reply-To: <4CDE5712.2040306@delphij.net> X-Enigmail-Version: 1.1.2 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Cc: zfs-devel@freebsd.org Subject: Re: Dead/Live lock near VOP_LOCK1_APV? X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 13 Nov 2010 10:05:27 -0000 on 13/11/2010 11:14 Xin LI said the following: > Hi, > > I have observed a live/dead lock on a system serving both NFS and Samba > CIFS, running 8.1-STABLE. Is this a known issue or should I investigate > further? Yes, it looks like a known issue. A fix has been recently committed and MFC-ed. It was a lock leak in NFS. > The system is not running with WITNESS but if further debugging is > needed I can add it. > > procstat -kk shows something like: > > 7 100068 zfskern l2arc_feed_threa mi_switch > sleepq_timedwait _cv_timedwait l2arc_feed_thread fork_exit fork_trampoline > *7 100122 zfskern txg_thread_enter mi_switch sleepq_wait > _cv_wait txg_thread_wait txg_quiesce_thread fork_exit fork_trampoline > 7 100123 zfskern txg_thread_enter mi_switch > sleepq_timedwait _cv_timedwait txg_thread_wait txg_sync_thread fork_exit > fork_trampoline > *7 100184 zfskern txg_thread_enter mi_switch sleepq_wait > _cv_wait txg_thread_wait txg_quiesce_thread fork_exit fork_trampoline > 7 100185 zfskern txg_thread_enter mi_switch > sleepq_timedwait _cv_timedwait txg_thread_wait txg_sync_thread fork_exit > fork_trampoline > > 1502 100284 nfsd nfsd: service mi_switch sleepq_wait > __lockmgr_args vop_stdlock VOP_LOCK1_APV _vn_lock zfs_fhtovp > nfsrv_fhtovp nfsrv_statfs nfssvc_program svc_run_internal > svc_thread_start fork_exit fork_trampoline > 27663 100779 smbd - mi_switch sleepq_wait > __lockmgr_args vop_stdlock VOP_LOCK1_APV _vn_lock cache_lookup > vfs_cache_lookup VOP_LOOK > UP_APV lookup namei vn_open_cred kern_openat syscallenter syscall > Xfast_syscall > > Full procstat -kk output can be found at: > > http://neptune.delphij.net/procstat.txt > > Cheers, _______________________________________________ zfs-devel@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/zfs-devel To unsubscribe, send any mail to "zfs-devel-unsubscribe@freebsd.org" -- Andriy Gapon From owner-zfs-devel@FreeBSD.ORG Mon Dec 13 22:33:48 2010 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id DCD1F1065670 for ; Mon, 13 Dec 2010 22:33:48 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (60.wheelsystems.com [83.12.187.60]) by mx1.freebsd.org (Postfix) with ESMTP id 879568FC15 for ; Mon, 13 Dec 2010 22:33:47 +0000 (UTC) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id A10A745CAC; Mon, 13 Dec 2010 23:01:05 +0100 (CET) Received: from localhost (89-73-192-49.dynamic.chello.pl [89.73.192.49]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id 713E145C99 for ; Mon, 13 Dec 2010 23:01:00 +0100 (CET) Date: Mon, 13 Dec 2010 23:00:56 +0100 From: Pawel Jakub Dawidek To: zfs-devel@FreeBSD.org Message-ID: <20101213220056.GG2038@garage.freebsd.pl> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="kjpMrWxdCilgNbo1" Content-Disposition: inline User-Agent: Mutt/1.4.2.3i X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 9.0-CURRENT amd64 X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-0.6 required=4.5 tests=BAYES_00,RCVD_IN_SORBS_DUL autolearn=no version=3.0.4 Cc: Subject: Fwd: Next ZFSv28 patchset ready for testing. X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 13 Dec 2010 22:33:48 -0000 --kjpMrWxdCilgNbo1 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable FYI: The new patchset is ready for testing: http://people.freebsd.org/~pjd/patches/zfs_20101212.patch.bz2 When applying the patch be sure to use correct options for patch(1)!: # cd /usr/src # fetch http://people.freebsd.org/~pjd/patches/zfs_20101212.patch.bz2 # bzip2 -d zfs_20101212.patch.bz2 # patch -E -p0 < zfs_20101212.patch The patch is against FreeBSD HEAD as of 2010-12-12. Some of the changes since the last patchset (zfs_20100831.patch): - Boot support for ZFS v28 (only RAIDZ3 is not yet supported). - Various fixes for the existing ZFS boot code. - Support for sendfile(2) (by avg@). - Userland<->kernel compatibility with v13-v15 (by mm@). - ACL fixes (by trasz@). - Various bug fixes. Please test, test, test. Chances are this is the last patchset before v28 going to HEAD (finally). Especially test new changes, like boot support and sendfile(2) support. Also be sure to verify if you can import for existing ZFS pools (v13-v15) when running v28 or boot from your existing pools. Enjoy! PS. Martin (mm@) will be providing patch against 8-STABLE soon. --=20 Pawel Jakub Dawidek http://www.wheelsystems.com pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --kjpMrWxdCilgNbo1 Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (FreeBSD) iEUEARECAAYFAk0Gl5cACgkQForvXbEpPzR/zwCXc1bmawHui1+BqpzvD/FdUTfw gQCdFolhu/bZXcXBJ2lYXs7l3DyrfpU= =K1Xb -----END PGP SIGNATURE----- --kjpMrWxdCilgNbo1-- From owner-zfs-devel@FreeBSD.ORG Mon Jan 31 21:58:00 2011 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 86106106566B for ; Mon, 31 Jan 2011 21:58:00 +0000 (UTC) (envelope-from gibbs@scsiguy.com) Received: from aslan.scsiguy.com (ns1.scsiguy.com [70.89.174.89]) by mx1.freebsd.org (Postfix) with ESMTP id 3AAE08FC29 for ; Mon, 31 Jan 2011 21:57:59 +0000 (UTC) Received: from [192.168.4.138] (207-225-98-3.dia.static.qwest.net [207.225.98.3]) (authenticated bits=0) by aslan.scsiguy.com (8.14.4/8.14.4) with ESMTP id p0VLdCJv074828 (version=TLSv1/SSLv3 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO) for ; Mon, 31 Jan 2011 14:39:13 -0700 (MST) (envelope-from gibbs@scsiguy.com) Message-ID: <4D472BFC.5070404@scsiguy.com> Date: Mon, 31 Jan 2011 14:39:08 -0700 From: "Justin T. Gibbs" Organization: SCSIGuy.com User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.2.14) Gecko/20110123 Thunderbird/3.1.8 MIME-Version: 1.0 To: zfs-devel@freebsd.org Content-Type: multipart/mixed; boundary="------------090502030703040508070704" X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.6 (aslan.scsiguy.com [70.89.174.89]); Mon, 31 Jan 2011 14:39:13 -0700 (MST) Subject: Device departure processing X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: gibbs@scsiguy.com List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 31 Jan 2011 21:58:00 -0000 This is a multi-part message in MIME format. --------------090502030703040508070704 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit On both -current and -stable (v16), I've been able to reliably panic the system by pulling a drive (leaf vdev in a RAIDZ2) while I/O is in progress. The source of the panic is either from GEOM complaining that vdev_geom's consumer isn't idle, or "use after free" bugs caused by in-flight I/O (often due to a re-probe). The following patch resolves the issue for me. >From a brief review of the v28 code, it seems appropriate there too. The code in vdev_geom_orphan is identical save for an added call to zfs_post_remove(). In our port, this call appears to do nothing more than to post the devctl notice event a little earlier than via the scheduled removal of the device by the spa code. Is there some benefit I'm missing that makes the zfs_post_remove() necessary? -- Justin --------------090502030703040508070704 Content-Type: text/plain; name="vdev_geom.c.diff" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="vdev_geom.c.diff" Change 475132 by justing@justing-ns1 on 2011/01/14 13:08:34 Correct ZFS panic during device removal. sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_geom.c: When the ZFS geom device consumer is orphaned, simply signal the SPA to begin device removal proceedings instead of attempting to destroy the consumer directly. The orphan handler is called with the geom topology lock held from the geom event thread. In order to perform a complete consumer shutdown from this context, the SPA ZIO configuration lock would need to be held as a writer to insure that no ZIOs are active on this consumer. This lock acquisition order is the reverse of the normal SPA config lock -> geom topology lock order. We could probably drop locks and acquire in the correct order, but it is much simpler to just defer all of this until the SPA's eventual close of the vdev_geom instance. Affected files ... ... //depot/SpectraBSD/head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_geom.c#7 edit Differences ... ==== //depot/SpectraBSD/head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_geom.c#7 (text) ==== @@ -50,28 +50,26 @@ static void vdev_geom_orphan(struct g_consumer *cp) { - struct g_geom *gp; vdev_t *vd; - int error; g_topology_assert(); vd = cp->private; - gp = cp->geom; - error = cp->provider->error; - ZFS_LOG(1, "Closing access to %s.", cp->provider->name); - if (cp->acr + cp->acw + cp->ace > 0) - g_access(cp, -cp->acr, -cp->acw, -cp->ace); - ZFS_LOG(1, "Destroyed consumer to %s.", cp->provider->name); - g_detach(cp); - g_destroy_consumer(cp); - /* Destroy geom if there are no consumers left. */ - if (LIST_EMPTY(&gp->consumer)) { - ZFS_LOG(1, "Destroyed geom %s.", gp->name); - g_wither_geom(gp, error); - } - vd->vdev_tsd = NULL; + /* + * Orphan callbacks occur from the GEOM event thread. + * Concurrent with this call, new I/O requests may be + * working their way through GEOM about to find out + * (only once executed by the g_down thread) that we've + * been orphaned from our disk provider. These I/Os + * must be retired before we can detach our consumer. + * This is most easily achieved by acquiring the + * SPA ZIO configuration lock as a writer, but doing + * so with the GEOM topology lock held would cause + * a lock order reversal. Instead, rely on the SPA's + * async removal support to invoke a close on this + * vdev once it is safe to do so. + */ vd->vdev_remove_wanted = B_TRUE; spa_async_request(vd->vdev_spa, SPA_ASYNC_REMOVE); } --------------090502030703040508070704-- From owner-zfs-devel@FreeBSD.ORG Fri Feb 4 23:11:18 2011 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6B7071065672 for ; Fri, 4 Feb 2011 23:11:18 +0000 (UTC) (envelope-from pawel@dawidek.net) Received: from mail.garage.freebsd.pl (60.wheelsystems.com [83.12.187.60]) by mx1.freebsd.org (Postfix) with ESMTP id 1719A8FC08 for ; Fri, 4 Feb 2011 23:11:18 +0000 (UTC) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id E529645E93; Fri, 4 Feb 2011 23:52:55 +0100 (CET) Received: from localhost (89-73-195-149.dynamic.chello.pl [89.73.195.149]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id CA6AD4569A; Fri, 4 Feb 2011 23:52:50 +0100 (CET) Date: Fri, 4 Feb 2011 23:52:34 +0100 From: Pawel Jakub Dawidek To: "Justin T. Gibbs" Message-ID: <20110204225234.GP2035@garage.freebsd.pl> References: <4D472BFC.5070404@scsiguy.com> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="/d7X7C0hV/blnKmH" Content-Disposition: inline In-Reply-To: <4D472BFC.5070404@scsiguy.com> User-Agent: Mutt/1.4.2.3i X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 9.0-CURRENT amd64 X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-0.6 required=4.5 tests=BAYES_00,RCVD_IN_SORBS_DUL autolearn=no version=3.0.4 Cc: zfs-devel@freebsd.org Subject: Re: Device departure processing X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 04 Feb 2011 23:11:18 -0000 --/d7X7C0hV/blnKmH Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Mon, Jan 31, 2011 at 02:39:08PM -0700, Justin T. Gibbs wrote: > On both -current and -stable (v16), I've been able to reliably panic the > system by pulling a drive (leaf vdev in a RAIDZ2) while I/O is > in progress. The source of the panic is either from GEOM complaining > that vdev_geom's consumer isn't idle, or "use after free" bugs caused > by in-flight I/O (often due to a re-probe). The following patch resolves > the issue for me. I was able to reproduce the panic on v28 and I integrated your patch there. > >From a brief review of the v28 code, it seems appropriate there too. > The code in vdev_geom_orphan is identical save for an added call to > zfs_post_remove(). In our port, this call appears to do nothing more > than to post the devctl notice event a little earlier than via the > scheduled removal of the device by the spa code. Is there some benefit > I'm missing that makes the zfs_post_remove() necessary? Without it I see no info about vdev removal in devd. --=20 Pawel Jakub Dawidek http://www.wheelsystems.com pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --/d7X7C0hV/blnKmH Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (FreeBSD) iEYEARECAAYFAk1MgzEACgkQForvXbEpPzT5tQCgwy7D63QdPQBkVPdSxIXJ3NiP CKIAoIgAo6Ce/X0l5YmwKJHo7x3lKzM1 =d7XB -----END PGP SIGNATURE----- --/d7X7C0hV/blnKmH-- From owner-zfs-devel@FreeBSD.ORG Sun Feb 6 02:56:07 2011 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2AE16106564A; Sun, 6 Feb 2011 02:56:07 +0000 (UTC) (envelope-from gibbs@scsiguy.com) Received: from aslan.scsiguy.com (ns1.scsiguy.com [70.89.174.89]) by mx1.freebsd.org (Postfix) with ESMTP id EDAA98FC08; Sun, 6 Feb 2011 02:56:06 +0000 (UTC) Received: from [192.168.0.21] ([192.168.0.21]) (authenticated bits=0) by aslan.scsiguy.com (8.14.4/8.14.4) with ESMTP id p162u5oJ031680 (version=TLSv1/SSLv3 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Sat, 5 Feb 2011 19:56:05 -0700 (MST) (envelope-from gibbs@scsiguy.com) Message-ID: <4D4E0DC1.4030002@scsiguy.com> Date: Sat, 05 Feb 2011 19:56:01 -0700 From: "Justin T. Gibbs" Organization: SCSIGuy.com User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.2.14) Gecko/20110123 Thunderbird/3.1.8 MIME-Version: 1.0 To: Pawel Jakub Dawidek References: <4D472BFC.5070404@scsiguy.com> <20110204225234.GP2035@garage.freebsd.pl> In-Reply-To: <20110204225234.GP2035@garage.freebsd.pl> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.6 (aslan.scsiguy.com [70.89.174.89]); Sat, 05 Feb 2011 19:56:05 -0700 (MST) Cc: zfs-devel@FreeBSD.org Subject: Re: Device departure processing X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: gibbs@scsiguy.com List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Feb 2011 02:56:07 -0000 On 2/4/2011 3:52 PM, Pawel Jakub Dawidek wrote: > On Mon, Jan 31, 2011 at 02:39:08PM -0700, Justin T. Gibbs wrote: > >> >From a brief review of the v28 code, it seems appropriate there too. >> The code in vdev_geom_orphan is identical save for an added call to >> zfs_post_remove(). In our port, this call appears to do nothing more >> than to post the devctl notice event a little earlier than via the >> scheduled removal of the device by the spa code. Is there some benefit >> I'm missing that makes the zfs_post_remove() necessary? > > Without it I see no info about vdev removal in devd. Do you know why the call to zfs_post_remove() was removed from vdev_set_state()? This is where the notification is made in -current (v15). It seems wrong to put this policy into each vdev driver instead of just doing it in the common vdev code. -- Justin From owner-zfs-devel@FreeBSD.ORG Sun Feb 6 14:30:25 2011 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 28F98106564A for ; Sun, 6 Feb 2011 14:30:25 +0000 (UTC) (envelope-from pawel@dawidek.net) Received: from mail.garage.freebsd.pl (60.wheelsystems.com [83.12.187.60]) by mx1.freebsd.org (Postfix) with ESMTP id A82608FC18 for ; Sun, 6 Feb 2011 14:30:24 +0000 (UTC) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id 1348B45E9D; Sun, 6 Feb 2011 15:30:23 +0100 (CET) Received: from localhost (89-73-195-149.dynamic.chello.pl [89.73.195.149]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id 55C024569A; Sun, 6 Feb 2011 15:30:17 +0100 (CET) Date: Sun, 6 Feb 2011 15:29:56 +0100 From: Pawel Jakub Dawidek To: "Justin T. Gibbs" Message-ID: <20110206142956.GG2035@garage.freebsd.pl> References: <4D472BFC.5070404@scsiguy.com> <20110204225234.GP2035@garage.freebsd.pl> <4D4E0DC1.4030002@scsiguy.com> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="+bs7B30DeWCM5QK8" Content-Disposition: inline In-Reply-To: <4D4E0DC1.4030002@scsiguy.com> User-Agent: Mutt/1.4.2.3i X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 9.0-CURRENT amd64 X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-0.6 required=4.5 tests=BAYES_00,RCVD_IN_SORBS_DUL autolearn=no version=3.0.4 Cc: zfs-devel@FreeBSD.org Subject: Re: Device departure processing X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Feb 2011 14:30:25 -0000 --+bs7B30DeWCM5QK8 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sat, Feb 05, 2011 at 07:56:01PM -0700, Justin T. Gibbs wrote: > On 2/4/2011 3:52 PM, Pawel Jakub Dawidek wrote: > > On Mon, Jan 31, 2011 at 02:39:08PM -0700, Justin T. Gibbs wrote: > >=20 > >> >From a brief review of the v28 code, it seems appropriate there too. > >> The code in vdev_geom_orphan is identical save for an added call to > >> zfs_post_remove(). In our port, this call appears to do nothing more > >> than to post the devctl notice event a little earlier than via the > >> scheduled removal of the device by the spa code. Is there some benefit > >> I'm missing that makes the zfs_post_remove() necessary? > >=20 > > Without it I see no info about vdev removal in devd. >=20 > Do you know why the call to zfs_post_remove() was removed from > vdev_set_state()? This is where the notification is made in -current (v1= 5). > It seems wrong to put this policy into each vdev driver instead of > just doing it in the common vdev code. This was changed in 2a8816c5173b. And the added comment states: /* * We post the resource as soon as possible, instead of * when the async removal actually happens, because the * DE is using this information to discard previous I/O * errors. */ zfs_post_remove(zio->io_spa, vd); This is from vdev_disk.c. --=20 Pawel Jakub Dawidek http://www.wheelsystems.com pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --+bs7B30DeWCM5QK8 Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (FreeBSD) iEYEARECAAYFAk1OsGQACgkQForvXbEpPzS4rACg99/PHRApFriPZbN1kFhovYTC pJYAnA4wiPAePum5Di8SJK7/3ocJpTuo =c49u -----END PGP SIGNATURE----- --+bs7B30DeWCM5QK8-- From owner-zfs-devel@FreeBSD.ORG Fri May 6 23:43:27 2011 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: by hub.freebsd.org (Postfix, from userid 1233) id BDFBF1065675; Fri, 6 May 2011 23:43:27 +0000 (UTC) Date: Fri, 6 May 2011 23:43:27 +0000 From: Alexander Best To: zfs-devel@freebsd.org Message-ID: <20110506234327.GA71369@freebsd.org> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="x+6KMIRAuhnl3hBn" Content-Disposition: inline Subject: lots of missing declarations X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 May 2011 23:43:27 -0000 --x+6KMIRAuhnl3hBn Content-Type: text/plain; charset=us-ascii Content-Disposition: inline hi everybody, i'm trying to successfully run 'make MAKE_JUST_KERNELS=yes tinderbox' with the -Wmissing-declarations flag. apart from one issue in the xfs code, all warnings issued by this flag come from inside the zfs sources. i've attached a log file, which documents all of them. is there any possibility these can be fixed and we can then permanently add -Wmissing-declarations to CWARNFLAGS in sys/conf/kern.mk? cheers. alex -- a13x --x+6KMIRAuhnl3hBn Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename=zfs-missing-decl /usr/git-freebsd-head/sys/modules/zfs/../../cddl/compat/opensolaris/kern/opensolaris_kobj.c:169: warning: no previous declaration for 'kobj_read_file_vnode' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/compat/opensolaris/kern/opensolaris_kobj.c:200: warning: no previous declaration for 'kobj_read_file_loader' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/os/callb.c:99: warning: no previous declaration for 'callb_init' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/os/callb.c:107: warning: no previous declaration for 'callb_fini' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/os/fm.c:1167: warning: no previous declaration for 'fm_ena_generate_cpu' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/zmod/zmod_subr.c:43: warning: no previous declaration for 'zcalloc' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/zmod/zmod_subr.c:59: warning: no previous declaration for 'zcfree' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/zmod/zmod_subr.c:70: warning: no previous declaration for 'zmemcpy' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/zmod/zmod_subr.c:76: warning: no previous declaration for 'zmemcmp' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/zmod/zmod_subr.c:82: warning: no previous declaration for 'zmemzero' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:1651: warning: no previous declaration for 'arc_buf_free' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:2183: warning: no previous declaration for 'arc_shrink' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/ddt.c:612: warning: no previous declaration for 'ddt_select_by_checksum' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/dsl_pool.c:806: warning: no previous declaration for 'dsl_pool_user_hold_create_obj' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/dsl_prop.c:550: warning: no previous declaration for 'dsl_prop_set_sync' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/gzip.c:41: warning: no previous declaration for 'gzip_compress' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/gzip.c:60: warning: no previous declaration for 'gzip_decompress' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/lzjb.c:51: warning: no previous declaration for 'lzjb_compress' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/lzjb.c:99: warning: no previous declaration for 'lzjb_decompress' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/metaslab.c:445: warning: no previous declaration for 'metaslab_pp_maxsize' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/metaslab.c:473: warning: no previous declaration for 'metaslab_ff_fragmented' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/sa.c:276: warning: no previous declaration for 'sa_layout_equal' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/sa.c:327: warning: no previous declaration for 'sa_attr_op' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/sa.c:1131: warning: no previous declaration for 'sa_build_idx_tab' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/sa.c:1195: warning: no previous declaration for 'sa_byteswap_cb' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/sa.c:1204: warning: no previous declaration for 'sa_byteswap' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/sa.c:1279: warning: no previous declaration for 'sa_evict' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/sa.c:1409: warning: no previous declaration for 'sa_lookup_impl' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/sha256.c:35: warning: no previous declaration for 'zio_checksum_SHA256' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/spa.c:4924: warning: no previous declaration for 'spa_vdev_set_common' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/spa_misc.c:646: warning: no previous declaration for 'spa_aux_add' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/spa_misc.c:664: warning: no previous declaration for 'spa_aux_remove' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/spa_misc.c:684: warning: no previous declaration for 'spa_aux_exists' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/spa_misc.c:709: warning: no previous declaration for 'spa_aux_activate' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/vdev.c:1778: warning: no previous declaration for 'vdev_dtl_sync' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/vdev.c:1987: warning: no previous declaration for 'vdev_remove' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_label.c:963: warning: no previous declaration for 'vdev_uberblock_sync_list' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_label.c:1072: warning: no previous declaration for 'vdev_label_sync_list' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:91: warning: no previous declaration for 'vdev_queue_deadline_compare' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:115: warning: no previous declaration for 'vdev_queue_offset_compare' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zap_micro.c:203: warning: no previous declaration for 'zap_name_alloc_uint64' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_znode.c:1998: warning: no previous declaration for 'zfs_release_sa_handle' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1447: warning: no previous declaration for 'zio_rewrite_gang' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1485: warning: no previous declaration for 'zio_free_gang' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1493: warning: no previous declaration for 'zio_claim_gang' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zle.c:38: warning: no previous declaration for 'zle_compress' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zle.c:68: warning: no previous declaration for 'zle_decompress' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zrlock.c:165: warning: no previous declaration for 'zrl_refcount' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/common/zfs/zfs_fletcher.c:136: warning: no previous declaration for 'fletcher_2_native' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/common/zfs/zfs_fletcher.c:153: warning: no previous declaration for 'fletcher_2_byteswap' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/common/zfs/zfs_fletcher.c:170: warning: no previous declaration for 'fletcher_4_native' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/common/zfs/zfs_fletcher.c:187: warning: no previous declaration for 'fletcher_4_byteswap' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/common/zfs/zfs_fletcher.c:205: warning: no previous declaration for 'fletcher_4_incremental_native' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/common/zfs/zfs_fletcher.c:228: warning: no previous declaration for 'fletcher_4_incremental_byteswap' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_acl.c:658: warning: no previous declaration for 'zfs_copy_ace_2_fuid' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_ctldir.c:583: warning: no previous declaration for 'zfsctl_freebsd_root_lookup' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_ctldir.c:919: warning: no previous declaration for 'zfsctl_snapdir_lookup' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_ctldir.c:1082: warning: no previous declaration for 'zfsctl_shares_lookup' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_ioctl.c:350: warning: no previous declaration for 'zfs_secpolicy_write_perms' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_ioctl.c:365: warning: no previous declaration for 'zfs_secpolicy_write_perms_ds' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_ioctl.c:534: warning: no previous declaration for 'zfs_secpolicy_fsacl' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_ioctl.c:550: warning: no previous declaration for 'zfs_secpolicy_rollback' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_ioctl.c:557: warning: no previous declaration for 'zfs_secpolicy_send' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_ioctl.c:618: warning: no previous declaration for 'zfs_secpolicy_share' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_ioctl.c:631: warning: no previous declaration for 'zfs_secpolicy_smb_acl' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_ioctl.c:1904: warning: no previous declaration for 'dataset_name_hidden' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_ioctl.c:4786: warning: no previous declaration for 'pool_status_check' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vfsops.c:1175: warning: no previous declaration for 'zfs_unregister_callbacks' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vfsops.c:2259: warning: no previous declaration for 'zfs_init' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vfsops.c:2285: warning: no previous declaration for 'zfs_fini' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c:1078: warning: no previous declaration for 'zfs_get_done' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c:4526: warning: no previous declaration for 'zfs_inactive' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c:6354: warning: no previous declaration for 'zfs_deleteextattr' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c:6597: warning: no previous declaration for 'zfs_freebsd_getacl' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c:6624: warning: no previous declaration for 'zfs_freebsd_setacl' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c:6671: warning: no previous declaration for 'zfs_freebsd_aclcheck' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zvol.c:637: warning: no previous declaration for 'zvol_first_open' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zvol.c:677: warning: no previous declaration for 'zvol_last_close' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zvol.c:729: warning: no previous declaration for 'zvol_update_volsize' /usr/git-freebsd-head/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zvol.c:1194: warning: no previous declaration for 'zvol_strategy' --x+6KMIRAuhnl3hBn-- From owner-zfs-devel@FreeBSD.ORG Mon Jun 20 09:15:24 2011 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D2731106566B; Mon, 20 Jun 2011 09:15:24 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from mail.vx.sk (mail.vx.sk [IPv6:2a01:4f8:100:1043::3]) by mx1.freebsd.org (Postfix) with ESMTP id 6C1048FC1B; Mon, 20 Jun 2011 09:15:24 +0000 (UTC) Received: from core.vx.sk (localhost [127.0.0.1]) by mail.vx.sk (Postfix) with ESMTP id 9C51917912B; Mon, 20 Jun 2011 11:15:23 +0200 (CEST) X-Virus-Scanned: amavisd-new at mail.vx.sk Received: from mail.vx.sk ([127.0.0.1]) by core.vx.sk (mail.vx.sk [127.0.0.1]) (amavisd-new, port 10024) with LMTP id FE5ft0ggqakW; Mon, 20 Jun 2011 11:15:21 +0200 (CEST) Received: from [10.0.3.3] (188-167-66-148.dynamic.chello.sk [188.167.66.148]) by mail.vx.sk (Postfix) with ESMTPSA id DAAD5179117; Mon, 20 Jun 2011 11:15:20 +0200 (CEST) Message-ID: <4DFF0FA8.5070805@FreeBSD.org> Date: Mon, 20 Jun 2011 11:15:20 +0200 From: Martin Matuska User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.17) Gecko/20110516 Thunderbird/3.1.10 MIME-Version: 1.0 To: zfs-devel@FreeBSD.org Content-Type: multipart/mixed; boundary="------------080801060903000605080804" Cc: Pawel Jakub Dawidek Subject: [CFR][ZFS] merge Illumos rev. 13295 (zfs get /path) X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 20 Jun 2011 09:15:24 -0000 This is a multi-part message in MIME format. --------------080801060903000605080804 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit 'zfs get' command should be able to deal with mountpoint as an argument. It already works with 'zfs list' command. https://www.illumos.org/issues/510 https://www.illumos.org/projects/illumos-gate/repository/revisions/13295 If there are no objections, I will commit this to -HEAD. -- Martin Matuska FreeBSD committer http://blog.vx.sk --------------080801060903000605080804 Content-Type: text/x-patch; name="13295.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="13295.patch" Index: cddl/contrib/opensolaris/cmd/zfs/zfs_main.c =================================================================== --- cddl/contrib/opensolaris/cmd/zfs/zfs_main.c (revision 223325) +++ cddl/contrib/opensolaris/cmd/zfs/zfs_main.c (working copy) @@ -21,7 +21,7 @@ /* * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved. - * Copyright 2010 Nexenta Systems, Inc. All rights reserved. + * Copyright 2011 Nexenta Systems, Inc. All rights reserved. */ #include @@ -1292,7 +1292,7 @@ zfs_do_get(int argc, char **argv) { zprop_get_cbdata_t cb = { 0 }; - int i, c, flags = 0; + int i, c, flags = ZFS_ITER_ARGS_CAN_BE_PATHS; char *value, *fields; int ret; int limit = 0; --------------080801060903000605080804-- From owner-zfs-devel@FreeBSD.ORG Mon Jun 20 09:16:15 2011 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2BA50106566B; Mon, 20 Jun 2011 09:16:15 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from mail.vx.sk (mail.vx.sk [IPv6:2a01:4f8:100:1043::3]) by mx1.freebsd.org (Postfix) with ESMTP id E0C6A8FC0A; Mon, 20 Jun 2011 09:16:14 +0000 (UTC) Received: from core.vx.sk (localhost [127.0.0.1]) by mail.vx.sk (Postfix) with ESMTP id 43C121791A4; Mon, 20 Jun 2011 11:16:14 +0200 (CEST) X-Virus-Scanned: amavisd-new at mail.vx.sk Received: from mail.vx.sk ([127.0.0.1]) by core.vx.sk (mail.vx.sk [127.0.0.1]) (amavisd-new, port 10024) with LMTP id uWS8aXhrBjaR; Mon, 20 Jun 2011 11:16:09 +0200 (CEST) Received: from [10.0.3.3] (188-167-66-148.dynamic.chello.sk [188.167.66.148]) by mail.vx.sk (Postfix) with ESMTPSA id B3492179185; Mon, 20 Jun 2011 11:16:08 +0200 (CEST) Message-ID: <4DFF0FD8.6040708@FreeBSD.org> Date: Mon, 20 Jun 2011 11:16:08 +0200 From: Martin Matuska User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.17) Gecko/20110516 Thunderbird/3.1.10 MIME-Version: 1.0 To: zfs-devel@FreeBSD.org Content-Type: multipart/mixed; boundary="------------060909070202060002080902" Cc: Pawel Jakub Dawidek Subject: [CFR][ZFS] merge Illumos rev. 13346 (set vdev_cache to zero) X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 20 Jun 2011 09:16:15 -0000 This is a multi-part message in MIME format. --------------060909070202060002080902 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit zfs vdev cache (readahead) consumes excessive memory and is very underutilized. Set zfs_vdev_cache_size to zero. This change is present in Solaris 11. https://www.illumos.org/issues/175 https://www.illumos.org/projects/illumos-gate/repository/revisions/13346 If there are no objections, I will commit this to -HEAD. -- Martin Matuska FreeBSD committer http://blog.vx.sk --------------060909070202060002080902 Content-Type: text/x-patch; name="13346.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="13346.patch" Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_cache.c =================================================================== --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_cache.c (revision 223325) +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_cache.c (working copy) @@ -71,9 +71,16 @@ * 1< Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A77BB106564A; Mon, 20 Jun 2011 09:16:40 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from mail.vx.sk (mail.vx.sk [IPv6:2a01:4f8:100:1043::3]) by mx1.freebsd.org (Postfix) with ESMTP id 06A2D8FC13; Mon, 20 Jun 2011 09:16:40 +0000 (UTC) Received: from core.vx.sk (localhost [127.0.0.1]) by mail.vx.sk (Postfix) with ESMTP id 669FA1791F8; Mon, 20 Jun 2011 11:16:39 +0200 (CEST) X-Virus-Scanned: amavisd-new at mail.vx.sk Received: from mail.vx.sk ([127.0.0.1]) by core.vx.sk (mail.vx.sk [127.0.0.1]) (amavisd-new, port 10024) with LMTP id CPgBW3+n-63O; Mon, 20 Jun 2011 11:16:36 +0200 (CEST) Received: from [10.0.3.3] (188-167-66-148.dynamic.chello.sk [188.167.66.148]) by mail.vx.sk (Postfix) with ESMTPSA id 1A87C1791E7; Mon, 20 Jun 2011 11:16:36 +0200 (CEST) Message-ID: <4DFF0FF3.1030308@FreeBSD.org> Date: Mon, 20 Jun 2011 11:16:35 +0200 From: Martin Matuska User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.17) Gecko/20110516 Thunderbird/3.1.10 MIME-Version: 1.0 To: zfs-devel@FreeBSD.org Content-Type: multipart/mixed; boundary="------------010907030303010503000607" Cc: Pawel Jakub Dawidek , Edward Tomasz Napierala Subject: [CFR][ZFS] merge Illumos rev. 13370 (resurrect "aclmode" property) X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 20 Jun 2011 09:16:40 -0000 This is a multi-part message in MIME format. --------------010907030303010503000607 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Resurrect "aclmode" property for ZFS. The Illumos changeset includes Issues submitted by trasz (279, 664). https://www.illumos.org/issues/742 https://www.illumos.org/issues/664 https://www.illumos.org/issues/279 https://www.illumos.org/projects/illumos-gate/repository/revisions/13370 If there are no objections, I will commit this to -HEAD. -- Martin Matuska FreeBSD committer http://blog.vx.sk --------------010907030303010503000607 Content-Type: text/x-patch; name="13370.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="13370.patch" Index: cddl/contrib/opensolaris/cmd/zfs/zfs.8 =================================================================== --- cddl/contrib/opensolaris/cmd/zfs/zfs.8 (revision 223325) +++ cddl/contrib/opensolaris/cmd/zfs/zfs.8 (working copy) @@ -6,6 +6,7 @@ .\" The contents of this file are subject to the terms of the Common Development and Distribution License (the "License"). You may not use this file except in compliance with the License. You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE or http://www.opensolaris.org/os/licensing. .\" See the License for the specific language governing permissions and limitations under the License. When distributing Covered Code, include this CDDL HEADER in each file and include the License file at usr/src/OPENSOLARIS.LICENSE. If applicable, add the following below this CDDL HEADER, with .\" the fields enclosed by brackets "[]" replaced with your own identifying information: Portions Copyright [yyyy] [name of copyright owner] +.\" Copyright 2011 Nexenta Systems, Inc. All rights reserved. .TH zfs 1M "24 Sep 2009" "SunOS 5.11" "System Administration Commands" .SH NAME zfs \- configures ZFS file systems @@ -630,7 +631,7 @@ .ad .sp .6 .RS 4n -Controls how an \fBACL\fR is modified during \fBchmod\fR(2). A file system with an \fBaclmode\fR property of \fBdiscard\fR deletes all \fBACL\fR entries that do not represent the mode of the file. An \fBaclmode\fR property of \fBgroupmask\fR (the default) reduces user or group permissions. The permissions are reduced, such that they are no greater than the group permission bits, unless it is a user entry that has the same \fBUID\fR as the owner of the file or directory. In this case, the \fBACL\fR permissions are reduced so that they are no greater than owner permission bits. A file system with an \fBaclmode\fR property of \fBpassthrough\fR indicates that no changes are made to the \fBACL\fR other than generating the necessary \fBACL\fR entries to represent the new mode of the file or directory. +Controls how an \fBACL\fR is modified during \fBchmod\fR(2). A file system with an \fBaclmode\fR property of \fBdiscard\fR (the default) deletes all \fBACL\fR entries that do not represent the mode of the file. An \fBaclmode\fR property of \fBgroupmask\fR reduces permissions granted in all \fBALLOW\fR entries found in the \fBACL\fR such that they are no greater than the group permissions specified by \fBchmod\fR. A file system with an \fBaclmode\fR property of \fBpassthrough\fR indicates that no changes are made to the \fBACL\fR other than creating or updating the necessary \fBACL\fR entries to represent the new mode of the file or directory. .RE .sp @@ -2685,7 +2686,7 @@ pool/home/bob readonly off default pool/home/bob zoned off default pool/home/bob snapdir hidden default -pool/home/bob aclmode groupmask default +pool/home/bob aclmode discard default pool/home/bob aclinherit restricted default pool/home/bob canmount on default pool/home/bob shareiscsi off default Index: sys/cddl/contrib/opensolaris/common/acl/acl_common.c =================================================================== --- sys/cddl/contrib/opensolaris/common/acl/acl_common.c (revision 223325) +++ sys/cddl/contrib/opensolaris/common/acl/acl_common.c (working copy) @@ -20,6 +20,7 @@ */ /* * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved. + * Copyright 2011 Nexenta Systems, Inc. All rights reserved. */ #include @@ -376,7 +377,7 @@ * by nfsace, assuming aclent_t -> nfsace semantics. */ static uint32_t -mode_to_ace_access(mode_t mode, int isdir, int isowner, int isallow) +mode_to_ace_access(mode_t mode, boolean_t isdir, int isowner, int isallow) { uint32_t access = 0; int haswriteperm = 0; @@ -419,7 +420,7 @@ access |= ACE_DELETE_CHILD; } /* exec */ - if (mode & 01) { + if (mode & S_IXOTH) { access |= ACE_EXECUTE; } @@ -670,7 +671,7 @@ } static int -convert_aent_to_ace(aclent_t *aclentp, int aclcnt, int isdir, +convert_aent_to_ace(aclent_t *aclentp, int aclcnt, boolean_t isdir, ace_t **retacep, int *retacecnt) { ace_t *acep; @@ -696,7 +697,7 @@ dfaclcnt = aclcnt - i; } - if (dfaclcnt && isdir == 0) { + if (dfaclcnt && !isdir) { return (EINVAL); } @@ -734,7 +735,7 @@ } static int -ace_mask_to_mode(uint32_t mask, o_mode_t *modep, int isdir) +ace_mask_to_mode(uint32_t mask, o_mode_t *modep, boolean_t isdir) { int error = 0; o_mode_t mode = 0; @@ -1031,7 +1032,7 @@ } static int -ace_allow_to_mode(uint32_t mask, o_mode_t *modep, int isdir) +ace_allow_to_mode(uint32_t mask, o_mode_t *modep, boolean_t isdir) { /* ACE_READ_ACL and ACE_READ_ATTRIBUTES must both be set */ if ((mask & (ACE_READ_ACL | ACE_READ_ATTRIBUTES)) != @@ -1044,7 +1045,7 @@ static int acevals_to_aent(acevals_t *vals, aclent_t *dest, ace_list_t *list, - uid_t owner, gid_t group, int isdir) + uid_t owner, gid_t group, boolean_t isdir) { int error; uint32_t flips = ACE_POSIX_SUPPORTED_BITS; @@ -1084,7 +1085,7 @@ static int ace_list_to_aent(ace_list_t *list, aclent_t **aclentp, int *aclcnt, - uid_t owner, gid_t group, int isdir) + uid_t owner, gid_t group, boolean_t isdir) { int error = 0; aclent_t *aent, *result = NULL; @@ -1264,7 +1265,7 @@ static int ln_ace_to_aent(ace_t *ace, int n, uid_t owner, gid_t group, aclent_t **aclentp, int *aclcnt, aclent_t **dfaclentp, int *dfaclcnt, - int isdir) + boolean_t isdir) { int error = 0; ace_t *acep; @@ -1459,7 +1460,7 @@ } static int -convert_ace_to_aent(ace_t *acebufp, int acecnt, int isdir, +convert_ace_to_aent(ace_t *acebufp, int acecnt, boolean_t isdir, uid_t owner, gid_t group, aclent_t **retaclentp, int *retaclcnt) { int error = 0; @@ -1501,7 +1502,7 @@ int -acl_translate(acl_t *aclp, int target_flavor, int isdir, uid_t owner, +acl_translate(acl_t *aclp, int target_flavor, boolean_t isdir, uid_t owner, gid_t group) { int aclcnt; @@ -1573,101 +1574,105 @@ } void -acl_trivial_access_masks(mode_t mode, uint32_t *allow0, uint32_t *deny1, - uint32_t *deny2, uint32_t *owner, uint32_t *group, uint32_t *everyone) +acl_trivial_access_masks(mode_t mode, boolean_t isdir, trivial_acl_t *masks) { - *deny1 = *deny2 = *allow0 = *group = 0; + uint32_t read_mask = ACE_READ_DATA; + uint32_t write_mask = ACE_WRITE_DATA|ACE_APPEND_DATA; + uint32_t execute_mask = ACE_EXECUTE; + (void) isdir; /* will need this later */ + + masks->deny1 = 0; if (!(mode & S_IRUSR) && (mode & (S_IRGRP|S_IROTH))) - *deny1 |= ACE_READ_DATA; + masks->deny1 |= read_mask; if (!(mode & S_IWUSR) && (mode & (S_IWGRP|S_IWOTH))) - *deny1 |= ACE_WRITE_DATA|ACE_APPEND_DATA; + masks->deny1 |= write_mask; if (!(mode & S_IXUSR) && (mode & (S_IXGRP|S_IXOTH))) - *deny1 |= ACE_EXECUTE; + masks->deny1 |= execute_mask; + masks->deny2 = 0; if (!(mode & S_IRGRP) && (mode & S_IROTH)) - *deny2 = ACE_READ_DATA; + masks->deny2 |= read_mask; if (!(mode & S_IWGRP) && (mode & S_IWOTH)) - *deny2 |= ACE_WRITE_DATA|ACE_APPEND_DATA; + masks->deny2 |= write_mask; if (!(mode & S_IXGRP) && (mode & S_IXOTH)) - *deny2 |= ACE_EXECUTE; + masks->deny2 |= execute_mask; + masks->allow0 = 0; if ((mode & S_IRUSR) && (!(mode & S_IRGRP) && (mode & S_IROTH))) - *allow0 |= ACE_READ_DATA; + masks->allow0 |= read_mask; if ((mode & S_IWUSR) && (!(mode & S_IWGRP) && (mode & S_IWOTH))) - *allow0 |= ACE_WRITE_DATA|ACE_APPEND_DATA; + masks->allow0 |= write_mask; if ((mode & S_IXUSR) && (!(mode & S_IXGRP) && (mode & S_IXOTH))) - *allow0 |= ACE_EXECUTE; + masks->allow0 |= execute_mask; - *owner = ACE_WRITE_ATTRIBUTES|ACE_WRITE_OWNER|ACE_WRITE_ACL| + masks->owner = ACE_WRITE_ATTRIBUTES|ACE_WRITE_OWNER|ACE_WRITE_ACL| ACE_WRITE_NAMED_ATTRS|ACE_READ_ACL|ACE_READ_ATTRIBUTES| ACE_READ_NAMED_ATTRS|ACE_SYNCHRONIZE; if (mode & S_IRUSR) - *owner |= ACE_READ_DATA; + masks->owner |= read_mask; if (mode & S_IWUSR) - *owner |= ACE_WRITE_DATA|ACE_APPEND_DATA; + masks->owner |= write_mask; if (mode & S_IXUSR) - *owner |= ACE_EXECUTE; + masks->owner |= execute_mask; - *group = ACE_READ_ACL|ACE_READ_ATTRIBUTES| ACE_READ_NAMED_ATTRS| + masks->group = ACE_READ_ACL|ACE_READ_ATTRIBUTES|ACE_READ_NAMED_ATTRS| ACE_SYNCHRONIZE; if (mode & S_IRGRP) - *group |= ACE_READ_DATA; + masks->group |= read_mask; if (mode & S_IWGRP) - *group |= ACE_WRITE_DATA|ACE_APPEND_DATA; + masks->group |= write_mask; if (mode & S_IXGRP) - *group |= ACE_EXECUTE; + masks->group |= execute_mask; - *everyone = ACE_READ_ACL|ACE_READ_ATTRIBUTES| ACE_READ_NAMED_ATTRS| + masks->everyone = ACE_READ_ACL|ACE_READ_ATTRIBUTES|ACE_READ_NAMED_ATTRS| ACE_SYNCHRONIZE; if (mode & S_IROTH) - *everyone |= ACE_READ_DATA; + masks->everyone |= read_mask; if (mode & S_IWOTH) - *everyone |= ACE_WRITE_DATA|ACE_APPEND_DATA; + masks->everyone |= write_mask; if (mode & S_IXOTH) - *everyone |= ACE_EXECUTE; + masks->everyone |= execute_mask; } int -acl_trivial_create(mode_t mode, ace_t **acl, int *count) +acl_trivial_create(mode_t mode, boolean_t isdir, ace_t **acl, int *count) { - uint32_t deny1, deny2; - uint32_t allow0; - uint32_t owner, group, everyone; - int index = 0; + int index = 0; int error; + trivial_acl_t masks; *count = 3; - acl_trivial_access_masks(mode, &allow0, &deny1, &deny2, &owner, &group, - &everyone); + acl_trivial_access_masks(mode, isdir, &masks); - if (allow0) + if (masks.allow0) (*count)++; - if (deny1) + if (masks.deny1) (*count)++; - if (deny2) + if (masks.deny2) (*count)++; if ((error = cacl_malloc((void **)acl, *count * sizeof (ace_t))) != 0) return (error); - if (allow0) { - SET_ACE(acl, index, -1, allow0, ACE_ACCESS_ALLOWED_ACE_TYPE, - ACE_OWNER); + if (masks.allow0) { + SET_ACE(acl, index, -1, masks.allow0, + ACE_ACCESS_ALLOWED_ACE_TYPE, ACE_OWNER); } - if (deny1) { - SET_ACE(acl, index, -1, deny1, ACE_ACCESS_DENIED_ACE_TYPE, - ACE_OWNER); + if (masks.deny1) { + SET_ACE(acl, index, -1, masks.deny1, + ACE_ACCESS_DENIED_ACE_TYPE, ACE_OWNER); } - if (deny2) { - SET_ACE(acl, index, -1, deny2, ACE_ACCESS_DENIED_ACE_TYPE, - ACE_GROUP|ACE_IDENTIFIER_GROUP); + if (masks.deny2) { + SET_ACE(acl, index, -1, masks.deny2, + ACE_ACCESS_DENIED_ACE_TYPE, ACE_GROUP|ACE_IDENTIFIER_GROUP); } - SET_ACE(acl, index, -1, owner, ACE_ACCESS_ALLOWED_ACE_TYPE, ACE_OWNER); - SET_ACE(acl, index, -1, group, ACE_ACCESS_ALLOWED_ACE_TYPE, + SET_ACE(acl, index, -1, masks.owner, ACE_ACCESS_ALLOWED_ACE_TYPE, + ACE_OWNER); + SET_ACE(acl, index, -1, masks.group, ACE_ACCESS_ALLOWED_ACE_TYPE, ACE_IDENTIFIER_GROUP|ACE_GROUP); - SET_ACE(acl, index, -1, everyone, ACE_ACCESS_ALLOWED_ACE_TYPE, + SET_ACE(acl, index, -1, masks.everyone, ACE_ACCESS_ALLOWED_ACE_TYPE, ACE_EVERYONE); return (0); Index: sys/cddl/contrib/opensolaris/common/acl/acl_common.h =================================================================== --- sys/cddl/contrib/opensolaris/common/acl/acl_common.h (revision 223325) +++ sys/cddl/contrib/opensolaris/common/acl/acl_common.h (working copy) @@ -20,6 +20,7 @@ */ /* * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved. + * Copyright 2011 Nexenta Systems, Inc. All rights reserved. */ #ifndef _ACL_COMMON_H @@ -33,7 +34,14 @@ extern "C" { #endif -extern ace_t trivial_acl[6]; +typedef struct trivial_acl { + uint32_t allow0; /* allow mask for bits only in owner */ + uint32_t deny1; /* deny mask for bits not in owner */ + uint32_t deny2; /* deny mask for bits not in group */ + uint32_t owner; /* allow mask matching mode */ + uint32_t group; /* allow mask matching mode */ + uint32_t everyone; /* allow mask matching mode */ +} trivial_acl_t; extern int acltrivial(const char *); extern void adjust_ace_pair(ace_t *pair, mode_t mode); @@ -45,14 +53,14 @@ #if !defined(_KERNEL) extern acl_t *acl_alloc(acl_type_t); extern void acl_free(acl_t *aclp); -extern int acl_translate(acl_t *aclp, int target_flavor, - int isdir, uid_t owner, gid_t group); +extern int acl_translate(acl_t *aclp, int target_flavor, boolean_t isdir, + uid_t owner, gid_t group); #endif /* !_KERNEL */ void ksort(caddr_t v, int n, int s, int (*f)()); int cmp2acls(void *a, void *b); -int acl_trivial_create(mode_t mode, ace_t **acl, int *count); -void acl_trivial_access_masks(mode_t mode, uint32_t *allow0, uint32_t *deny1, - uint32_t *deny2, uint32_t *owner, uint32_t *group, uint32_t *everyone); +int acl_trivial_create(mode_t mode, boolean_t isdir, ace_t **acl, int *count); +void acl_trivial_access_masks(mode_t mode, boolean_t isdir, + trivial_acl_t *masks); #ifdef __cplusplus } Index: sys/cddl/contrib/opensolaris/common/zfs/zfs_prop.c =================================================================== --- sys/cddl/contrib/opensolaris/common/zfs/zfs_prop.c (revision 223325) +++ sys/cddl/contrib/opensolaris/common/zfs/zfs_prop.c (working copy) @@ -104,6 +104,13 @@ { NULL } }; + static zprop_index_t acl_mode_table[] = { + { "discard", ZFS_ACL_DISCARD }, + { "groupmask", ZFS_ACL_GROUPMASK }, + { "passthrough", ZFS_ACL_PASSTHROUGH }, + { NULL } + }; + static zprop_index_t acl_inherit_table[] = { { "discard", ZFS_ACL_DISCARD }, { "noallow", ZFS_ACL_NOALLOW }, @@ -207,6 +214,9 @@ zprop_register_index(ZFS_PROP_SNAPDIR, "snapdir", ZFS_SNAPDIR_HIDDEN, PROP_INHERIT, ZFS_TYPE_FILESYSTEM, "hidden | visible", "SNAPDIR", snapdir_table); + zprop_register_index(ZFS_PROP_ACLMODE, "aclmode", ZFS_ACL_DISCARD, + PROP_INHERIT, ZFS_TYPE_FILESYSTEM, + "discard | groupmask | passthrough", "ACLMODE", acl_mode_table); zprop_register_index(ZFS_PROP_ACLINHERIT, "aclinherit", ZFS_ACL_RESTRICTED, PROP_INHERIT, ZFS_TYPE_FILESYSTEM, "discard | noallow | restricted | passthrough | passthrough-x", @@ -370,13 +380,6 @@ zprop_register_hidden(ZFS_PROP_OBJSETID, "objsetid", PROP_TYPE_NUMBER, PROP_READONLY, ZFS_TYPE_DATASET, "OBJSETID"); - /* - * Property to be removed once libbe is integrated - */ - zprop_register_hidden(ZFS_PROP_PRIVATE, "priv_prop", - PROP_TYPE_NUMBER, PROP_READONLY, ZFS_TYPE_FILESYSTEM, - "PRIV_PROP"); - /* oddball properties */ zprop_register_impl(ZFS_PROP_CREATION, "creation", PROP_TYPE_NUMBER, 0, NULL, PROP_READONLY, ZFS_TYPE_DATASET, Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c =================================================================== --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c (revision 223325) +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c (working copy) @@ -3223,7 +3223,8 @@ uint64_t acl_obj; new_mode = (pmode & S_IFMT) | (vap->va_mode & ~S_IFMT); - zfs_acl_chmod_setattr(zp, &aclp, new_mode); + if (err = zfs_acl_chmod_setattr(zp, &aclp, new_mode)) + goto out; mutex_enter(&zp->z_lock); if (!zp->z_is_sa && ((acl_obj = zfs_external_acl(zp)) != 0)) { Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vfsops.c =================================================================== --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vfsops.c (revision 223325) +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vfsops.c (working copy) @@ -364,6 +364,14 @@ } static void +acl_mode_changed_cb(void *arg, uint64_t newval) +{ + zfsvfs_t *zfsvfs = arg; + + zfsvfs->z_acl_mode = newval; +} + +static void acl_inherit_changed_cb(void *arg, uint64_t newval) { zfsvfs_t *zfsvfs = arg; @@ -488,6 +496,8 @@ error = error ? error : dsl_prop_register(ds, "snapdir", snapdir_changed_cb, zfsvfs); error = error ? error : dsl_prop_register(ds, + "aclmode", acl_mode_changed_cb, zfsvfs); + error = error ? error : dsl_prop_register(ds, "aclinherit", acl_inherit_changed_cb, zfsvfs); error = error ? error : dsl_prop_register(ds, "vscan", vscan_changed_cb, zfsvfs); @@ -525,6 +535,7 @@ (void) dsl_prop_unregister(ds, "setuid", setuid_changed_cb, zfsvfs); (void) dsl_prop_unregister(ds, "exec", exec_changed_cb, zfsvfs); (void) dsl_prop_unregister(ds, "snapdir", snapdir_changed_cb, zfsvfs); + (void) dsl_prop_unregister(ds, "aclmode", acl_mode_changed_cb, zfsvfs); (void) dsl_prop_unregister(ds, "aclinherit", acl_inherit_changed_cb, zfsvfs); (void) dsl_prop_unregister(ds, "vscan", vscan_changed_cb, zfsvfs); @@ -1202,6 +1213,9 @@ VERIFY(dsl_prop_unregister(ds, "snapdir", snapdir_changed_cb, zfsvfs) == 0); + VERIFY(dsl_prop_unregister(ds, "aclmode", acl_mode_changed_cb, + zfsvfs) == 0); + VERIFY(dsl_prop_unregister(ds, "aclinherit", acl_inherit_changed_cb, zfsvfs) == 0); Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_acl.c =================================================================== --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_acl.c (revision 223325) +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_acl.c (working copy) @@ -20,6 +20,7 @@ */ /* * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved. + * Copyright 2011 Nexenta Systems, Inc. All rights reserved. */ #include @@ -1327,76 +1328,9 @@ return (sa_bulk_update(zp->z_sa_hdl, bulk, count, tx)); } -/* - * Update access mask for prepended ACE - * - * This applies the "groupmask" value for aclmode property. - */ static void -zfs_acl_prepend_fixup(zfs_acl_t *aclp, void *acep, void *origacep, - mode_t mode, uint64_t owner) +zfs_acl_chmod(vtype_t vtype, uint64_t mode, boolean_t trim, zfs_acl_t *aclp) { - int rmask, wmask, xmask; - int user_ace; - uint16_t aceflags; - uint32_t origmask, acepmask; - uint64_t fuid; - - aceflags = aclp->z_ops.ace_flags_get(acep); - fuid = aclp->z_ops.ace_who_get(acep); - origmask = aclp->z_ops.ace_mask_get(origacep); - acepmask = aclp->z_ops.ace_mask_get(acep); - - user_ace = (!(aceflags & - (ACE_OWNER|ACE_GROUP|ACE_IDENTIFIER_GROUP))); - - if (user_ace && (fuid == owner)) { - rmask = S_IRUSR; - wmask = S_IWUSR; - xmask = S_IXUSR; - } else { - rmask = S_IRGRP; - wmask = S_IWGRP; - xmask = S_IXGRP; - } - - if (origmask & ACE_READ_DATA) { - if (mode & rmask) { - acepmask &= ~ACE_READ_DATA; - } else { - acepmask |= ACE_READ_DATA; - } - } - - if (origmask & ACE_WRITE_DATA) { - if (mode & wmask) { - acepmask &= ~ACE_WRITE_DATA; - } else { - acepmask |= ACE_WRITE_DATA; - } - } - - if (origmask & ACE_APPEND_DATA) { - if (mode & wmask) { - acepmask &= ~ACE_APPEND_DATA; - } else { - acepmask |= ACE_APPEND_DATA; - } - } - - if (origmask & ACE_EXECUTE) { - if (mode & xmask) { - acepmask &= ~ACE_EXECUTE; - } else { - acepmask |= ACE_EXECUTE; - } - } - aclp->z_ops.ace_mask_set(acep, acepmask); -} - -static void -zfs_acl_chmod(zfsvfs_t *zfsvfs, uint64_t mode, zfs_acl_t *aclp) -{ void *acep = NULL; uint64_t who; int new_count, new_bytes; @@ -1407,30 +1341,31 @@ zfs_acl_node_t *newnode; size_t abstract_size = aclp->z_ops.ace_abstract_size(); void *zacep; - uint32_t owner, group, everyone; - uint32_t deny1, deny2, allow0; + boolean_t isdir; + trivial_acl_t masks; new_count = new_bytes = 0; - acl_trivial_access_masks((mode_t)mode, &allow0, &deny1, &deny2, - &owner, &group, &everyone); + isdir = (vtype == VDIR); + acl_trivial_access_masks((mode_t)mode, isdir, &masks); + newnode = zfs_acl_node_alloc((abstract_size * 6) + aclp->z_acl_bytes); zacep = newnode->z_acldata; - if (allow0) { - zfs_set_ace(aclp, zacep, allow0, ALLOW, -1, ACE_OWNER); + if (masks.allow0) { + zfs_set_ace(aclp, zacep, masks.allow0, ALLOW, -1, ACE_OWNER); zacep = (void *)((uintptr_t)zacep + abstract_size); new_count++; new_bytes += abstract_size; - } if (deny1) { - zfs_set_ace(aclp, zacep, deny1, DENY, -1, ACE_OWNER); + } if (masks.deny1) { + zfs_set_ace(aclp, zacep, masks.deny1, DENY, -1, ACE_OWNER); zacep = (void *)((uintptr_t)zacep + abstract_size); new_count++; new_bytes += abstract_size; } - if (deny2) { - zfs_set_ace(aclp, zacep, deny2, DENY, -1, OWNING_GROUP); + if (masks.deny2) { + zfs_set_ace(aclp, zacep, masks.deny2, DENY, -1, OWNING_GROUP); zacep = (void *)((uintptr_t)zacep + abstract_size); new_count++; new_bytes += abstract_size; @@ -1449,10 +1384,17 @@ continue; } + /* + * If this ACL has any inheritable ACEs, mark that in + * the hints (which are later masked into the pflags) + * so create knows to do inheritance. + */ + if (isdir && (inherit_flags & + (ACE_FILE_INHERIT_ACE|ACE_DIRECTORY_INHERIT_ACE))) + aclp->z_hints |= ZFS_INHERIT_ACE; + if ((type != ALLOW && type != DENY) || (inherit_flags & ACE_INHERIT_ONLY_ACE)) { - if (inherit_flags) - aclp->z_hints |= ZFS_INHERIT_ACE; switch (type) { case ACE_ACCESS_ALLOWED_OBJECT_ACE_TYPE: case ACE_ACCESS_DENIED_OBJECT_ACE_TYPE: @@ -1465,20 +1407,13 @@ /* * Limit permissions to be no greater than - * group permissions + * group permissions. + * The "aclinherit" and "aclmode" properties + * affect policy for create and chmod(2), + * respectively. */ - if (type == ALLOW && zfsvfs->z_acl_inherit == ZFS_ACL_RESTRICTED) { - if (!(mode & S_IRGRP)) - access_mask &= ~ACE_READ_DATA; - if (!(mode & S_IWGRP)) - access_mask &= - ~(ACE_WRITE_DATA|ACE_APPEND_DATA); - if (!(mode & S_IXGRP)) - access_mask &= ~ACE_EXECUTE; - access_mask &= - ~(ACE_WRITE_OWNER|ACE_WRITE_ACL| - ACE_WRITE_ATTRIBUTES|ACE_WRITE_NAMED_ATTRS); - } + if ((type == ALLOW) && trim) + access_mask &= masks.group; } zfs_set_ace(aclp, zacep, access_mask, type, who, iflags); ace_size = aclp->z_ops.ace_size(acep); @@ -1486,11 +1421,11 @@ new_count++; new_bytes += ace_size; } - zfs_set_ace(aclp, zacep, owner, 0, -1, ACE_OWNER); + zfs_set_ace(aclp, zacep, masks.owner, 0, -1, ACE_OWNER); zacep = (void *)((uintptr_t)zacep + abstract_size); - zfs_set_ace(aclp, zacep, group, 0, -1, OWNING_GROUP); + zfs_set_ace(aclp, zacep, masks.group, 0, -1, OWNING_GROUP); zacep = (void *)((uintptr_t)zacep + abstract_size); - zfs_set_ace(aclp, zacep, everyone, 0, -1, ACE_EVERYONE); + zfs_set_ace(aclp, zacep, masks.everyone, 0, -1, ACE_EVERYONE); new_count += 3; new_bytes += abstract_size * 3; @@ -1502,17 +1437,27 @@ list_insert_tail(&aclp->z_acl, newnode); } -void +int zfs_acl_chmod_setattr(znode_t *zp, zfs_acl_t **aclp, uint64_t mode) { + int error = 0; + mutex_enter(&zp->z_acl_lock); mutex_enter(&zp->z_lock); - *aclp = zfs_acl_alloc(zfs_acl_version_zp(zp)); - (*aclp)->z_hints = zp->z_pflags & V4_ACL_WIDE_FLAGS; - zfs_acl_chmod(zp->z_zfsvfs, mode, *aclp); + if (zp->z_zfsvfs->z_acl_mode == ZFS_ACL_DISCARD) + *aclp = zfs_acl_alloc(zfs_acl_version_zp(zp)); + else + error = zfs_acl_node_read(zp, B_TRUE, aclp, B_TRUE); + + if (error == 0) { + (*aclp)->z_hints = zp->z_pflags & V4_ACL_WIDE_FLAGS; + zfs_acl_chmod(ZTOV(zp)->v_type, mode, + (zp->z_zfsvfs->z_acl_mode == ZFS_ACL_GROUPMASK), *aclp); + } mutex_exit(&zp->z_lock); mutex_exit(&zp->z_acl_lock); - ASSERT(*aclp); + + return (error); } /* @@ -1764,8 +1709,8 @@ if (acl_ids->z_aclp == NULL) { mutex_enter(&dzp->z_acl_lock); mutex_enter(&dzp->z_lock); - if (!(flag & IS_ROOT_NODE) && (ZTOV(dzp)->v_type == VDIR && - (dzp->z_pflags & ZFS_INHERIT_ACE)) && + if (!(flag & IS_ROOT_NODE) && + (dzp->z_pflags & ZFS_INHERIT_ACE) && !(dzp->z_pflags & ZFS_XATTR)) { VERIFY(0 == zfs_acl_node_read(dzp, B_TRUE, &paclp, B_FALSE)); @@ -1782,7 +1727,9 @@ if (need_chmod) { acl_ids->z_aclp->z_hints |= (vap->va_type == VDIR) ? ZFS_ACL_AUTO_INHERIT : 0; - zfs_acl_chmod(zfsvfs, acl_ids->z_mode, acl_ids->z_aclp); + zfs_acl_chmod(vap->va_type, acl_ids->z_mode, + (zfsvfs->z_acl_inherit == ZFS_ACL_RESTRICTED), + acl_ids->z_aclp); } } Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/zfs_vfsops.h =================================================================== --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/zfs_vfsops.h (revision 223325) +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/zfs_vfsops.h (working copy) @@ -55,6 +55,7 @@ boolean_t z_fuid_dirty; /* need to sync fuid table ? */ struct zfs_fuid_info *z_fuid_replay; /* fuid info for replay */ zilog_t *z_log; /* intent log pointer */ + uint_t z_acl_mode; /* acl chmod/mode behavior */ uint_t z_acl_inherit; /* acl inheritance behavior */ zfs_case_t z_case; /* case-sense */ boolean_t z_utf8; /* utf8-only */ Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/zfs_acl.h =================================================================== --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/zfs_acl.h (revision 223325) +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/zfs_acl.h (working copy) @@ -217,7 +217,7 @@ extern int zfs_zaccess_rwx(struct znode *, mode_t, int, cred_t *); extern int zfs_zaccess_unix(struct znode *, mode_t, cred_t *); extern int zfs_acl_access(struct znode *, int, cred_t *); -void zfs_acl_chmod_setattr(struct znode *, zfs_acl_t **, uint64_t); +int zfs_acl_chmod_setattr(struct znode *, zfs_acl_t **, uint64_t); int zfs_zaccess_delete(struct znode *, struct znode *, cred_t *); int zfs_zaccess_rename(struct znode *, struct znode *, struct znode *, struct znode *, cred_t *cr); Index: sys/cddl/contrib/opensolaris/uts/common/sys/fs/zfs.h =================================================================== --- sys/cddl/contrib/opensolaris/uts/common/sys/fs/zfs.h (revision 223325) +++ sys/cddl/contrib/opensolaris/uts/common/sys/fs/zfs.h (working copy) @@ -89,7 +89,7 @@ ZFS_PROP_READONLY, ZFS_PROP_ZONED, ZFS_PROP_SNAPDIR, - ZFS_PROP_PRIVATE, /* not exposed to user, temporary */ + ZFS_PROP_ACLMODE, ZFS_PROP_ACLINHERIT, ZFS_PROP_CREATETXG, /* not exposed to the user */ ZFS_PROP_NAME, /* not exposed to the user */ --------------010907030303010503000607-- From owner-zfs-devel@FreeBSD.ORG Mon Jun 20 09:16:58 2011 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 014BA106566C; Mon, 20 Jun 2011 09:16:58 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from mail.vx.sk (mail.vx.sk [IPv6:2a01:4f8:100:1043::3]) by mx1.freebsd.org (Postfix) with ESMTP id 541748FC15; Mon, 20 Jun 2011 09:16:57 +0000 (UTC) Received: from core.vx.sk (localhost [127.0.0.1]) by mail.vx.sk (Postfix) with ESMTP id AED68179252; Mon, 20 Jun 2011 11:16:56 +0200 (CEST) X-Virus-Scanned: amavisd-new at mail.vx.sk Received: from mail.vx.sk ([127.0.0.1]) by core.vx.sk (mail.vx.sk [127.0.0.1]) (amavisd-new, port 10024) with LMTP id UltTu+JW96Z7; Mon, 20 Jun 2011 11:16:54 +0200 (CEST) Received: from [10.0.3.3] (188-167-66-148.dynamic.chello.sk [188.167.66.148]) by mail.vx.sk (Postfix) with ESMTPSA id BE1F4179215; Mon, 20 Jun 2011 11:16:50 +0200 (CEST) Message-ID: <4DFF1002.3070402@FreeBSD.org> Date: Mon, 20 Jun 2011 11:16:50 +0200 From: Martin Matuska User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.17) Gecko/20110516 Thunderbird/3.1.10 MIME-Version: 1.0 To: zfs-devel@FreeBSD.org Content-Type: multipart/mixed; boundary="------------070809000803040501070106" Cc: Pawel Jakub Dawidek Subject: [CFR][ZFS] merge Illumos rev. 13379 (writing to imbalanced vdevs) X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 20 Jun 2011 09:16:58 -0000 This is a multi-part message in MIME format. --------------070809000803040501070106 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit zfs tries to allocate blocks evenly across all devices. This means when devices are imbalanced zfs will lots of CPU searching for space on devices which tend to be pretty full. https://www.illumos.org/issues/1051 https://www.illumos.org/projects/illumos-gate/repository/revisions/13379 If there are no objections, I will commit this to -HEAD -- Martin Matuska FreeBSD committer http://blog.vx.sk --------------070809000803040501070106 Content-Type: text/x-patch; name="13379.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="13379.patch" Index: cddl/contrib/opensolaris/cmd/ztest/ztest.c =================================================================== --- cddl/contrib/opensolaris/cmd/ztest/ztest.c (revision 223325) +++ cddl/contrib/opensolaris/cmd/ztest/ztest.c (working copy) @@ -20,6 +20,7 @@ */ /* * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 2011 by Delphix. All rights reserved. */ /* @@ -5137,6 +5138,7 @@ */ kernel_init(FREAD | FWRITE); VERIFY(spa_open(zs->zs_pool, &spa, FTAG) == 0); + spa->spa_debug = B_TRUE; zs->zs_spa = spa; spa->spa_dedup_ditto = 2 * ZIO_DEDUPDITTO_MIN; Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa_misc.c =================================================================== --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa_misc.c (revision 223325) +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa_misc.c (working copy) @@ -20,6 +20,7 @@ */ /* * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 2011 by Delphix. All rights reserved. */ #include @@ -1674,3 +1675,9 @@ return (0); } + +boolean_t +spa_debug_enabled(spa_t *spa) +{ + return (spa->spa_debug); +} Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/metaslab.c =================================================================== --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/metaslab.c (revision 223325) +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/metaslab.c (working copy) @@ -20,6 +20,7 @@ */ /* * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 2011 by Delphix. All rights reserved. */ #include @@ -30,10 +31,29 @@ #include #include +/* + * Allow allocations to switch to gang blocks quickly. We do this to + * avoid having to load lots of space_maps in a given txg. There are, + * however, some cases where we want to avoid "fast" ganging and instead + * we want to do an exhaustive search of all metaslabs on this device. + * Currently we don't allow any gang or dump device related allocations + * to "fast" gang. + */ +#define CAN_FASTGANG(flags) \ + (!((flags) & (METASLAB_GANG_CHILD | METASLAB_GANG_HEADER | \ + METASLAB_GANG_AVOID))) + uint64_t metaslab_aliquot = 512ULL << 10; uint64_t metaslab_gang_bang = SPA_MAXBLOCKSIZE + 1; /* force gang blocks */ /* + * This value defines the number of allowed allocation failures per vdev. + * If a device reaches this threshold in a given txg then we consider skipping + * allocations on that device. + */ +int zfs_mg_alloc_failures; + +/* * Metaslab debugging: when set, keeps all space maps in core to verify frees. */ static int metaslab_debug = 0; @@ -671,7 +691,7 @@ metaslab_ndf_fragmented }; -space_map_ops_t *zfs_metaslab_ops = &metaslab_ndf_ops; +space_map_ops_t *zfs_metaslab_ops = &metaslab_df_ops; /* * ========================================================================== @@ -844,7 +864,7 @@ } static int -metaslab_activate(metaslab_t *msp, uint64_t activation_weight, uint64_t size) +metaslab_activate(metaslab_t *msp, uint64_t activation_weight) { metaslab_group_t *mg = msp->ms_group; space_map_t *sm = &msp->ms_map; @@ -877,13 +897,6 @@ mutex_exit(&mg->mg_lock); } - /* - * If we were able to load the map then make sure - * that this map is still able to satisfy our request. - */ - if (msp->ms_weight < size) - return (ENOSPC); - metaslab_group_sort(msp->ms_group, msp, msp->ms_weight | activation_weight); } @@ -1099,6 +1112,7 @@ metaslab_sync_reassess(metaslab_group_t *mg) { vdev_t *vd = mg->mg_vd; + int64_t failures = mg->mg_alloc_failures; /* * Re-evaluate all metaslabs which have lower offsets than the @@ -1115,6 +1129,8 @@ mutex_exit(&msp->ms_lock); } + atomic_add_64(&mg->mg_alloc_failures, -failures); + /* * Prefetch the next potential metaslabs */ @@ -1139,9 +1155,10 @@ } static uint64_t -metaslab_group_alloc(metaslab_group_t *mg, uint64_t size, uint64_t txg, - uint64_t min_distance, dva_t *dva, int d) +metaslab_group_alloc(metaslab_group_t *mg, uint64_t psize, uint64_t asize, + uint64_t txg, uint64_t min_distance, dva_t *dva, int d, int flags) { + spa_t *spa = mg->mg_vd->vdev_spa; metaslab_t *msp = NULL; uint64_t offset = -1ULL; avl_tree_t *t = &mg->mg_metaslab_tree; @@ -1162,11 +1179,17 @@ mutex_enter(&mg->mg_lock); for (msp = avl_first(t); msp; msp = AVL_NEXT(t, msp)) { - if (msp->ms_weight < size) { + if (msp->ms_weight < asize) { + spa_dbgmsg(spa, "%s: failed to meet weight " + "requirement: vdev %llu, txg %llu, mg %p, " + "msp %p, psize %llu, asize %llu, " + "failures %llu, weight %llu", + spa_name(spa), mg->mg_vd->vdev_id, txg, + mg, msp, psize, asize, + mg->mg_alloc_failures, msp->ms_weight); mutex_exit(&mg->mg_lock); return (-1ULL); } - was_active = msp->ms_weight & METASLAB_ACTIVE_MASK; if (activation_weight == METASLAB_WEIGHT_PRIMARY) break; @@ -1185,6 +1208,25 @@ if (msp == NULL) return (-1ULL); + /* + * If we've already reached the allowable number of failed + * allocation attempts on this metaslab group then we + * consider skipping it. We skip it only if we're allowed + * to "fast" gang, the physical size is larger than + * a gang block, and we're attempting to allocate from + * the primary metaslab. + */ + if (mg->mg_alloc_failures > zfs_mg_alloc_failures && + CAN_FASTGANG(flags) && psize > SPA_GANGBLOCKSIZE && + activation_weight == METASLAB_WEIGHT_PRIMARY) { + spa_dbgmsg(spa, "%s: skipping metaslab group: " + "vdev %llu, txg %llu, mg %p, psize %llu, " + "asize %llu, failures %llu", spa_name(spa), + mg->mg_vd->vdev_id, txg, mg, psize, asize, + mg->mg_alloc_failures); + return (-1ULL); + } + mutex_enter(&msp->ms_lock); /* @@ -1193,7 +1235,7 @@ * another thread may have changed the weight while we * were blocked on the metaslab lock. */ - if (msp->ms_weight < size || (was_active && + if (msp->ms_weight < asize || (was_active && !(msp->ms_weight & METASLAB_ACTIVE_MASK) && activation_weight == METASLAB_WEIGHT_PRIMARY)) { mutex_exit(&msp->ms_lock); @@ -1208,14 +1250,16 @@ continue; } - if (metaslab_activate(msp, activation_weight, size) != 0) { + if (metaslab_activate(msp, activation_weight) != 0) { mutex_exit(&msp->ms_lock); continue; } - if ((offset = space_map_alloc(&msp->ms_map, size)) != -1ULL) + if ((offset = space_map_alloc(&msp->ms_map, asize)) != -1ULL) break; + atomic_inc_64(&mg->mg_alloc_failures); + metaslab_passivate(msp, space_map_maxsize(&msp->ms_map)); mutex_exit(&msp->ms_lock); @@ -1224,7 +1268,7 @@ if (msp->ms_allocmap[txg & TXG_MASK].sm_space == 0) vdev_dirty(mg->mg_vd, VDD_METASLAB, msp, txg); - space_map_add(&msp->ms_allocmap[txg & TXG_MASK], offset, size); + space_map_add(&msp->ms_allocmap[txg & TXG_MASK], offset, asize); mutex_exit(&msp->ms_lock); @@ -1351,7 +1395,8 @@ asize = vdev_psize_to_asize(vd, psize); ASSERT(P2PHASE(asize, 1ULL << vd->vdev_ashift) == 0); - offset = metaslab_group_alloc(mg, asize, txg, distance, dva, d); + offset = metaslab_group_alloc(mg, psize, asize, txg, distance, + dva, d, flags); if (offset != -1ULL) { /* * If we've just selected this metaslab group, @@ -1363,18 +1408,24 @@ vdev_stat_t *vs = &vd->vdev_stat; int64_t vu, cu; - /* - * Determine percent used in units of 0..1024. - * (This is just to avoid floating point.) - */ - vu = (vs->vs_alloc << 10) / (vs->vs_space + 1); - cu = (mc->mc_alloc << 10) / (mc->mc_space + 1); + vu = (vs->vs_alloc * 100) / (vs->vs_space + 1); + cu = (mc->mc_alloc * 100) / (mc->mc_space + 1); /* - * Bias by at most +/- 25% of the aliquot. + * Calculate how much more or less we should + * try to allocate from this device during + * this iteration around the rotor. + * For example, if a device is 80% full + * and the pool is 20% full then we should + * reduce allocations by 60% on this device. + * + * mg_bias = (20 - 80) * 512K / 100 = -307K + * + * This reduces allocations by 307K for this + * iteration. */ mg->mg_bias = ((cu - vu) * - (int64_t)mg->mg_aliquot) / (1024 * 4); + (int64_t)mg->mg_aliquot) / 100; } if (atomic_add_64_nv(&mc->mc_aliquot, asize) >= @@ -1488,7 +1539,7 @@ mutex_enter(&msp->ms_lock); if ((txg != 0 && spa_writeable(spa)) || !msp->ms_map.sm_loaded) - error = metaslab_activate(msp, METASLAB_WEIGHT_SECONDARY, 0); + error = metaslab_activate(msp, METASLAB_WEIGHT_SECONDARY); if (error == 0 && !space_map_contains(&msp->ms_map, offset, size)) error = ENOENT; Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/metaslab.h =================================================================== --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/metaslab.h (revision 223325) +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/metaslab.h (working copy) @@ -20,6 +20,7 @@ */ /* * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 2011 by Delphix. All rights reserved. */ #ifndef _SYS_METASLAB_H @@ -47,6 +48,8 @@ #define METASLAB_HINTBP_FAVOR 0x0 #define METASLAB_HINTBP_AVOID 0x1 #define METASLAB_GANG_HEADER 0x2 +#define METASLAB_GANG_CHILD 0x4 +#define METASLAB_GANG_AVOID 0x8 extern int metaslab_alloc(spa_t *spa, metaslab_class_t *mc, uint64_t psize, blkptr_t *bp, int ncopies, uint64_t txg, blkptr_t *hintbp, int flags); Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/spa_impl.h =================================================================== --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/spa_impl.h (revision 223325) +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/spa_impl.h (working copy) @@ -20,6 +20,7 @@ */ /* * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 2011 by Delphix. All rights reserved. */ #ifndef _SYS_SPA_IMPL_H @@ -196,6 +197,7 @@ kcondvar_t spa_suspend_cv; /* notification of resume */ uint8_t spa_suspended; /* pool is suspended */ uint8_t spa_claiming; /* pool is doing zil_claim() */ + boolean_t spa_debug; /* debug enabled? */ boolean_t spa_is_root; /* pool is root */ int spa_minref; /* num refs when first opened */ int spa_mode; /* FREAD | FWRITE */ Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/metaslab_impl.h =================================================================== --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/metaslab_impl.h (revision 223325) +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/metaslab_impl.h (working copy) @@ -21,6 +21,7 @@ /* * Copyright 2009 Sun Microsystems, Inc. All rights reserved. * Use is subject to license terms. + * Copyright (c) 2011 by Delphix. All rights reserved. */ #ifndef _SYS_METASLAB_IMPL_H @@ -52,6 +53,7 @@ avl_tree_t mg_metaslab_tree; uint64_t mg_aliquot; uint64_t mg_bonus_area; + uint64_t mg_alloc_failures; int64_t mg_bias; int64_t mg_activation_count; metaslab_class_t *mg_class; Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/spa.h =================================================================== --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/spa.h (revision 223325) +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/spa.h (working copy) @@ -20,6 +20,7 @@ */ /* * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 2011 by Delphix. All rights reserved. */ #ifndef _SYS_SPA_H @@ -698,6 +699,13 @@ #define dprintf_bp(bp, fmt, ...) #endif +extern boolean_t spa_debug_enabled(spa_t *spa); +#define spa_dbgmsg(spa, ...) \ +{ \ + if (spa_debug_enabled(spa)) \ + zfs_dbgmsg(__VA_ARGS__); \ +} + extern int spa_mode_global; /* mode, e.g. FREAD | FWRITE */ #ifdef __cplusplus Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c =================================================================== --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c (revision 223325) +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c (working copy) @@ -20,6 +20,7 @@ */ /* * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 2011 by Delphix. All rights reserved. */ #include @@ -85,6 +86,7 @@ #ifdef _KERNEL extern vmem_t *zio_alloc_arena; #endif +extern int zfs_mg_alloc_failures; /* * An allocating zio is one that either currently has the DVA allocate @@ -160,6 +162,12 @@ zio_data_buf_cache[c - 1] = zio_data_buf_cache[c]; } + /* + * The zio write taskqs have 1 thread per cpu, allow 1/2 of the taskqs + * to fail 3 times per txg or 8 failures, whichever is greater. + */ + zfs_mg_alloc_failures = MAX((3 * max_ncpus / 2), 8); + zio_inject_init(); } @@ -2135,6 +2143,7 @@ metaslab_class_t *mc = spa_normal_class(spa); blkptr_t *bp = zio->io_bp; int error; + int flags = 0; if (zio->io_gang_leader == NULL) { ASSERT(zio->io_child_type > ZIO_CHILD_GANG); @@ -2147,10 +2156,21 @@ ASSERT3U(zio->io_prop.zp_copies, <=, spa_max_replication(spa)); ASSERT3U(zio->io_size, ==, BP_GET_PSIZE(bp)); + /* + * The dump device does not support gang blocks so allocation on + * behalf of the dump device (i.e. ZIO_FLAG_NODATA) must avoid + * the "fast" gang feature. + */ + flags |= (zio->io_flags & ZIO_FLAG_NODATA) ? METASLAB_GANG_AVOID : 0; + flags |= (zio->io_flags & ZIO_FLAG_GANG_CHILD) ? + METASLAB_GANG_CHILD : 0; error = metaslab_alloc(spa, mc, zio->io_size, bp, - zio->io_prop.zp_copies, zio->io_txg, NULL, 0); + zio->io_prop.zp_copies, zio->io_txg, NULL, flags); if (error) { + spa_dbgmsg(spa, "%s: metaslab allocation failure: zio %p, " + "size %llu, error %d", spa_name(spa), zio, zio->io_size, + error); if (error == ENOSPC && zio->io_size > SPA_MINBLOCKSIZE) return (zio_write_gang_block(zio)); zio->io_error = error; --------------070809000803040501070106-- From owner-zfs-devel@FreeBSD.ORG Mon Jun 20 09:17:27 2011 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3420D106564A; Mon, 20 Jun 2011 09:17:27 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from mail.vx.sk (mail.vx.sk [IPv6:2a01:4f8:100:1043::3]) by mx1.freebsd.org (Postfix) with ESMTP id AEAC18FC08; Mon, 20 Jun 2011 09:17:26 +0000 (UTC) Received: from core.vx.sk (localhost [127.0.0.1]) by mail.vx.sk (Postfix) with ESMTP id 341721792E4; Mon, 20 Jun 2011 11:17:24 +0200 (CEST) X-Virus-Scanned: amavisd-new at mail.vx.sk Received: from mail.vx.sk ([127.0.0.1]) by core.vx.sk (mail.vx.sk [127.0.0.1]) (amavisd-new, port 10024) with LMTP id vFgJzTwso-+g; Mon, 20 Jun 2011 11:17:22 +0200 (CEST) Received: from [10.0.3.3] (188-167-66-148.dynamic.chello.sk [188.167.66.148]) by mail.vx.sk (Postfix) with ESMTPSA id AB5841792D4; Mon, 20 Jun 2011 11:17:21 +0200 (CEST) Message-ID: <4DFF1021.8050706@FreeBSD.org> Date: Mon, 20 Jun 2011 11:17:21 +0200 From: Martin Matuska User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.17) Gecko/20110516 Thunderbird/3.1.10 MIME-Version: 1.0 To: zfs-devel@FreeBSD.org Content-Type: multipart/mixed; boundary="------------050704070709090909090007" Cc: Pawel Jakub Dawidek Subject: [CFR][ZFS] merge Illumos rev. 13387 (refcompressratio property) X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 20 Jun 2011 09:17:27 -0000 This is a multi-part message in MIME format. --------------050704070709090909090007 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Add a "REFCOMPRESSRATIO" property, which is the compression ratio based on data referenced. For snapshots, this is the same as COMPRESSRATIO, but for filesystems/volumes, the COMPRESSRATIO is based on the data "USED" (ie, includes blocks in children, but not blocks shared with the origin). This is needed to figure out how much space a filesystem would use if it were not compressed (ignoring snapshots). https://www.illumos.org/issues/1092 https://www.illumos.org/projects/illumos-gate/repository/revisions/13387 If there are no objections, I will commit this to -HEAD -- Martin Matuska FreeBSD committer http://blog.vx.sk --------------050704070709090909090007 Content-Type: text/x-patch; name="13387.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="13387.patch" Index: cddl/contrib/opensolaris/cmd/zfs/zfs.8 =================================================================== --- cddl/contrib/opensolaris/cmd/zfs/zfs.8 (revision 223325) +++ cddl/contrib/opensolaris/cmd/zfs/zfs.8 (working copy) @@ -6,6 +6,7 @@ .\" The contents of this file are subject to the terms of the Common Development and Distribution License (the "License"). You may not use this file except in compliance with the License. You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE or http://www.opensolaris.org/os/licensing. .\" See the License for the specific language governing permissions and limitations under the License. When distributing Covered Code, include this CDDL HEADER in each file and include the License file at usr/src/OPENSOLARIS.LICENSE. If applicable, add the following below this CDDL HEADER, with .\" the fields enclosed by brackets "[]" replaced with your own identifying information: Portions Copyright [yyyy] [name of copyright owner] +.\" Copyright 2011 by Delphix. All rights reserved. .TH zfs 1M "24 Sep 2009" "SunOS 5.11" "System Administration Commands" .SH NAME zfs \- configures ZFS file systems @@ -389,7 +390,7 @@ .ad .sp .6 .RS 4n -The compression ratio achieved for this dataset, expressed as a multiplier. Compression can be turned on by running: \fBzfs set compression=on \fIdataset\fR\fR. The default value is \fBoff\fR. +For non-snapshots, the compression ratio achieved for the \fBused\fR space of this dataset, expressed as a multiplier. The \fBused\fR property includes descendant datasets, and, for clones, does not include the space shared with the origin snapshot. For snapshots, the \fBcompressratio\fR is the same as the \fBrefcompressratio\fR property. Compression can be turned on by running: \fBzfs set compression=on \fIdataset\fR\fR. The default value is \fBoff\fR. .RE .sp @@ -453,6 +454,17 @@ .ne 2 .mk .na +\fB\fBrefcompressratio\fR\fR +.ad +.sp .6 +.RS 4n +The compression ratio achieved for the \fBreferenced\fR space of this dataset, expressed as a multiplier. See also the \fBcompressratio\fR property. +.RE + +.sp +.ne 2 +.mk +.na \fB\fBtype\fR\fR .ad .sp .6 @@ -1278,7 +1290,7 @@ Force an unmount of any file systems using the \fBunmount -f\fR command. This option has no effect on non-file systems or unmounted file systems. .RE -Extreme care should be taken when applying either the \fB-r\fR or the \fB-f\fR options, as they can destroy large portions of a pool and cause unexpected behavior for mounted file systems in use. +Extreme care should be taken when applying either the \fB-r\fR or the \fB-R\fR options, as they can destroy large portions of a pool and cause unexpected behavior for mounted file systems in use. .RE .sp Index: cddl/contrib/opensolaris/lib/libzfs/common/libzfs_dataset.c =================================================================== --- cddl/contrib/opensolaris/lib/libzfs/common/libzfs_dataset.c (revision 223325) +++ cddl/contrib/opensolaris/lib/libzfs/common/libzfs_dataset.c (working copy) @@ -22,6 +22,7 @@ /* * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved. * Copyright 2010 Nexenta Systems, Inc. All rights reserved. + * Copyright (c) 2011 by Delphix. All rights reserved. */ #include @@ -2038,6 +2039,7 @@ } break; + case ZFS_PROP_REFRATIO: case ZFS_PROP_COMPRESSRATIO: if (get_numeric_property(zhp, prop, src, &source, &val) != 0) return (-1); Index: sys/cddl/contrib/opensolaris/common/zfs/zfs_prop.c =================================================================== --- sys/cddl/contrib/opensolaris/common/zfs/zfs_prop.c (revision 223325) +++ sys/cddl/contrib/opensolaris/common/zfs/zfs_prop.c (working copy) @@ -20,6 +20,7 @@ */ /* * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 2011 by Delphix. All rights reserved. */ /* Portions Copyright 2010 Robert Milkowski */ @@ -311,6 +312,9 @@ zprop_register_number(ZFS_PROP_COMPRESSRATIO, "compressratio", 0, PROP_READONLY, ZFS_TYPE_DATASET, "<1.00x or higher if compressed>", "RATIO"); + zprop_register_number(ZFS_PROP_REFRATIO, "refcompressratio", 0, + PROP_READONLY, ZFS_TYPE_DATASET, + "<1.00x or higher if compressed>", "REFRATIO"); zprop_register_number(ZFS_PROP_VOLBLOCKSIZE, "volblocksize", ZVOL_DEFAULT_BLOCKSIZE, PROP_ONETIME, ZFS_TYPE_VOLUME, "512 to 128k, power of 2", "VOLBLOCK"); Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dsl_dataset.c =================================================================== --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dsl_dataset.c (revision 223325) +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dsl_dataset.c (working copy) @@ -20,6 +20,7 @@ */ /* * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 2011 by Delphix. All rights reserved. */ #include @@ -2150,7 +2151,7 @@ void dsl_dataset_stats(dsl_dataset_t *ds, nvlist_t *nv) { - uint64_t refd, avail, uobjs, aobjs; + uint64_t refd, avail, uobjs, aobjs, ratio; dsl_dir_stats(ds->ds_dir, nv); @@ -2177,6 +2178,11 @@ dsl_prop_nvlist_add_uint64(nv, ZFS_PROP_DEFER_DESTROY, DS_IS_DEFER_DESTROY(ds) ? 1 : 0); + ratio = ds->ds_phys->ds_compressed_bytes == 0 ? 100 : + (ds->ds_phys->ds_uncompressed_bytes * 100 / + ds->ds_phys->ds_compressed_bytes); + dsl_prop_nvlist_add_uint64(nv, ZFS_PROP_REFRATIO, ratio); + if (ds->ds_phys->ds_next_snap_obj) { /* * This is a snapshot; override the dd's space used with @@ -2184,10 +2190,7 @@ */ dsl_prop_nvlist_add_uint64(nv, ZFS_PROP_USED, ds->ds_phys->ds_unique_bytes); - dsl_prop_nvlist_add_uint64(nv, ZFS_PROP_COMPRESSRATIO, - ds->ds_phys->ds_compressed_bytes == 0 ? 100 : - (ds->ds_phys->ds_uncompressed_bytes * 100 / - ds->ds_phys->ds_compressed_bytes)); + dsl_prop_nvlist_add_uint64(nv, ZFS_PROP_COMPRESSRATIO, ratio); } } Index: sys/cddl/contrib/opensolaris/uts/common/sys/fs/zfs.h =================================================================== --- sys/cddl/contrib/opensolaris/uts/common/sys/fs/zfs.h (revision 223325) +++ sys/cddl/contrib/opensolaris/uts/common/sys/fs/zfs.h (working copy) @@ -21,6 +21,7 @@ /* * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 2011 by Delphix. All rights reserved. */ /* Portions Copyright 2010 Robert Milkowski */ @@ -124,6 +125,7 @@ ZFS_PROP_DEDUP, ZFS_PROP_MLSLABEL, ZFS_PROP_SYNC, + ZFS_PROP_REFRATIO, ZFS_NUM_PROPS } zfs_prop_t; --------------050704070709090909090007-- From owner-zfs-devel@FreeBSD.ORG Wed Jun 29 17:49:12 2011 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 00E2D106566B for ; Wed, 29 Jun 2011 17:49:12 +0000 (UTC) (envelope-from pawel@dawidek.net) Received: from mail.dawidek.net (60.wheelsystems.com [83.12.187.60]) by mx1.freebsd.org (Postfix) with ESMTP id 9672F8FC16 for ; Wed, 29 Jun 2011 17:49:11 +0000 (UTC) Received: from localhost (89-73-195-149.dynamic.chello.pl [89.73.195.149]) by mail.dawidek.net (Postfix) with ESMTPSA id BD2125CD; Wed, 29 Jun 2011 18:33:35 +0200 (CEST) Date: Wed, 29 Jun 2011 19:42:09 +0200 From: Pawel Jakub Dawidek To: Martin Matuska Message-ID: <20110629174209.GB1741@garage.freebsd.pl> References: <4DFF1002.3070402@FreeBSD.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="b5gNqxB1S1yM7hjW" Content-Disposition: inline In-Reply-To: <4DFF1002.3070402@FreeBSD.org> X-OS: FreeBSD 9.0-CURRENT amd64 User-Agent: Mutt/1.5.21 (2010-09-15) Cc: zfs-devel@FreeBSD.org Subject: Re: [CFR][ZFS] merge Illumos rev. 13379 (writing to imbalanced vdevs) X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 29 Jun 2011 17:49:12 -0000 --b5gNqxB1S1yM7hjW Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Mon, Jun 20, 2011 at 11:16:50AM +0200, Martin Matuska wrote: > zfs tries to allocate blocks evenly across all devices. This means when > devices are imbalanced zfs will lots of CPU searching for space on devices > which tend to be pretty full. >=20 > https://www.illumos.org/issues/1051 > https://www.illumos.org/projects/illumos-gate/repository/revisions/13379 >=20 > If there are no objections, I will commit this to -HEAD Some comments inline. > @@ -5137,6 +5138,7 @@ > */ > kernel_init(FREAD | FWRITE); > VERIFY(spa_open(zs->zs_pool, &spa, FTAG) =3D=3D 0); > + spa->spa_debug =3D B_TRUE; > zs->zs_spa =3D spa; > =20 > spa->spa_dedup_ditto =3D 2 * ZIO_DEDUPDITTO_MIN; Do we want spa debug to be enabled by default? If we don't then please enable it only when DEBUG is defined. > /* > + * This value defines the number of allowed allocation failures per vdev. > + * If a device reaches this threshold in a given txg then we consider sk= ipping > + * allocations on that device. > + */ > +int zfs_mg_alloc_failures; In FreeBSD we probably want sysctl and loader tunable for this. I think it should be fine to make sysctl read/write, but I haven't looked very closely. --=20 Pawel Jakub Dawidek http://www.wheelsystems.com FreeBSD committer http://www.FreeBSD.org Am I Evil? Yes, I Am! http://yomoli.com --b5gNqxB1S1yM7hjW Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (FreeBSD) iEYEARECAAYFAk4LY/AACgkQForvXbEpPzS6xwCgu+AnY2WzueibMYyI+B+ifhxX gEAAoJYHHL+8S2yacFqk2p3Qh8TjpE/e =Fy5P -----END PGP SIGNATURE----- --b5gNqxB1S1yM7hjW-- From owner-zfs-devel@FreeBSD.ORG Wed Jun 29 17:54:13 2011 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6B4D4106564A for ; Wed, 29 Jun 2011 17:54:13 +0000 (UTC) (envelope-from pawel@dawidek.net) Received: from mail.dawidek.net (60.wheelsystems.com [83.12.187.60]) by mx1.freebsd.org (Postfix) with ESMTP id CAA6B8FC16 for ; Wed, 29 Jun 2011 17:54:11 +0000 (UTC) Received: from localhost (89-73-195-149.dynamic.chello.pl [89.73.195.149]) by mail.dawidek.net (Postfix) with ESMTPSA id 365ED5C0; Wed, 29 Jun 2011 18:27:57 +0200 (CEST) Date: Wed, 29 Jun 2011 19:36:30 +0200 From: Pawel Jakub Dawidek To: Martin Matuska Message-ID: <20110629173630.GA1741@garage.freebsd.pl> References: <4DFF0FF3.1030308@FreeBSD.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="G4iJoqBmSsgzjUCe" Content-Disposition: inline In-Reply-To: <4DFF0FF3.1030308@FreeBSD.org> X-OS: FreeBSD 9.0-CURRENT amd64 User-Agent: Mutt/1.5.21 (2010-09-15) Cc: zfs-devel@FreeBSD.org, Edward Tomasz Napierala Subject: Re: [CFR][ZFS] merge Illumos rev. 13370 (resurrect "aclmode" property) X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 29 Jun 2011 17:54:13 -0000 --G4iJoqBmSsgzjUCe Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Mon, Jun 20, 2011 at 11:16:35AM +0200, Martin Matuska wrote: > Resurrect "aclmode" property for ZFS. > The Illumos changeset includes Issues submitted by trasz (279, 664). >=20 > https://www.illumos.org/issues/742 > https://www.illumos.org/issues/664 > https://www.illumos.org/issues/279 > https://www.illumos.org/projects/illumos-gate/repository/revisions/13370 >=20 > If there are no objections, I will commit this to -HEAD. No objection from me, although I'd prefer to wait for Edward's opinion. AFAIK he is also fine, but we should wait. It may take a short while as he just become a father:) > Index: cddl/contrib/opensolaris/cmd/zfs/zfs.8 > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > --- cddl/contrib/opensolaris/cmd/zfs/zfs.8 (revision 223325) > +++ cddl/contrib/opensolaris/cmd/zfs/zfs.8 (working copy) > @@ -6,6 +6,7 @@ > .\" The contents of this file are subject to the terms of the Common Dev= elopment and Distribution License (the "License"). You may not use this fi= le except in compliance with the License. You can obtain a copy of the lice= nse at usr/src/OPENSOLARIS.LICENSE or http://www.opensolaris.org/os/licensi= ng. > .\" See the License for the specific language governing permissions and= limitations under the License. When distributing Covered Code, include thi= s CDDL HEADER in each file and include the License file at usr/src/OPENSOLA= RIS.LICENSE. If applicable, add the following below this CDDL HEADER, with > .\" the fields enclosed by brackets "[]" replaced with your own identify= ing information: Portions Copyright [yyyy] [name of copyright owner] > +.\" Copyright 2011 Nexenta Systems, Inc. All rights reserved. > .TH zfs 1M "24 Sep 2009" "SunOS 5.11" "System Administration Commands" > .SH NAME > zfs \- configures ZFS file systems > @@ -630,7 +631,7 @@ > .ad > .sp .6 > .RS 4n > -Controls how an \fBACL\fR is modified during \fBchmod\fR(2). A file syst= em with an \fBaclmode\fR property of \fBdiscard\fR deletes all \fBACL\fR en= tries that do not represent the mode of the file. An \fBaclmode\fR property= of \fBgroupmask\fR (the default) reduces user or group permissions. The pe= rmissions are reduced, such that they are no greater than the group permiss= ion bits, unless it is a user entry that has the same \fBUID\fR as the owne= r of the file or directory. In this case, the \fBACL\fR permissions are red= uced so that they are no greater than owner permission bits. A file system = with an \fBaclmode\fR property of \fBpassthrough\fR indicates that no chang= es are made to the \fBACL\fR other than generating the necessary \fBACL\fR = entries to represent the new mode of the file or directory. > +Controls how an \fBACL\fR is modified during \fBchmod\fR(2). A file syst= em with an \fBaclmode\fR property of \fBdiscard\fR (the default) deletes al= l \fBACL\fR entries that do not represent the mode of the file. An \fBaclmo= de\fR property of \fBgroupmask\fR reduces permissions granted in all \fBALL= OW\fR entries found in the \fBACL\fR such that they are no greater than the= group permissions specified by \fBchmod\fR. A file system with an \fBaclm= ode\fR property of \fBpassthrough\fR indicates that no changes are made to = the \fBACL\fR other than creating or updating the necessary \fBACL\fR entri= es to represent the new mode of the file or directory. > .RE > =20 > .sp > @@ -2685,7 +2686,7 @@ > pool/home/bob readonly off default > pool/home/bob zoned off default > pool/home/bob snapdir hidden default > -pool/home/bob aclmode groupmask default > +pool/home/bob aclmode discard default > pool/home/bob aclinherit restricted default > pool/home/bob canmount on default > pool/home/bob shareiscsi off default > Index: sys/cddl/contrib/opensolaris/common/acl/acl_common.c > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > --- sys/cddl/contrib/opensolaris/common/acl/acl_common.c (revision 223325) > +++ sys/cddl/contrib/opensolaris/common/acl/acl_common.c (working copy) > @@ -20,6 +20,7 @@ > */ > /* > * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights re= served. > + * Copyright 2011 Nexenta Systems, Inc. All rights reserved. > */ > =20 > #include > @@ -376,7 +377,7 @@ > * by nfsace, assuming aclent_t -> nfsace semantics. > */ > static uint32_t > -mode_to_ace_access(mode_t mode, int isdir, int isowner, int isallow) > +mode_to_ace_access(mode_t mode, boolean_t isdir, int isowner, int isallo= w) > { > uint32_t access =3D 0; > int haswriteperm =3D 0; > @@ -419,7 +420,7 @@ > access |=3D ACE_DELETE_CHILD; > } > /* exec */ > - if (mode & 01) { > + if (mode & S_IXOTH) { > access |=3D ACE_EXECUTE; > } > =20 > @@ -670,7 +671,7 @@ > } > =20 > static int > -convert_aent_to_ace(aclent_t *aclentp, int aclcnt, int isdir, > +convert_aent_to_ace(aclent_t *aclentp, int aclcnt, boolean_t isdir, > ace_t **retacep, int *retacecnt) > { > ace_t *acep; > @@ -696,7 +697,7 @@ > dfaclcnt =3D aclcnt - i; > } > =20 > - if (dfaclcnt && isdir =3D=3D 0) { > + if (dfaclcnt && !isdir) { > return (EINVAL); > } > =20 > @@ -734,7 +735,7 @@ > } > =20 > static int > -ace_mask_to_mode(uint32_t mask, o_mode_t *modep, int isdir) > +ace_mask_to_mode(uint32_t mask, o_mode_t *modep, boolean_t isdir) > { > int error =3D 0; > o_mode_t mode =3D 0; > @@ -1031,7 +1032,7 @@ > } > =20 > static int > -ace_allow_to_mode(uint32_t mask, o_mode_t *modep, int isdir) > +ace_allow_to_mode(uint32_t mask, o_mode_t *modep, boolean_t isdir) > { > /* ACE_READ_ACL and ACE_READ_ATTRIBUTES must both be set */ > if ((mask & (ACE_READ_ACL | ACE_READ_ATTRIBUTES)) !=3D > @@ -1044,7 +1045,7 @@ > =20 > static int > acevals_to_aent(acevals_t *vals, aclent_t *dest, ace_list_t *list, > - uid_t owner, gid_t group, int isdir) > + uid_t owner, gid_t group, boolean_t isdir) > { > int error; > uint32_t flips =3D ACE_POSIX_SUPPORTED_BITS; > @@ -1084,7 +1085,7 @@ > =20 > static int > ace_list_to_aent(ace_list_t *list, aclent_t **aclentp, int *aclcnt, > - uid_t owner, gid_t group, int isdir) > + uid_t owner, gid_t group, boolean_t isdir) > { > int error =3D 0; > aclent_t *aent, *result =3D NULL; > @@ -1264,7 +1265,7 @@ > static int > ln_ace_to_aent(ace_t *ace, int n, uid_t owner, gid_t group, > aclent_t **aclentp, int *aclcnt, aclent_t **dfaclentp, int *dfaclcnt, > - int isdir) > + boolean_t isdir) > { > int error =3D 0; > ace_t *acep; > @@ -1459,7 +1460,7 @@ > } > =20 > static int > -convert_ace_to_aent(ace_t *acebufp, int acecnt, int isdir, > +convert_ace_to_aent(ace_t *acebufp, int acecnt, boolean_t isdir, > uid_t owner, gid_t group, aclent_t **retaclentp, int *retaclcnt) > { > int error =3D 0; > @@ -1501,7 +1502,7 @@ > =20 > =20 > int > -acl_translate(acl_t *aclp, int target_flavor, int isdir, uid_t owner, > +acl_translate(acl_t *aclp, int target_flavor, boolean_t isdir, uid_t own= er, > gid_t group) > { > int aclcnt; > @@ -1573,101 +1574,105 @@ > } > =20 > void > -acl_trivial_access_masks(mode_t mode, uint32_t *allow0, uint32_t *deny1, > - uint32_t *deny2, uint32_t *owner, uint32_t *group, uint32_t *everyon= e) > +acl_trivial_access_masks(mode_t mode, boolean_t isdir, trivial_acl_t *ma= sks) > { > - *deny1 =3D *deny2 =3D *allow0 =3D *group =3D 0; > + uint32_t read_mask =3D ACE_READ_DATA; > + uint32_t write_mask =3D ACE_WRITE_DATA|ACE_APPEND_DATA; > + uint32_t execute_mask =3D ACE_EXECUTE; > =20 > + (void) isdir; /* will need this later */ > + > + masks->deny1 =3D 0; > if (!(mode & S_IRUSR) && (mode & (S_IRGRP|S_IROTH))) > - *deny1 |=3D ACE_READ_DATA; > + masks->deny1 |=3D read_mask; > if (!(mode & S_IWUSR) && (mode & (S_IWGRP|S_IWOTH))) > - *deny1 |=3D ACE_WRITE_DATA|ACE_APPEND_DATA; > + masks->deny1 |=3D write_mask; > if (!(mode & S_IXUSR) && (mode & (S_IXGRP|S_IXOTH))) > - *deny1 |=3D ACE_EXECUTE; > + masks->deny1 |=3D execute_mask; > =20 > + masks->deny2 =3D 0; > if (!(mode & S_IRGRP) && (mode & S_IROTH)) > - *deny2 =3D ACE_READ_DATA; > + masks->deny2 |=3D read_mask; > if (!(mode & S_IWGRP) && (mode & S_IWOTH)) > - *deny2 |=3D ACE_WRITE_DATA|ACE_APPEND_DATA; > + masks->deny2 |=3D write_mask; > if (!(mode & S_IXGRP) && (mode & S_IXOTH)) > - *deny2 |=3D ACE_EXECUTE; > + masks->deny2 |=3D execute_mask; > =20 > + masks->allow0 =3D 0; > if ((mode & S_IRUSR) && (!(mode & S_IRGRP) && (mode & S_IROTH))) > - *allow0 |=3D ACE_READ_DATA; > + masks->allow0 |=3D read_mask; > if ((mode & S_IWUSR) && (!(mode & S_IWGRP) && (mode & S_IWOTH))) > - *allow0 |=3D ACE_WRITE_DATA|ACE_APPEND_DATA; > + masks->allow0 |=3D write_mask; > if ((mode & S_IXUSR) && (!(mode & S_IXGRP) && (mode & S_IXOTH))) > - *allow0 |=3D ACE_EXECUTE; > + masks->allow0 |=3D execute_mask; > =20 > - *owner =3D ACE_WRITE_ATTRIBUTES|ACE_WRITE_OWNER|ACE_WRITE_ACL| > + masks->owner =3D ACE_WRITE_ATTRIBUTES|ACE_WRITE_OWNER|ACE_WRITE_ACL| > ACE_WRITE_NAMED_ATTRS|ACE_READ_ACL|ACE_READ_ATTRIBUTES| > ACE_READ_NAMED_ATTRS|ACE_SYNCHRONIZE; > if (mode & S_IRUSR) > - *owner |=3D ACE_READ_DATA; > + masks->owner |=3D read_mask; > if (mode & S_IWUSR) > - *owner |=3D ACE_WRITE_DATA|ACE_APPEND_DATA; > + masks->owner |=3D write_mask; > if (mode & S_IXUSR) > - *owner |=3D ACE_EXECUTE; > + masks->owner |=3D execute_mask; > =20 > - *group =3D ACE_READ_ACL|ACE_READ_ATTRIBUTES| ACE_READ_NAMED_ATTRS| > + masks->group =3D ACE_READ_ACL|ACE_READ_ATTRIBUTES|ACE_READ_NAMED_ATTRS| > ACE_SYNCHRONIZE; > if (mode & S_IRGRP) > - *group |=3D ACE_READ_DATA; > + masks->group |=3D read_mask; > if (mode & S_IWGRP) > - *group |=3D ACE_WRITE_DATA|ACE_APPEND_DATA; > + masks->group |=3D write_mask; > if (mode & S_IXGRP) > - *group |=3D ACE_EXECUTE; > + masks->group |=3D execute_mask; > =20 > - *everyone =3D ACE_READ_ACL|ACE_READ_ATTRIBUTES| ACE_READ_NAMED_ATTRS| > + masks->everyone =3D ACE_READ_ACL|ACE_READ_ATTRIBUTES|ACE_READ_NAMED_ATT= RS| > ACE_SYNCHRONIZE; > if (mode & S_IROTH) > - *everyone |=3D ACE_READ_DATA; > + masks->everyone |=3D read_mask; > if (mode & S_IWOTH) > - *everyone |=3D ACE_WRITE_DATA|ACE_APPEND_DATA; > + masks->everyone |=3D write_mask; > if (mode & S_IXOTH) > - *everyone |=3D ACE_EXECUTE; > + masks->everyone |=3D execute_mask; > } > =20 > int > -acl_trivial_create(mode_t mode, ace_t **acl, int *count) > +acl_trivial_create(mode_t mode, boolean_t isdir, ace_t **acl, int *count) > { > - uint32_t deny1, deny2; > - uint32_t allow0; > - uint32_t owner, group, everyone; > - int index =3D 0; > + int index =3D 0; > int error; > + trivial_acl_t masks; > =20 > *count =3D 3; > - acl_trivial_access_masks(mode, &allow0, &deny1, &deny2, &owner, &group, > - &everyone); > + acl_trivial_access_masks(mode, isdir, &masks); > =20 > - if (allow0) > + if (masks.allow0) > (*count)++; > - if (deny1) > + if (masks.deny1) > (*count)++; > - if (deny2) > + if (masks.deny2) > (*count)++; > =20 > if ((error =3D cacl_malloc((void **)acl, *count * sizeof (ace_t))) !=3D= 0) > return (error); > =20 > - if (allow0) { > - SET_ACE(acl, index, -1, allow0, ACE_ACCESS_ALLOWED_ACE_TYPE, > - ACE_OWNER); > + if (masks.allow0) { > + SET_ACE(acl, index, -1, masks.allow0, > + ACE_ACCESS_ALLOWED_ACE_TYPE, ACE_OWNER); > } > - if (deny1) { > - SET_ACE(acl, index, -1, deny1, ACE_ACCESS_DENIED_ACE_TYPE, > - ACE_OWNER); > + if (masks.deny1) { > + SET_ACE(acl, index, -1, masks.deny1, > + ACE_ACCESS_DENIED_ACE_TYPE, ACE_OWNER); > } > - if (deny2) { > - SET_ACE(acl, index, -1, deny2, ACE_ACCESS_DENIED_ACE_TYPE, > - ACE_GROUP|ACE_IDENTIFIER_GROUP); > + if (masks.deny2) { > + SET_ACE(acl, index, -1, masks.deny2, > + ACE_ACCESS_DENIED_ACE_TYPE, ACE_GROUP|ACE_IDENTIFIER_GROUP); > } > =20 > - SET_ACE(acl, index, -1, owner, ACE_ACCESS_ALLOWED_ACE_TYPE, ACE_OWNER); > - SET_ACE(acl, index, -1, group, ACE_ACCESS_ALLOWED_ACE_TYPE, > + SET_ACE(acl, index, -1, masks.owner, ACE_ACCESS_ALLOWED_ACE_TYPE, > + ACE_OWNER); > + SET_ACE(acl, index, -1, masks.group, ACE_ACCESS_ALLOWED_ACE_TYPE, > ACE_IDENTIFIER_GROUP|ACE_GROUP); > - SET_ACE(acl, index, -1, everyone, ACE_ACCESS_ALLOWED_ACE_TYPE, > + SET_ACE(acl, index, -1, masks.everyone, ACE_ACCESS_ALLOWED_ACE_TYPE, > ACE_EVERYONE); > =20 > return (0); > Index: sys/cddl/contrib/opensolaris/common/acl/acl_common.h > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > --- sys/cddl/contrib/opensolaris/common/acl/acl_common.h (revision 223325) > +++ sys/cddl/contrib/opensolaris/common/acl/acl_common.h (working copy) > @@ -20,6 +20,7 @@ > */ > /* > * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights re= served. > + * Copyright 2011 Nexenta Systems, Inc. All rights reserved. > */ > =20 > #ifndef _ACL_COMMON_H > @@ -33,7 +34,14 @@ > extern "C" { > #endif > =20 > -extern ace_t trivial_acl[6]; > +typedef struct trivial_acl { > + uint32_t allow0; /* allow mask for bits only in owner */ > + uint32_t deny1; /* deny mask for bits not in owner */ > + uint32_t deny2; /* deny mask for bits not in group */ > + uint32_t owner; /* allow mask matching mode */ > + uint32_t group; /* allow mask matching mode */ > + uint32_t everyone; /* allow mask matching mode */ > +} trivial_acl_t; > =20 > extern int acltrivial(const char *); > extern void adjust_ace_pair(ace_t *pair, mode_t mode); > @@ -45,14 +53,14 @@ > #if !defined(_KERNEL) > extern acl_t *acl_alloc(acl_type_t); > extern void acl_free(acl_t *aclp); > -extern int acl_translate(acl_t *aclp, int target_flavor, > - int isdir, uid_t owner, gid_t group); > +extern int acl_translate(acl_t *aclp, int target_flavor, boolean_t isdir, > + uid_t owner, gid_t group); > #endif /* !_KERNEL */ > void ksort(caddr_t v, int n, int s, int (*f)()); > int cmp2acls(void *a, void *b); > -int acl_trivial_create(mode_t mode, ace_t **acl, int *count); > -void acl_trivial_access_masks(mode_t mode, uint32_t *allow0, uint32_t *d= eny1, > - uint32_t *deny2, uint32_t *owner, uint32_t *group, uint32_t *everyon= e); > +int acl_trivial_create(mode_t mode, boolean_t isdir, ace_t **acl, int *c= ount); > +void acl_trivial_access_masks(mode_t mode, boolean_t isdir, > + trivial_acl_t *masks); > =20 > #ifdef __cplusplus > } > Index: sys/cddl/contrib/opensolaris/common/zfs/zfs_prop.c > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > --- sys/cddl/contrib/opensolaris/common/zfs/zfs_prop.c (revision 223325) > +++ sys/cddl/contrib/opensolaris/common/zfs/zfs_prop.c (working copy) > @@ -104,6 +104,13 @@ > { NULL } > }; > =20 > + static zprop_index_t acl_mode_table[] =3D { > + { "discard", ZFS_ACL_DISCARD }, > + { "groupmask", ZFS_ACL_GROUPMASK }, > + { "passthrough", ZFS_ACL_PASSTHROUGH }, > + { NULL } > + }; > + > static zprop_index_t acl_inherit_table[] =3D { > { "discard", ZFS_ACL_DISCARD }, > { "noallow", ZFS_ACL_NOALLOW }, > @@ -207,6 +214,9 @@ > zprop_register_index(ZFS_PROP_SNAPDIR, "snapdir", ZFS_SNAPDIR_HIDDEN, > PROP_INHERIT, ZFS_TYPE_FILESYSTEM, > "hidden | visible", "SNAPDIR", snapdir_table); > + zprop_register_index(ZFS_PROP_ACLMODE, "aclmode", ZFS_ACL_DISCARD, > + PROP_INHERIT, ZFS_TYPE_FILESYSTEM, > + "discard | groupmask | passthrough", "ACLMODE", acl_mode_table); > zprop_register_index(ZFS_PROP_ACLINHERIT, "aclinherit", > ZFS_ACL_RESTRICTED, PROP_INHERIT, ZFS_TYPE_FILESYSTEM, > "discard | noallow | restricted | passthrough | passthrough-x", > @@ -370,13 +380,6 @@ > zprop_register_hidden(ZFS_PROP_OBJSETID, "objsetid", PROP_TYPE_NUMBER, > PROP_READONLY, ZFS_TYPE_DATASET, "OBJSETID"); > =20 > - /* > - * Property to be removed once libbe is integrated > - */ > - zprop_register_hidden(ZFS_PROP_PRIVATE, "priv_prop", > - PROP_TYPE_NUMBER, PROP_READONLY, ZFS_TYPE_FILESYSTEM, > - "PRIV_PROP"); > - > /* oddball properties */ > zprop_register_impl(ZFS_PROP_CREATION, "creation", PROP_TYPE_NUMBER, 0, > NULL, PROP_READONLY, ZFS_TYPE_DATASET, > Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c (revision = 223325) > +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c (working c= opy) > @@ -3223,7 +3223,8 @@ > uint64_t acl_obj; > new_mode =3D (pmode & S_IFMT) | (vap->va_mode & ~S_IFMT); > =20 > - zfs_acl_chmod_setattr(zp, &aclp, new_mode); > + if (err =3D zfs_acl_chmod_setattr(zp, &aclp, new_mode)) > + goto out; > =20 > mutex_enter(&zp->z_lock); > if (!zp->z_is_sa && ((acl_obj =3D zfs_external_acl(zp)) !=3D 0)) { > Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vfsops.c > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vfsops.c (revision= 223325) > +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vfsops.c (working = copy) > @@ -364,6 +364,14 @@ > } > =20 > static void > +acl_mode_changed_cb(void *arg, uint64_t newval) > +{ > + zfsvfs_t *zfsvfs =3D arg; > + > + zfsvfs->z_acl_mode =3D newval; > +} > + > +static void > acl_inherit_changed_cb(void *arg, uint64_t newval) > { > zfsvfs_t *zfsvfs =3D arg; > @@ -488,6 +496,8 @@ > error =3D error ? error : dsl_prop_register(ds, > "snapdir", snapdir_changed_cb, zfsvfs); > error =3D error ? error : dsl_prop_register(ds, > + "aclmode", acl_mode_changed_cb, zfsvfs); > + error =3D error ? error : dsl_prop_register(ds, > "aclinherit", acl_inherit_changed_cb, zfsvfs); > error =3D error ? error : dsl_prop_register(ds, > "vscan", vscan_changed_cb, zfsvfs); > @@ -525,6 +535,7 @@ > (void) dsl_prop_unregister(ds, "setuid", setuid_changed_cb, zfsvfs); > (void) dsl_prop_unregister(ds, "exec", exec_changed_cb, zfsvfs); > (void) dsl_prop_unregister(ds, "snapdir", snapdir_changed_cb, zfsvfs); > + (void) dsl_prop_unregister(ds, "aclmode", acl_mode_changed_cb, zfsvfs); > (void) dsl_prop_unregister(ds, "aclinherit", acl_inherit_changed_cb, > zfsvfs); > (void) dsl_prop_unregister(ds, "vscan", vscan_changed_cb, zfsvfs); > @@ -1202,6 +1213,9 @@ > VERIFY(dsl_prop_unregister(ds, "snapdir", snapdir_changed_cb, > zfsvfs) =3D=3D 0); > =20 > + VERIFY(dsl_prop_unregister(ds, "aclmode", acl_mode_changed_cb, > + zfsvfs) =3D=3D 0); > + > VERIFY(dsl_prop_unregister(ds, "aclinherit", > acl_inherit_changed_cb, zfsvfs) =3D=3D 0); > =20 > Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_acl.c > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_acl.c (revision 22= 3325) > +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_acl.c (working cop= y) > @@ -20,6 +20,7 @@ > */ > /* > * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights re= served. > + * Copyright 2011 Nexenta Systems, Inc. All rights reserved. > */ > =20 > #include > @@ -1327,76 +1328,9 @@ > return (sa_bulk_update(zp->z_sa_hdl, bulk, count, tx)); > } > =20 > -/* > - * Update access mask for prepended ACE > - * > - * This applies the "groupmask" value for aclmode property. > - */ > static void > -zfs_acl_prepend_fixup(zfs_acl_t *aclp, void *acep, void *origacep, > - mode_t mode, uint64_t owner) > +zfs_acl_chmod(vtype_t vtype, uint64_t mode, boolean_t trim, zfs_acl_t *a= clp) > { > - int rmask, wmask, xmask; > - int user_ace; > - uint16_t aceflags; > - uint32_t origmask, acepmask; > - uint64_t fuid; > - > - aceflags =3D aclp->z_ops.ace_flags_get(acep); > - fuid =3D aclp->z_ops.ace_who_get(acep); > - origmask =3D aclp->z_ops.ace_mask_get(origacep); > - acepmask =3D aclp->z_ops.ace_mask_get(acep); > - > - user_ace =3D (!(aceflags & > - (ACE_OWNER|ACE_GROUP|ACE_IDENTIFIER_GROUP))); > - > - if (user_ace && (fuid =3D=3D owner)) { > - rmask =3D S_IRUSR; > - wmask =3D S_IWUSR; > - xmask =3D S_IXUSR; > - } else { > - rmask =3D S_IRGRP; > - wmask =3D S_IWGRP; > - xmask =3D S_IXGRP; > - } > - > - if (origmask & ACE_READ_DATA) { > - if (mode & rmask) { > - acepmask &=3D ~ACE_READ_DATA; > - } else { > - acepmask |=3D ACE_READ_DATA; > - } > - } > - > - if (origmask & ACE_WRITE_DATA) { > - if (mode & wmask) { > - acepmask &=3D ~ACE_WRITE_DATA; > - } else { > - acepmask |=3D ACE_WRITE_DATA; > - } > - } > - > - if (origmask & ACE_APPEND_DATA) { > - if (mode & wmask) { > - acepmask &=3D ~ACE_APPEND_DATA; > - } else { > - acepmask |=3D ACE_APPEND_DATA; > - } > - } > - > - if (origmask & ACE_EXECUTE) { > - if (mode & xmask) { > - acepmask &=3D ~ACE_EXECUTE; > - } else { > - acepmask |=3D ACE_EXECUTE; > - } > - } > - aclp->z_ops.ace_mask_set(acep, acepmask); > -} > - > -static void > -zfs_acl_chmod(zfsvfs_t *zfsvfs, uint64_t mode, zfs_acl_t *aclp) > -{ > void *acep =3D NULL; > uint64_t who; > int new_count, new_bytes; > @@ -1407,30 +1341,31 @@ > zfs_acl_node_t *newnode; > size_t abstract_size =3D aclp->z_ops.ace_abstract_size(); > void *zacep; > - uint32_t owner, group, everyone; > - uint32_t deny1, deny2, allow0; > + boolean_t isdir; > + trivial_acl_t masks; > =20 > new_count =3D new_bytes =3D 0; > =20 > - acl_trivial_access_masks((mode_t)mode, &allow0, &deny1, &deny2, > - &owner, &group, &everyone); > + isdir =3D (vtype =3D=3D VDIR); > =20 > + acl_trivial_access_masks((mode_t)mode, isdir, &masks); > + > newnode =3D zfs_acl_node_alloc((abstract_size * 6) + aclp->z_acl_bytes); > =20 > zacep =3D newnode->z_acldata; > - if (allow0) { > - zfs_set_ace(aclp, zacep, allow0, ALLOW, -1, ACE_OWNER); > + if (masks.allow0) { > + zfs_set_ace(aclp, zacep, masks.allow0, ALLOW, -1, ACE_OWNER); > zacep =3D (void *)((uintptr_t)zacep + abstract_size); > new_count++; > new_bytes +=3D abstract_size; > - } if (deny1) { > - zfs_set_ace(aclp, zacep, deny1, DENY, -1, ACE_OWNER); > + } if (masks.deny1) { > + zfs_set_ace(aclp, zacep, masks.deny1, DENY, -1, ACE_OWNER); > zacep =3D (void *)((uintptr_t)zacep + abstract_size); > new_count++; > new_bytes +=3D abstract_size; > } > - if (deny2) { > - zfs_set_ace(aclp, zacep, deny2, DENY, -1, OWNING_GROUP); > + if (masks.deny2) { > + zfs_set_ace(aclp, zacep, masks.deny2, DENY, -1, OWNING_GROUP); > zacep =3D (void *)((uintptr_t)zacep + abstract_size); > new_count++; > new_bytes +=3D abstract_size; > @@ -1449,10 +1384,17 @@ > continue; > } > =20 > + /* > + * If this ACL has any inheritable ACEs, mark that in > + * the hints (which are later masked into the pflags) > + * so create knows to do inheritance. > + */ > + if (isdir && (inherit_flags & > + (ACE_FILE_INHERIT_ACE|ACE_DIRECTORY_INHERIT_ACE))) > + aclp->z_hints |=3D ZFS_INHERIT_ACE; > + > if ((type !=3D ALLOW && type !=3D DENY) || > (inherit_flags & ACE_INHERIT_ONLY_ACE)) { > - if (inherit_flags) > - aclp->z_hints |=3D ZFS_INHERIT_ACE; > switch (type) { > case ACE_ACCESS_ALLOWED_OBJECT_ACE_TYPE: > case ACE_ACCESS_DENIED_OBJECT_ACE_TYPE: > @@ -1465,20 +1407,13 @@ > =20 > /* > * Limit permissions to be no greater than > - * group permissions > + * group permissions. > + * The "aclinherit" and "aclmode" properties > + * affect policy for create and chmod(2), > + * respectively. > */ > - if (type =3D=3D ALLOW && zfsvfs->z_acl_inherit =3D=3D ZFS_ACL_RESTRIC= TED) { > - if (!(mode & S_IRGRP)) > - access_mask &=3D ~ACE_READ_DATA; > - if (!(mode & S_IWGRP)) > - access_mask &=3D > - ~(ACE_WRITE_DATA|ACE_APPEND_DATA); > - if (!(mode & S_IXGRP)) > - access_mask &=3D ~ACE_EXECUTE; > - access_mask &=3D > - ~(ACE_WRITE_OWNER|ACE_WRITE_ACL| > - ACE_WRITE_ATTRIBUTES|ACE_WRITE_NAMED_ATTRS); > - } > + if ((type =3D=3D ALLOW) && trim) > + access_mask &=3D masks.group; > } > zfs_set_ace(aclp, zacep, access_mask, type, who, iflags); > ace_size =3D aclp->z_ops.ace_size(acep); > @@ -1486,11 +1421,11 @@ > new_count++; > new_bytes +=3D ace_size; > } > - zfs_set_ace(aclp, zacep, owner, 0, -1, ACE_OWNER); > + zfs_set_ace(aclp, zacep, masks.owner, 0, -1, ACE_OWNER); > zacep =3D (void *)((uintptr_t)zacep + abstract_size); > - zfs_set_ace(aclp, zacep, group, 0, -1, OWNING_GROUP); > + zfs_set_ace(aclp, zacep, masks.group, 0, -1, OWNING_GROUP); > zacep =3D (void *)((uintptr_t)zacep + abstract_size); > - zfs_set_ace(aclp, zacep, everyone, 0, -1, ACE_EVERYONE); > + zfs_set_ace(aclp, zacep, masks.everyone, 0, -1, ACE_EVERYONE); > =20 > new_count +=3D 3; > new_bytes +=3D abstract_size * 3; > @@ -1502,17 +1437,27 @@ > list_insert_tail(&aclp->z_acl, newnode); > } > =20 > -void > +int > zfs_acl_chmod_setattr(znode_t *zp, zfs_acl_t **aclp, uint64_t mode) > { > + int error =3D 0; > + > mutex_enter(&zp->z_acl_lock); > mutex_enter(&zp->z_lock); > - *aclp =3D zfs_acl_alloc(zfs_acl_version_zp(zp)); > - (*aclp)->z_hints =3D zp->z_pflags & V4_ACL_WIDE_FLAGS; > - zfs_acl_chmod(zp->z_zfsvfs, mode, *aclp); > + if (zp->z_zfsvfs->z_acl_mode =3D=3D ZFS_ACL_DISCARD) > + *aclp =3D zfs_acl_alloc(zfs_acl_version_zp(zp)); > + else > + error =3D zfs_acl_node_read(zp, B_TRUE, aclp, B_TRUE); > + > + if (error =3D=3D 0) { > + (*aclp)->z_hints =3D zp->z_pflags & V4_ACL_WIDE_FLAGS; > + zfs_acl_chmod(ZTOV(zp)->v_type, mode, > + (zp->z_zfsvfs->z_acl_mode =3D=3D ZFS_ACL_GROUPMASK), *aclp); > + } > mutex_exit(&zp->z_lock); > mutex_exit(&zp->z_acl_lock); > - ASSERT(*aclp); > + > + return (error); > } > =20 > /* > @@ -1764,8 +1709,8 @@ > if (acl_ids->z_aclp =3D=3D NULL) { > mutex_enter(&dzp->z_acl_lock); > mutex_enter(&dzp->z_lock); > - if (!(flag & IS_ROOT_NODE) && (ZTOV(dzp)->v_type =3D=3D VDIR && > - (dzp->z_pflags & ZFS_INHERIT_ACE)) && > + if (!(flag & IS_ROOT_NODE) && > + (dzp->z_pflags & ZFS_INHERIT_ACE) && > !(dzp->z_pflags & ZFS_XATTR)) { > VERIFY(0 =3D=3D zfs_acl_node_read(dzp, B_TRUE, > &paclp, B_FALSE)); > @@ -1782,7 +1727,9 @@ > if (need_chmod) { > acl_ids->z_aclp->z_hints |=3D (vap->va_type =3D=3D VDIR) ? > ZFS_ACL_AUTO_INHERIT : 0; > - zfs_acl_chmod(zfsvfs, acl_ids->z_mode, acl_ids->z_aclp); > + zfs_acl_chmod(vap->va_type, acl_ids->z_mode, > + (zfsvfs->z_acl_inherit =3D=3D ZFS_ACL_RESTRICTED), > + acl_ids->z_aclp); > } > } > =20 > Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/zfs_vfsops.h > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/zfs_vfsops.h (revi= sion 223325) > +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/zfs_vfsops.h (work= ing copy) > @@ -55,6 +55,7 @@ > boolean_t z_fuid_dirty; /* need to sync fuid table ? */ > struct zfs_fuid_info *z_fuid_replay; /* fuid info for replay */ > zilog_t *z_log; /* intent log pointer */ > + uint_t z_acl_mode; /* acl chmod/mode behavior */ > uint_t z_acl_inherit; /* acl inheritance behavior */ > zfs_case_t z_case; /* case-sense */ > boolean_t z_utf8; /* utf8-only */ > Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/zfs_acl.h > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/zfs_acl.h (revisio= n 223325) > +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/zfs_acl.h (working= copy) > @@ -217,7 +217,7 @@ > extern int zfs_zaccess_rwx(struct znode *, mode_t, int, cred_t *); > extern int zfs_zaccess_unix(struct znode *, mode_t, cred_t *); > extern int zfs_acl_access(struct znode *, int, cred_t *); > -void zfs_acl_chmod_setattr(struct znode *, zfs_acl_t **, uint64_t); > +int zfs_acl_chmod_setattr(struct znode *, zfs_acl_t **, uint64_t); > int zfs_zaccess_delete(struct znode *, struct znode *, cred_t *); > int zfs_zaccess_rename(struct znode *, struct znode *, > struct znode *, struct znode *, cred_t *cr); > Index: sys/cddl/contrib/opensolaris/uts/common/sys/fs/zfs.h > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > --- sys/cddl/contrib/opensolaris/uts/common/sys/fs/zfs.h (revision 223325) > +++ sys/cddl/contrib/opensolaris/uts/common/sys/fs/zfs.h (working copy) > @@ -89,7 +89,7 @@ > ZFS_PROP_READONLY, > ZFS_PROP_ZONED, > ZFS_PROP_SNAPDIR, > - ZFS_PROP_PRIVATE, /* not exposed to user, temporary */ > + ZFS_PROP_ACLMODE, > ZFS_PROP_ACLINHERIT, > ZFS_PROP_CREATETXG, /* not exposed to the user */ > ZFS_PROP_NAME, /* not exposed to the user */ >=20 --=20 Pawel Jakub Dawidek http://www.wheelsystems.com FreeBSD committer http://www.FreeBSD.org Am I Evil? Yes, I Am! http://yomoli.com --G4iJoqBmSsgzjUCe Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (FreeBSD) iEYEARECAAYFAk4LYp4ACgkQForvXbEpPzT18QCgzMvS3Qun9iemGIKFLQ0MyqEA mTsAn0Cn7B86/h3dMxxIQSmUnP2XILhJ =RSEU -----END PGP SIGNATURE----- --G4iJoqBmSsgzjUCe-- From owner-zfs-devel@FreeBSD.ORG Wed Jul 6 12:45:16 2011 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E406C106566C; Wed, 6 Jul 2011 12:45:16 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from mail.vx.sk (mail.vx.sk [IPv6:2a01:4f8:100:1043::3]) by mx1.freebsd.org (Postfix) with ESMTP id 7DF988FC0A; Wed, 6 Jul 2011 12:45:16 +0000 (UTC) Received: from core.vx.sk (localhost [127.0.0.1]) by mail.vx.sk (Postfix) with ESMTP id A8CF615C525; Wed, 6 Jul 2011 14:45:15 +0200 (CEST) X-Virus-Scanned: amavisd-new at mail.vx.sk Received: from mail.vx.sk ([127.0.0.1]) by core.vx.sk (mail.vx.sk [127.0.0.1]) (amavisd-new, port 10024) with LMTP id Hs+au5mHY7Nx; Wed, 6 Jul 2011 14:45:13 +0200 (CEST) Received: from [10.0.3.3] (188-167-66-148.dynamic.chello.sk [188.167.66.148]) by mail.vx.sk (Postfix) with ESMTPSA id 13D5F15C51A; Wed, 6 Jul 2011 14:45:13 +0200 (CEST) Message-ID: <4E1458D8.8000602@FreeBSD.org> Date: Wed, 06 Jul 2011 14:45:12 +0200 From: Martin Matuska User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.17) Gecko/20110516 Thunderbird/3.1.10 MIME-Version: 1.0 To: Pawel Jakub Dawidek References: <4DFF1002.3070402@FreeBSD.org> <20110629174209.GB1741@garage.freebsd.pl> In-Reply-To: <20110629174209.GB1741@garage.freebsd.pl> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: zfs-devel@FreeBSD.org Subject: Re: [CFR][ZFS] merge Illumos rev. 13379 (writing to imbalanced vdevs) X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Jul 2011 12:45:17 -0000 Dňa 29.06.2011 19:42, Pawel Jakub Dawidek wrote / napísal(a): > On Mon, Jun 20, 2011 at 11:16:50AM +0200, Martin Matuska wrote: >> zfs tries to allocate blocks evenly across all devices. This means when >> devices are imbalanced zfs will lots of CPU searching for space on devices >> which tend to be pretty full. >> >> https://www.illumos.org/issues/1051 >> https://www.illumos.org/projects/illumos-gate/repository/revisions/13379 >> >> If there are no objections, I will commit this to -HEAD > Some comments inline. > >> @@ -5137,6 +5138,7 @@ >> */ >> kernel_init(FREAD | FWRITE); >> VERIFY(spa_open(zs->zs_pool, &spa, FTAG) == 0); >> + spa->spa_debug = B_TRUE; >> zs->zs_spa = spa; >> >> spa->spa_dedup_ditto = 2 * ZIO_DEDUPDITTO_MIN; > Do we want spa debug to be enabled by default? If we don't then please > enable it only when DEBUG is defined. > We are here in ztest.c, spa_debug (new boolean) is disabled by default. >> /* >> + * This value defines the number of allowed allocation failures per vdev. >> + * If a device reaches this threshold in a given txg then we consider skipping >> + * allocations on that device. >> + */ >> +int zfs_mg_alloc_failures; > In FreeBSD we probably want sysctl and loader tunable for this. > I think it should be fine to make sysctl read/write, but I haven't > looked very closely. zio.c: This is called in zio_init(), called by spa_init(): /* * The zio write taskqs have 1 thread per cpu, allow 1/2 of the taskqs * to fail 3 times per txg or 8 failures, whichever is greater. */ zfs_mg_alloc_failures = MAX((3 * max_ncpus / 2), 8); Looks like it is set on runtime so we have to change the code to make it properly tunable and set a default value. Maye comment out the code above and directly initialize with: int zfs_mg_alloc_failures = MAX((3 * max_ncpus / 2), 8); The setup might look like this: TUNABLE_INT("vfs.zfs.mg_alloc_failures", &zfs_mg_alloc_failures); SYSCTL_INT(_vfs_zfs, OID_AUTO, mg_alloc_failures, CTLFLAG_RW, &zfs_mg_alloc_failures, 0, "Number of allowed allocation failures per vdev"); Another question is if it should be under vfs.zfs. or vfs.zfs.vdev. -- Martin Matuska FreeBSD committer http://blog.vx.sk From owner-zfs-devel@FreeBSD.ORG Tue Jul 12 13:23:40 2011 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 477C3106564A for ; Tue, 12 Jul 2011 13:23:40 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from mail.vx.sk (mail.vx.sk [IPv6:2a01:4f8:100:1043::3]) by mx1.freebsd.org (Postfix) with ESMTP id D92118FC1B for ; Tue, 12 Jul 2011 13:23:39 +0000 (UTC) Received: from core.vx.sk (localhost [127.0.0.1]) by mail.vx.sk (Postfix) with ESMTP id E5B7E313F for ; Tue, 12 Jul 2011 15:23:38 +0200 (CEST) X-Virus-Scanned: amavisd-new at mail.vx.sk Received: from mail.vx.sk ([127.0.0.1]) by core.vx.sk (mail.vx.sk [127.0.0.1]) (amavisd-new, port 10024) with LMTP id WA856HSb4ldg for ; Tue, 12 Jul 2011 15:23:37 +0200 (CEST) Received: from [10.0.3.3] (188-167-66-148.dynamic.chello.sk [188.167.66.148]) by mail.vx.sk (Postfix) with ESMTPSA id DC7C13134 for ; Tue, 12 Jul 2011 15:23:36 +0200 (CEST) Message-ID: <4E1C4AD8.4080101@FreeBSD.org> Date: Tue, 12 Jul 2011 15:23:36 +0200 From: Martin Matuska User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:5.0) Gecko/20110627 Thunderbird/5.0 MIME-Version: 1.0 To: zfs-devel@FreeBSD.org Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Cc: Subject: Temporary clones not destroyed on incremental snapshot receive X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Jul 2011 13:23:40 -0000 In kern/157728, I describe a problem (with reproduction steps and a temporary workaround) of temporary clones not destroyed on incremental snapshot receive. This does apparently not happen on OpenIndiana or Solaris. Does anyone have a clue what could cause this bug in our ZFS port? URL:http://www.freebsd.org/cgi/query-pr.cgi?pr=157728 Thanks, mm -- Martin Matuska FreeBSD committer http://blog.vx.sk From owner-zfs-devel@FreeBSD.ORG Thu Jul 14 01:36:34 2011 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D2C9C1065674; Thu, 14 Jul 2011 01:36:34 +0000 (UTC) (envelope-from jhellenthal@gmail.com) Received: from mail-iy0-f182.google.com (mail-iy0-f182.google.com [209.85.210.182]) by mx1.freebsd.org (Postfix) with ESMTP id 873468FC0C; Thu, 14 Jul 2011 01:36:34 +0000 (UTC) Received: by iyb11 with SMTP id 11so7738816iyb.13 for ; Wed, 13 Jul 2011 18:36:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:in-reply-to; bh=2L6xlxMNHlWPcEgQbDnSh8lBkhgaQFMAp4ka3vvSVaE=; b=RohtWt5I0vOfZe06URLWMyeRVbgYepiMVvuQtUaTIcRjRCeEwEg2P5tGmNbtzJgusl mSpoHcHHi6nKieRXG8G/vCAKO5RzCXD7tZYiPYRZTiva4552ryucYnCJdhohJEZUwdJx e2v+gDNP4ET0SWtRXAlEpLwkEjGfwewUTn/YY= Received: by 10.42.221.4 with SMTP id ia4mr2065760icb.111.1310605499111; Wed, 13 Jul 2011 18:04:59 -0700 (PDT) Received: from DataIX.net (adsl-99-181-128-71.dsl.klmzmi.sbcglobal.net [99.181.128.71]) by mx.google.com with ESMTPS id q4sm93768ibb.66.2011.07.13.18.04.57 (version=TLSv1/SSLv3 cipher=OTHER); Wed, 13 Jul 2011 18:04:57 -0700 (PDT) Sender: "J. Hellenthal" Received: from DataIX.net (localhost [127.0.0.1]) by DataIX.net (8.14.5/8.14.5) with ESMTP id p6E14sUW078445 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 13 Jul 2011 21:04:55 -0400 (EDT) (envelope-from jhell@DataIX.net) Received: (from jhell@localhost) by DataIX.net (8.14.5/8.14.5/Submit) id p6E14sq5078444; Wed, 13 Jul 2011 21:04:54 -0400 (EDT) (envelope-from jhell@DataIX.net) Date: Wed, 13 Jul 2011 21:04:54 -0400 From: Jason Hellenthal To: Martin Matuska Message-ID: <20110714010454.GA73973@DataIX.net> References: <4E1C4AD8.4080101@FreeBSD.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="pWyiEgJYm5f9v55/" Content-Disposition: inline In-Reply-To: <4E1C4AD8.4080101@FreeBSD.org> Cc: zfs-devel@freebsd.org Subject: Re: Temporary clones not destroyed on incremental snapshot receive X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 14 Jul 2011 01:36:35 -0000 --pWyiEgJYm5f9v55/ Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Tue, Jul 12, 2011 at 03:23:36PM +0200, Martin Matuska wrote: > In kern/157728, I describe a problem (with reproduction steps and a > temporary workaround) of temporary clones not destroyed on incremental > snapshot receive. > This does apparently not happen on OpenIndiana or Solaris. >=20 > Does anyone have a clue what could cause this bug in our ZFS port? >=20 > URL:http://www.freebsd.org/cgi/query-pr.cgi?pr=3D157728 >=20 This bug may be related to a very similiar one that was reported to me last night. The user was attempting to rename a populated dataset with less than ~50 snapshots (maybe far less) did not get an exact number but his steps to reproduce this is listed below for reference. zfs umount pool/(...)/dataset # This fails, dataset somehow in use. zfs umount -f pool/(...)/dataset # This works, problem waiting here. zfs rename pool/(...)/dataset pool/(...)/dataset2 # Live-Lock There was no coredump at this point that could be obtained and the backtrace information he had was shaky at best. So ... I had him remove all the snapshots before dataset renaming... zfs destroy pool/(...)/dataset@(...) zfs umount pool/(...)/dataset # Not clear if this worked zfs umount -f pool/(...)/dataset # Could have worked. zfs rename pool/(...)/dataset pool/(...)/dataset2 # Worked. He probably could have gone without the (umount) here but I suspect thats why he was using it because the rename probably failed with EBUSY or some error. The backtrace he gave here: (pasted in IRC) translation needed. 25674 100871 zfs initial thread mi_switch+0x176 sleepq_wait+0x42 _cv_wait+0x129 txg_wait_synced+0x85 dsl_sync_task_group_wait+0x128 dsl_sync_task_do+0x5 4 dsl_dir_rename+0x8f dsl_dataset_rename+0x272 zfsdev_ioctl+0xe6 devfs_ioctl_f+0x7b kern_ioctl+0x102 ioctl+0xfd syscallenter+0x1e5 syscall+0x4b Xfast_syscall+0xe2 By the way he explained it, this felt like a locking problem or something not being free'd from something similiar to what we experienced before from the .zfs/* directories causing panics. Martin, He also explained that he was using a version of mfsBSD image to create this system too that he continued to run if its of any help. IIRC this is the same thing that was MFC'd to stable/8. --pWyiEgJYm5f9v55/ Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.17 (FreeBSD) Comment: http://bit.ly/0x89D8547E iQEcBAEBAgAGBQJOHkC2AAoJEJBXh4mJ2FR+itIH/0LmPtPD86RwYm6Je4FQ7CKB L/IVLiyZMMIHRqFITYqszRMs8w+6Y26RAAM/UTEpHZm7jLVhRH585av6ZaXDUUZz mcfs8sgePnXNmJVrEktoekoH3Dq22iTpX6nzd2ZX136hsTEBGChl0vB9Os2x/H3d /dTxRqJzWbllg2y2mAGdj+ItvTbkv/epYwHLc4zg4YGF1Guv6sLZUL9z/tCWQPiR fuBNEiFFXD512WsEpZYanCqmWcaK2y21Ad1OeT/C7rYQHUyrBmqghg7jGo7WtbBY /A8HuAdfxTkQWNiDcwlLgjcQhI25QDass24qxItwhgyJ2innDjJHNAMm1/Hju1w= =oI+Z -----END PGP SIGNATURE----- --pWyiEgJYm5f9v55/-- From owner-zfs-devel@FreeBSD.ORG Mon Jul 25 20:26:37 2011 Return-Path: Delivered-To: zfs-devel@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9CC951065674 for ; Mon, 25 Jul 2011 20:26:37 +0000 (UTC) (envelope-from delphij@delphij.net) Received: from anubis.delphij.net (anubis.delphij.net [IPv6:2001:470:1:117::25]) by mx1.freebsd.org (Postfix) with ESMTP id 8269B8FC1A for ; Mon, 25 Jul 2011 20:26:37 +0000 (UTC) Received: from delta.delphij.net (drawbridge.ixsystems.com [206.40.55.65]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by anubis.delphij.net (Postfix) with ESMTPSA id 4F35214310; Mon, 25 Jul 2011 13:26:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=delphij.net; s=anubis; t=1311625597; bh=K+/6FOxa1iitLik5Rj27a5CYymkNXydcmfDVaYM1rd4=; h=Message-ID:Date:From:Reply-To:MIME-Version:To:Subject: Content-Type; b=zpJFDqpKz0+xXttitcTanLvnaaTaFmqn5bkQAEd3/XXo/iXERODlG6OHB8a0+p5Yz jOD8EvpuXcMrHFncNwbVb4VBkj413vppwPyOdwKwVd7WlCKYx9UOtvlIcK2SCaQSqo dXIK6dRY21nNCJ4x6Mj7CDb3TpbdBEGpDFPkTCkA= Message-ID: <4E2DD17C.60606@delphij.net> Date: Mon, 25 Jul 2011 13:26:36 -0700 From: Xin LI Organization: The FreeBSD Project MIME-Version: 1.0 To: zfs-devel@FreeBSD.ORG OpenPGP: id=3FCA37C1; url=http://www.delphij.net/delphij.asc Content-Type: multipart/mixed; boundary="------------060708020906060603030803" Cc: Subject: [PATCH] Consistently use zfs_ioctl() X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: d@delphij.net List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 25 Jul 2011 20:26:37 -0000 This is a multi-part message in MIME format. --------------060708020906060603030803 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 Hi, Here is a patch that makes libzfs use zfs_ioctl consistently. Some functionality in libzfs calls ioctl() directly with raw ZFS ioctls, which causes some kernel message like: WARNING pid 30031 (initial thread): ioctl sign-extension ioctl ffffffffd5985a3a Objections? I have kept #sun'ed out blocks out. Cheers, - -- Xin LI https://www.delphij.net/ FreeBSD - The Power to Serve! Live free or die -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.17 (FreeBSD) iQEcBAEBCAAGBQJOLdF8AAoJEATO+BI/yjfBI0AH/04tS03WW0vIW/ywXp/FsHV6 bkkZQw7rjN2dnenHBTg3DNcjYlXpZOgRb/mnuwMsTxm/Xf/RVekfCLGoljwv7NNK FUTozx7jVBvwAhjArvUTsVjpblnm2b+mS9TCWh9BWVhIr5N6sMALZxZkNvVTUVn+ ZXotIaqQPSVNdEC3BuvJsxyHAEutTPF8TvMojKhaRhxTPGGcrjQfVLHHR/YMLhY5 ED/7gJsvSTAi+gmtHWgF9q5oqN9sCE/Juy5XD0HcXok5dZ2cfjYYFacKbbjpXQhn eYj8zSVPna0avCJ0YEivEO5LCEF9MLgTMGFJtV38mRandNdbU5UFNqA+m4WJeiM= =zWDs -----END PGP SIGNATURE----- --------------060708020906060603030803 Content-Type: text/plain; name="zfs-ioctl.diff" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="zfs-ioctl.diff" SW5kZXg6IGNkZGwvY29udHJpYi9vcGVuc29sYXJpcy9saWIvbGliemZzL2NvbW1vbi9saWJ6 ZnNfZGF0YXNldC5jCj09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT0KLS0tIGNkZGwvY29udHJpYi9vcGVuc29sYXJp cy9saWIvbGliemZzL2NvbW1vbi9saWJ6ZnNfZGF0YXNldC5jCShyZXZpc2lvbiAyMjQzMzcp CisrKyBjZGRsL2NvbnRyaWIvb3BlbnNvbGFyaXMvbGliL2xpYnpmcy9jb21tb24vbGliemZz X2RhdGFzZXQuYwkod29ya2luZyBjb3B5KQpAQCAtMjM4Miw2ICsyMzgyLDcgQEAgemZzX3By b3BfZ2V0X3VzZXJxdW90YV9jb21tb24oemZzX2hhbmRsZV90ICp6aHAsIGMKIHsKIAlpbnQg ZXJyOwogCXpmc19jbWRfdCB6YyA9IHsgMCB9OworCWxpYnpmc19oYW5kbGVfdCAqaGRsID0g emhwLT56ZnNfaGRsOwogCiAJKHZvaWQpIHN0cm5jcHkoemMuemNfbmFtZSwgemhwLT56ZnNf bmFtZSwgc2l6ZW9mICh6Yy56Y19uYW1lKSk7CiAKQEAgLTIzOTIsNyArMjM5Myw3IEBAIHpm c19wcm9wX2dldF91c2VycXVvdGFfY29tbW9uKHpmc19oYW5kbGVfdCAqemhwLCBjCiAJaWYg KGVycikKIAkJcmV0dXJuIChlcnIpOwogCi0JZXJyID0gaW9jdGwoemhwLT56ZnNfaGRsLT5s aWJ6ZnNfZmQsIFpGU19JT0NfVVNFUlNQQUNFX09ORSwgJnpjKTsKKwllcnIgPSB6ZnNfaW9j dGwoaGRsLCBaRlNfSU9DX1VTRVJTUEFDRV9PTkUsICZ6Yyk7CiAJaWYgKGVycikKIAkJcmV0 dXJuIChlcnIpOwogCkBAIC0yNDU4LDExICsyNDU5LDEyIEBAIHpmc19kb19saXN0X2lvY3Rs KHpmc19oYW5kbGVfdCAqemhwLCB1bnNpZ25lZCBsb25nCiB7CiAJaW50IHJjOwogCXVpbnQ2 NF90CW9yaWdfY29va2llOworCWxpYnpmc19oYW5kbGVfdCAqaGRsID0gemhwLT56ZnNfaGRs OwogCiAJb3JpZ19jb29raWUgPSB6Yy0+emNfY29va2llOwogdG9wOgogCSh2b2lkKSBzdHJs Y3B5KHpjLT56Y19uYW1lLCB6aHAtPnpmc19uYW1lLCBzaXplb2YgKHpjLT56Y19uYW1lKSk7 Ci0JcmMgPSBpb2N0bCh6aHAtPnpmc19oZGwtPmxpYnpmc19mZCwgYXJnLCB6Yyk7CisJcmMg PSB6ZnNfaW9jdGwoaGRsLCBhcmcsIHpjKTsKIAogCWlmIChyYyA9PSAtMSkgewogCQlzd2l0 Y2ggKGVycm5vKSB7CkBAIC0zODAwLDcgKzM4MDIsNyBAQCB6ZnNfZGVsZWdfc2hhcmVfbmZz KGxpYnpmc19oYW5kbGVfdCAqaGRsLCBjaGFyICpkYQogCXpjLnpjX3NoYXJlLnpfZXhwb3J0 ZGF0YSA9ICh1aW50NjRfdCkodWludHB0cl90KWV4cG9ydDsKIAl6Yy56Y19zaGFyZS56X3No YXJldHlwZSA9IG9wZXJhdGlvbjsKIAl6Yy56Y19zaGFyZS56X3NoYXJlbWF4ID0gc2hhcmVt YXg7Ci0JZXJyb3IgPSBpb2N0bChoZGwtPmxpYnpmc19mZCwgWkZTX0lPQ19TSEFSRSwgJnpj KTsKKwllcnJvciA9IHpmc19pb2N0bChoZGwsIFpGU19JT0NfU0hBUkUsICZ6Yyk7CiAJcmV0 dXJuIChlcnJvcik7CiB9CiAKQEAgLTM5MjQsNiArMzkyNiw3IEBAIHpmc191c2Vyc3BhY2Uo emZzX2hhbmRsZV90ICp6aHAsIHpmc191c2VycXVvdGFfcHJvCiAgICAgemZzX3VzZXJzcGFj ZV9jYl90IGZ1bmMsIHZvaWQgKmFyZykKIHsKIAl6ZnNfY21kX3QgemMgPSB7IDAgfTsKKwls aWJ6ZnNfaGFuZGxlX3QgKmhkbCA9IHpocC0+emZzX2hkbDsKIAlpbnQgZXJyb3I7CiAJemZz X3VzZXJhY2N0X3QgYnVmWzEwMF07CiAKQEAgLTM5MzcsNyArMzk0MCw3IEBAIHpmc191c2Vy c3BhY2UoemZzX2hhbmRsZV90ICp6aHAsIHpmc191c2VycXVvdGFfcHJvCiAJCXpmc191c2Vy YWNjdF90ICp6dWEgPSBidWY7CiAKIAkJemMuemNfbnZsaXN0X2RzdF9zaXplID0gc2l6ZW9m IChidWYpOwotCQllcnJvciA9IGlvY3RsKHpocC0+emZzX2hkbC0+bGliemZzX2ZkLAorCQll cnJvciA9IHpmc19pb2N0bChoZGwsCiAJCSAgICBaRlNfSU9DX1VTRVJTUEFDRV9NQU5ZLCAm emMpOwogCQlpZiAoZXJyb3IgfHwgemMuemNfbnZsaXN0X2RzdF9zaXplID09IDApCiAJCQli cmVhazsKQEAgLTQzMTYsNyArNDMxOSw3IEBAIHpmc19qYWlsKHpmc19oYW5kbGVfdCAqemhw LCBpbnQgamFpbGlkLCBpbnQgYXR0YWNoCiAJemMuemNfamFpbGlkID0gamFpbGlkOwogCiAJ Y21kID0gYXR0YWNoID8gWkZTX0lPQ19KQUlMIDogWkZTX0lPQ19VTkpBSUw7Ci0JaWYgKChy ZXQgPSBpb2N0bChoZGwtPmxpYnpmc19mZCwgY21kLCAmemMpKSAhPSAwKQorCWlmICgocmV0 ID0gemZzX2lvY3RsKGhkbCwgY21kLCAmemMpKSAhPSAwKQogCQl6ZnNfc3RhbmRhcmRfZXJy b3IoaGRsLCBlcnJubywgZXJyYnVmKTsKIAogCXJldHVybiAocmV0KTsK --------------060708020906060603030803-- From owner-zfs-devel@FreeBSD.ORG Tue Jul 26 12:31:40 2011 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5F31C106566B; Tue, 26 Jul 2011 12:31:40 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from mail.vx.sk (mail.vx.sk [IPv6:2a01:4f8:100:1043::3]) by mx1.freebsd.org (Postfix) with ESMTP id B20778FC16; Tue, 26 Jul 2011 12:31:39 +0000 (UTC) Received: from core.vx.sk (localhost [127.0.0.1]) by mail.vx.sk (Postfix) with ESMTP id DECF4167197; Tue, 26 Jul 2011 14:31:38 +0200 (CEST) X-Virus-Scanned: amavisd-new at mail.vx.sk Received: from mail.vx.sk ([127.0.0.1]) by core.vx.sk (mail.vx.sk [127.0.0.1]) (amavisd-new, port 10024) with LMTP id DuQ-4GajIGZY; Tue, 26 Jul 2011 14:31:33 +0200 (CEST) Received: from [10.0.3.3] (188-167-66-148.dynamic.chello.sk [188.167.66.148]) by mail.vx.sk (Postfix) with ESMTPSA id 46661167187; Tue, 26 Jul 2011 14:31:33 +0200 (CEST) Message-ID: <4E2EB3A4.60007@FreeBSD.org> Date: Tue, 26 Jul 2011 14:31:32 +0200 From: Martin Matuska User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:5.0) Gecko/20110627 Thunderbird/5.0 MIME-Version: 1.0 To: Pawel Jakub Dawidek , Xin Li X-Priority: 2 (High) X-Enigmail-Version: 1.2pre Content-Type: multipart/mixed; boundary="------------020005050609060404020702" Cc: zfs-devel@FreeBSD.org Subject: [PATCH] Illumos #883 ZIL reuse during remount can lead to data corruption X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 26 Jul 2011 12:31:40 -0000 This is a multi-part message in MIME format. --------------020005050609060404020702 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Illumos developers have fixed a problem in ZIL that may lead to data corruption during remount. A detailed description of this problem is available at: https://www.illumos.org/issues/883 In my opinion this issue applies to FreeBSD, too, and we should fix this ASAP. Please review my patch (1:1 merge from illumos, no changes). References: illumos-gate revision: 13380:161b964a0e10 https://github.com/illumos/illumos-gate/commit/c9ba2a43cb76c223d115e021fdabd2c066e020ed -- Martin Matuska FreeBSD committer http://blog.vx.sk --------------020005050609060404020702 Content-Type: text/x-patch; name="illumos-zil.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="illumos-zil.patch" Index: cddl/contrib/opensolaris/cmd/ztest/ztest.c =================================================================== --- cddl/contrib/opensolaris/cmd/ztest/ztest.c (revision 224409) +++ cddl/contrib/opensolaris/cmd/ztest/ztest.c (working copy) @@ -205,6 +205,7 @@ */ typedef struct ztest_ds { objset_t *zd_os; + rwlock_t zd_zilog_lock; zilog_t *zd_zilog; uint64_t zd_seq; ztest_od_t *zd_od; /* debugging aid */ @@ -238,6 +239,7 @@ ztest_func_t ztest_zap; ztest_func_t ztest_zap_parallel; ztest_func_t ztest_zil_commit; +ztest_func_t ztest_zil_remount; ztest_func_t ztest_dmu_read_write_zcopy; ztest_func_t ztest_dmu_objset_create_destroy; ztest_func_t ztest_dmu_prealloc; @@ -273,6 +275,7 @@ { ztest_zap_parallel, 100, &zopt_always }, { ztest_split_pool, 1, &zopt_always }, { ztest_zil_commit, 1, &zopt_incessant }, + { ztest_zil_remount, 1, &zopt_sometimes }, { ztest_dmu_read_write_zcopy, 1, &zopt_often }, { ztest_dmu_objset_create_destroy, 1, &zopt_often }, { ztest_dsl_prop_get_set, 1, &zopt_often }, @@ -986,6 +989,7 @@ zd->zd_seq = 0; dmu_objset_name(os, zd->zd_name); + VERIFY(rwlock_init(&zd->zd_zilog_lock, USYNC_THREAD, NULL) == 0); VERIFY(_mutex_init(&zd->zd_dirobj_lock, USYNC_THREAD, NULL) == 0); for (int l = 0; l < ZTEST_OBJECT_LOCKS; l++) @@ -1965,6 +1969,8 @@ if (ztest_random(2) == 0) io_type = ZTEST_IO_WRITE_TAG; + (void) rw_rdlock(&zd->zd_zilog_lock); + switch (io_type) { case ZTEST_IO_WRITE_TAG: @@ -2000,6 +2006,8 @@ break; } + (void) rw_unlock(&zd->zd_zilog_lock); + umem_free(data, blocksize); } @@ -2054,6 +2062,8 @@ { zilog_t *zilog = zd->zd_zilog; + (void) rw_rdlock(&zd->zd_zilog_lock); + zil_commit(zilog, ztest_random(ZTEST_OBJECTS)); /* @@ -2065,9 +2075,34 @@ ASSERT(zd->zd_seq <= zilog->zl_commit_lr_seq); zd->zd_seq = zilog->zl_commit_lr_seq; mutex_exit(&zilog->zl_lock); + + (void) rw_unlock(&zd->zd_zilog_lock); } /* + * This function is designed to simulate the operations that occur during a + * mount/unmount operation. We hold the dataset across these operations in an + * attempt to expose any implicit assumptions about ZIL management. + */ +/* ARGSUSED */ +void +ztest_zil_remount(ztest_ds_t *zd, uint64_t id) +{ + objset_t *os = zd->zd_os; + + (void) rw_wrlock(&zd->zd_zilog_lock); + + /* zfsvfs_teardown() */ + zil_close(zd->zd_zilog); + + /* zfsvfs_setup() */ + VERIFY(zil_open(os, ztest_get_data) == zd->zd_zilog); + zil_replay(os, zd, ztest_replay_vector); + + (void) rw_unlock(&zd->zd_zilog_lock); +} + +/* * Verify that we can't destroy an active pool, create an existing pool, * or create a pool with a bad vdev spec. */ Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zil.c =================================================================== --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zil.c (revision 224409) +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zil.c (working copy) @@ -20,6 +20,7 @@ */ /* * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 2011 by Delphix. All rights reserved. */ /* Portions Copyright 2010 Robert Milkowski */ @@ -567,7 +568,7 @@ if (!list_is_empty(&zilog->zl_lwb_list)) { ASSERT(zh->zh_claim_txg == 0); - ASSERT(!keep_first); + VERIFY(!keep_first); while ((lwb = list_head(&zilog->zl_lwb_list)) != NULL) { list_remove(&zilog->zl_lwb_list, lwb); if (lwb->lwb_buf != NULL) @@ -1668,20 +1669,9 @@ void zil_free(zilog_t *zilog) { - lwb_t *head_lwb; - zilog->zl_stop_sync = 1; - /* - * After zil_close() there should only be one lwb with a buffer. - */ - head_lwb = list_head(&zilog->zl_lwb_list); - if (head_lwb) { - ASSERT(head_lwb == list_tail(&zilog->zl_lwb_list)); - list_remove(&zilog->zl_lwb_list, head_lwb); - zio_buf_free(head_lwb->lwb_buf, head_lwb->lwb_sz); - kmem_cache_free(zil_lwb_cache, head_lwb); - } + ASSERT(list_is_empty(&zilog->zl_lwb_list)); list_destroy(&zilog->zl_lwb_list); avl_destroy(&zilog->zl_vdev_tree); @@ -1721,6 +1711,10 @@ { zilog_t *zilog = dmu_objset_zil(os); + ASSERT(zilog->zl_clean_taskq == NULL); + ASSERT(zilog->zl_get_data == NULL); + ASSERT(list_is_empty(&zilog->zl_lwb_list)); + zilog->zl_get_data = get_data; zilog->zl_clean_taskq = taskq_create("zil_clean", 1, minclsyspri, 2, 2, TASKQ_PREPOPULATE); @@ -1734,7 +1728,7 @@ void zil_close(zilog_t *zilog) { - lwb_t *tail_lwb; + lwb_t *lwb; uint64_t txg = 0; zil_commit(zilog, 0); /* commit all itx */ @@ -1746,9 +1740,9 @@ * destroy the zl_clean_taskq. */ mutex_enter(&zilog->zl_lock); - tail_lwb = list_tail(&zilog->zl_lwb_list); - if (tail_lwb != NULL) - txg = tail_lwb->lwb_max_txg; + lwb = list_tail(&zilog->zl_lwb_list); + if (lwb != NULL) + txg = lwb->lwb_max_txg; mutex_exit(&zilog->zl_lock); if (txg) txg_wait_synced(zilog->zl_dmu_pool, txg); @@ -1756,6 +1750,19 @@ taskq_destroy(zilog->zl_clean_taskq); zilog->zl_clean_taskq = NULL; zilog->zl_get_data = NULL; + + /* + * We should have only one LWB left on the list; remove it now. + */ + mutex_enter(&zilog->zl_lock); + lwb = list_head(&zilog->zl_lwb_list); + if (lwb != NULL) { + ASSERT(lwb == list_tail(&zilog->zl_lwb_list)); + list_remove(&zilog->zl_lwb_list, lwb); + zio_buf_free(lwb->lwb_buf, lwb->lwb_sz); + kmem_cache_free(zil_lwb_cache, lwb); + } + mutex_exit(&zilog->zl_lock); } /* --------------020005050609060404020702-- From owner-zfs-devel@FreeBSD.ORG Tue Jul 26 12:34:37 2011 Return-Path: Delivered-To: zfs-devel@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0B83D1065670 for ; Tue, 26 Jul 2011 12:34:37 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from mail.vx.sk (mail.vx.sk [IPv6:2a01:4f8:100:1043::3]) by mx1.freebsd.org (Postfix) with ESMTP id BFF158FC0C for ; Tue, 26 Jul 2011 12:34:36 +0000 (UTC) Received: from core.vx.sk (localhost [127.0.0.1]) by mail.vx.sk (Postfix) with ESMTP id D1018167328; Tue, 26 Jul 2011 14:34:35 +0200 (CEST) X-Virus-Scanned: amavisd-new at mail.vx.sk Received: from mail.vx.sk ([127.0.0.1]) by core.vx.sk (mail.vx.sk [127.0.0.1]) (amavisd-new, port 10024) with LMTP id MwOB66KoK1Hg; Tue, 26 Jul 2011 14:34:33 +0200 (CEST) Received: from [10.0.3.3] (188-167-66-148.dynamic.chello.sk [188.167.66.148]) by mail.vx.sk (Postfix) with ESMTPSA id F2413167316; Tue, 26 Jul 2011 14:34:32 +0200 (CEST) Message-ID: <4E2EB457.4000706@FreeBSD.org> Date: Tue, 26 Jul 2011 14:34:31 +0200 From: Martin Matuska User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:5.0) Gecko/20110627 Thunderbird/5.0 MIME-Version: 1.0 To: d@delphij.net References: <4E2DD17C.60606@delphij.net> In-Reply-To: <4E2DD17C.60606@delphij.net> X-Enigmail-Version: 1.2pre Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: zfs-devel@FreeBSD.ORG Subject: Re: [PATCH] Consistently use zfs_ioctl() X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 26 Jul 2011 12:34:37 -0000 If it is also sucessfully tested on sparc64 (and/or powerpc64) I am ok with this patch. Dňa 25.07.2011 22:26, Xin LI wrote / napísal(a): > Hi, > > Here is a patch that makes libzfs use zfs_ioctl consistently. Some > functionality in libzfs calls ioctl() directly with raw ZFS ioctls, > which causes some kernel message like: > > WARNING pid 30031 (initial thread): ioctl sign-extension ioctl > ffffffffd5985a3a > > Objections? I have kept #sun'ed out blocks out. > > Cheers, > > > _______________________________________________ > zfs-devel@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/zfs-devel > To unsubscribe, send any mail to "zfs-devel-unsubscribe@freebsd.org" -- Martin Matuska FreeBSD committer http://blog.vx.sk From owner-zfs-devel@FreeBSD.ORG Tue Jul 26 13:21:21 2011 Return-Path: Delivered-To: zfs-devel@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id F296D106564A for ; Tue, 26 Jul 2011 13:21:20 +0000 (UTC) (envelope-from pawel@dawidek.net) Received: from mail.dawidek.net (60.wheelsystems.com [83.12.187.60]) by mx1.freebsd.org (Postfix) with ESMTP id A59DF8FC0A for ; Tue, 26 Jul 2011 13:21:20 +0000 (UTC) Received: from localhost (58.wheelsystems.com [83.12.187.58]) by mail.dawidek.net (Postfix) with ESMTPSA id 5F30962B; Tue, 26 Jul 2011 15:21:18 +0200 (CEST) Date: Tue, 26 Jul 2011 15:21:16 +0200 From: Pawel Jakub Dawidek To: d@delphij.net Message-ID: <20110726132116.GC1656@garage.freebsd.pl> References: <4E2DD17C.60606@delphij.net> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="4ZLFUWh1odzi/v6L" Content-Disposition: inline In-Reply-To: <4E2DD17C.60606@delphij.net> X-OS: FreeBSD 9.0-CURRENT amd64 User-Agent: Mutt/1.5.21 (2010-09-15) Cc: zfs-devel@FreeBSD.ORG Subject: Re: [PATCH] Consistently use zfs_ioctl() X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 26 Jul 2011 13:21:21 -0000 --4ZLFUWh1odzi/v6L Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Mon, Jul 25, 2011 at 01:26:36PM -0700, Xin LI wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA256 >=20 > Hi, >=20 > Here is a patch that makes libzfs use zfs_ioctl consistently. Some > functionality in libzfs calls ioctl() directly with raw ZFS ioctls, > which causes some kernel message like: >=20 > WARNING pid 30031 (initial thread): ioctl sign-extension ioctl > ffffffffd5985a3a >=20 > Objections? I have kept #sun'ed out blocks out. zfs_ioctl() does something with history records, I think. Are you sure zfs_ioctl() is 1-to-1 equivalent of plain ioctl()? Some minor nits inline. > Index: cddl/contrib/opensolaris/lib/libzfs/common/libzfs_dataset.c > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > --- cddl/contrib/opensolaris/lib/libzfs/common/libzfs_dataset.c (revision= 224337) > +++ cddl/contrib/opensolaris/lib/libzfs/common/libzfs_dataset.c (working = copy) > @@ -2382,6 +2382,7 @@ zfs_prop_get_userquota_common(zfs_handle_t *zhp, c > { > int err; > zfs_cmd_t zc =3D { 0 }; > + libzfs_handle_t *hdl =3D zhp->zfs_hdl; > =20 > (void) strncpy(zc.zc_name, zhp->zfs_name, sizeof (zc.zc_name)); > =20 > @@ -2392,7 +2393,7 @@ zfs_prop_get_userquota_common(zfs_handle_t *zhp, c > if (err) > return (err); > =20 > - err =3D ioctl(zhp->zfs_hdl->libzfs_fd, ZFS_IOC_USERSPACE_ONE, &zc); > + err =3D zfs_ioctl(hdl, ZFS_IOC_USERSPACE_ONE, &zc); Do we really need additional variable that is used only once? Wouldn't it be better to just replace ioctl(zhp->zfs_hdl->libzfs_fd) with zfs_ioctl(zhp->zfs_hdl)? > @@ -2458,11 +2459,12 @@ zfs_do_list_ioctl(zfs_handle_t *zhp, unsigned long > { > int rc; > uint64_t orig_cookie; > + libzfs_handle_t *hdl =3D zhp->zfs_hdl; > =20 > orig_cookie =3D zc->zc_cookie; > top: > (void) strlcpy(zc->zc_name, zhp->zfs_name, sizeof (zc->zc_name)); > - rc =3D ioctl(zhp->zfs_hdl->libzfs_fd, arg, zc); > + rc =3D zfs_ioctl(hdl, arg, zc); Same here. > @@ -3924,6 +3926,7 @@ zfs_userspace(zfs_handle_t *zhp, zfs_userquota_pro > zfs_userspace_cb_t func, void *arg) > { > zfs_cmd_t zc =3D { 0 }; > + libzfs_handle_t *hdl =3D zhp->zfs_hdl; > int error; > zfs_useracct_t buf[100]; > =20 > @@ -3937,7 +3940,7 @@ zfs_userspace(zfs_handle_t *zhp, zfs_userquota_pro > zfs_useracct_t *zua =3D buf; > =20 > zc.zc_nvlist_dst_size =3D sizeof (buf); > - error =3D ioctl(zhp->zfs_hdl->libzfs_fd, > + error =3D zfs_ioctl(hdl, > ZFS_IOC_USERSPACE_MANY, &zc); And here. --=20 Pawel Jakub Dawidek http://www.wheelsystems.com FreeBSD committer http://www.FreeBSD.org Am I Evil? Yes, I Am! http://yomoli.com --4ZLFUWh1odzi/v6L Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (FreeBSD) iEYEARECAAYFAk4uv0wACgkQForvXbEpPzTlGgCg7WXprmBbYNaMkeVpLEA9vWXY g7YAnitMSxnr+IHKHNYHvJOniEIEiiZH =cK5y -----END PGP SIGNATURE----- --4ZLFUWh1odzi/v6L-- From owner-zfs-devel@FreeBSD.ORG Tue Jul 26 20:42:58 2011 Return-Path: Delivered-To: zfs-devel@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 26787106564A; Tue, 26 Jul 2011 20:42:58 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from mail.vx.sk (mail.vx.sk [IPv6:2a01:4f8:100:1043::3]) by mx1.freebsd.org (Postfix) with ESMTP id 5C1538FC12; Tue, 26 Jul 2011 20:42:57 +0000 (UTC) Received: from core.vx.sk (localhost [127.0.0.1]) by mail.vx.sk (Postfix) with ESMTP id 334F3165FCD; Tue, 26 Jul 2011 22:42:56 +0200 (CEST) X-Virus-Scanned: amavisd-new at mail.vx.sk Received: from mail.vx.sk ([127.0.0.1]) by core.vx.sk (mail.vx.sk [127.0.0.1]) (amavisd-new, port 10024) with LMTP id FO7mVGG6Mb1Y; Tue, 26 Jul 2011 22:42:52 +0200 (CEST) Received: from [10.9.8.1] (chello085216231078.chello.sk [85.216.231.78]) by mail.vx.sk (Postfix) with ESMTPSA id D702C165FB8; Tue, 26 Jul 2011 22:42:51 +0200 (CEST) Message-ID: <4E2F26CD.30801@FreeBSD.org> Date: Tue, 26 Jul 2011 22:42:53 +0200 From: Martin Matuska User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:5.0) Gecko/20110624 Thunderbird/5.0 MIME-Version: 1.0 To: Pawel Jakub Dawidek References: <4E2DD17C.60606@delphij.net> <20110726132116.GC1656@garage.freebsd.pl> In-Reply-To: <20110726132116.GC1656@garage.freebsd.pl> X-Enigmail-Version: 1.2 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 8bit X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: zfs-devel@FreeBSD.ORG, d@delphij.net Subject: Re: [PATCH] Consistently use zfs_ioctl() X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 26 Jul 2011 20:42:58 -0000 Dn(a 26. 7. 2011 15:21, Pawel Jakub Dawidek wrote / napsal(a): > On Mon, Jul 25, 2011 at 01:26:36PM -0700, Xin LI wrote: >> -----BEGIN PGP SIGNED MESSAGE----- >> Hash: SHA256 >> >> Hi, >> >> Here is a patch that makes libzfs use zfs_ioctl consistently. Some >> functionality in libzfs calls ioctl() directly with raw ZFS ioctls, >> which causes some kernel message like: >> >> WARNING pid 30031 (initial thread): ioctl sign-extension ioctl >> ffffffffd5985a3a >> >> Objections? I have kept #sun'ed out blocks out. > > zfs_ioctl() does something with history records, I think. Are you sure > zfs_ioctl() is 1-to-1 equivalent of plain ioctl()? > > Some minor nits inline. Yes, zfs_ioctl deals with history and calls ioctl the very same way: cddl/contrib/opensolaris/lib/libzfs/common/libzfs_util.c: int zfs_ioctl(libzfs_handle_t *hdl, unsigned long request, zfs_cmd_t *zc) { int error; zc->zc_history = (uint64_t)(uintptr_t)hdl->libzfs_log_str; error = ioctl(hdl->libzfs_fd, request, zc); if (hdl->libzfs_log_str) { free(hdl->libzfs_log_str); hdl->libzfs_log_str = NULL; } zc->zc_history = 0; return (error); } Now to ioctl(): we are not calling ioctl directly. Ioctl() is actually zcmd_ioctl() - see bottom of the following code: cddl/contrib/opensolaris/lib/libzfs/common/libzfs_impl.h: static int zfs_kernel_version = 0; /* * This is FreeBSD version of ioctl, because Solaris' ioctl() updates * zc_nvlist_dst_size even if an error is returned, on FreeBSD if an * error is returned zc_nvlist_dst_size won't be updated. */ static __inline int zcmd_ioctl(int fd, unsigned long cmd, zfs_cmd_t *zc) { size_t oldsize, zfs_kernel_version_size; int version, ret, cflag = ZFS_CMD_COMPAT_NONE; zfs_kernel_version_size = sizeof(zfs_kernel_version); if (zfs_kernel_version == 0) { sysctlbyname("vfs.zfs.version.spa", &zfs_kernel_version, &zfs_kernel_version_size, NULL, 0); } if (zfs_kernel_version == SPA_VERSION_15 || zfs_kernel_version == SPA_VERSION_14 || zfs_kernel_version == SPA_VERSION_13) cflag = ZFS_CMD_COMPAT_V15; oldsize = zc->zc_nvlist_dst_size; ret = zcmd_ioctl_compat(fd, cmd, zc, cflag); if (ret == 0 && oldsize < zc->zc_nvlist_dst_size) { ret = -1; errno = ENOMEM; } return (ret); } #define ioctl(fd, cmd, zc) zcmd_ioctl((fd), (cmd), (zc)) This function calls my zcmd_ioctl_compat in: sys/cddl/contrib/opensolaris/common/zfs/zfs_ioctl_compat.c Probably there is a problem in this function (my compat code)? But for ZFS_CMD_COMPAT_NONE it is just passed to the kernel as it was before. > > >> Index: cddl/contrib/opensolaris/lib/libzfs/common/libzfs_dataset.c >> =================================================================== >> --- cddl/contrib/opensolaris/lib/libzfs/common/libzfs_dataset.c (revision 224337) >> +++ cddl/contrib/opensolaris/lib/libzfs/common/libzfs_dataset.c (working copy) >> @@ -2382,6 +2382,7 @@ zfs_prop_get_userquota_common(zfs_handle_t *zhp, c >> { >> int err; >> zfs_cmd_t zc = { 0 }; >> + libzfs_handle_t *hdl = zhp->zfs_hdl; >> >> (void) strncpy(zc.zc_name, zhp->zfs_name, sizeof (zc.zc_name)); >> >> @@ -2392,7 +2393,7 @@ zfs_prop_get_userquota_common(zfs_handle_t *zhp, c >> if (err) >> return (err); >> >> - err = ioctl(zhp->zfs_hdl->libzfs_fd, ZFS_IOC_USERSPACE_ONE, &zc); >> + err = zfs_ioctl(hdl, ZFS_IOC_USERSPACE_ONE, &zc); > > Do we really need additional variable that is used only once? > Wouldn't it be better to just replace ioctl(zhp->zfs_hdl->libzfs_fd) > with zfs_ioctl(zhp->zfs_hdl)? I agree, we don't and just for one use we waste resources. > >> @@ -2458,11 +2459,12 @@ zfs_do_list_ioctl(zfs_handle_t *zhp, unsigned long >> { >> int rc; >> uint64_t orig_cookie; >> + libzfs_handle_t *hdl = zhp->zfs_hdl; >> >> orig_cookie = zc->zc_cookie; >> top: >> (void) strlcpy(zc->zc_name, zhp->zfs_name, sizeof (zc->zc_name)); >> - rc = ioctl(zhp->zfs_hdl->libzfs_fd, arg, zc); >> + rc = zfs_ioctl(hdl, arg, zc); > > Same here. > >> @@ -3924,6 +3926,7 @@ zfs_userspace(zfs_handle_t *zhp, zfs_userquota_pro >> zfs_userspace_cb_t func, void *arg) >> { >> zfs_cmd_t zc = { 0 }; >> + libzfs_handle_t *hdl = zhp->zfs_hdl; >> int error; >> zfs_useracct_t buf[100]; >> >> @@ -3937,7 +3940,7 @@ zfs_userspace(zfs_handle_t *zhp, zfs_userquota_pro >> zfs_useracct_t *zua = buf; >> >> zc.zc_nvlist_dst_size = sizeof (buf); >> - error = ioctl(zhp->zfs_hdl->libzfs_fd, >> + error = zfs_ioctl(hdl, >> ZFS_IOC_USERSPACE_MANY, &zc); > > And here. > -- Martin Matuska FreeBSD committer http://blog.vx.sk From owner-zfs-devel@FreeBSD.ORG Tue Jul 26 20:55:13 2011 Return-Path: Delivered-To: zfs-devel@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 48A6B106564A for ; Tue, 26 Jul 2011 20:55:13 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from mail.vx.sk (mail.vx.sk [IPv6:2a01:4f8:100:1043::3]) by mx1.freebsd.org (Postfix) with ESMTP id F35448FC13 for ; Tue, 26 Jul 2011 20:55:12 +0000 (UTC) Received: from core.vx.sk (localhost [127.0.0.1]) by mail.vx.sk (Postfix) with ESMTP id 546F2183602; Tue, 26 Jul 2011 22:55:12 +0200 (CEST) X-Virus-Scanned: amavisd-new at mail.vx.sk Received: from mail.vx.sk ([127.0.0.1]) by core.vx.sk (mail.vx.sk [127.0.0.1]) (amavisd-new, port 10024) with LMTP id qiMBjk0DMulz; Tue, 26 Jul 2011 22:55:10 +0200 (CEST) Received: from [10.9.8.1] (chello085216231078.chello.sk [85.216.231.78]) by mail.vx.sk (Postfix) with ESMTPSA id 015051835F1; Tue, 26 Jul 2011 22:55:09 +0200 (CEST) Message-ID: <4E2F29AF.5030905@FreeBSD.org> Date: Tue, 26 Jul 2011 22:55:11 +0200 From: Martin Matuska User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:5.0) Gecko/20110624 Thunderbird/5.0 MIME-Version: 1.0 To: d@delphij.net References: <4E2DD17C.60606@delphij.net> In-Reply-To: <4E2DD17C.60606@delphij.net> X-Enigmail-Version: 1.2 Content-Type: multipart/mixed; boundary="------------030604010008080804060700" Cc: zfs-devel@FreeBSD.ORG Subject: Re: [PATCH] Consistently use zfs_ioctl() X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 26 Jul 2011 20:55:13 -0000 This is a multi-part message in MIME format. --------------030604010008080804060700 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit I have chatted with delphij@ and he said his problem was in zfs jail / unjail. The problem is in zfs_jail(), cmd passed to ioctl() is initialized as int instead of unsigned long. Patch suggested for fixing this problem is attached. -- Martin Matuska FreeBSD committer http://blog.vx.sk --------------030604010008080804060700 Content-Type: text/plain; name="libzfs_dataset.c.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="libzfs_dataset.c.patch" Index: cddl/contrib/opensolaris/lib/libzfs/common/libzfs_dataset.c =================================================================== --- cddl/contrib/opensolaris/lib/libzfs/common/libzfs_dataset.c (revision 224409) +++ cddl/contrib/opensolaris/lib/libzfs/common/libzfs_dataset.c (working copy) @@ -4289,7 +4289,8 @@ libzfs_handle_t *hdl = zhp->zfs_hdl; zfs_cmd_t zc = { 0 }; char errbuf[1024]; - int cmd, ret; + unsigned long cmd; + int ret; if (attach) { (void) snprintf(errbuf, sizeof (errbuf), --------------030604010008080804060700-- From owner-zfs-devel@FreeBSD.ORG Wed Jul 27 05:34:52 2011 Return-Path: Delivered-To: zfs-devel@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1BC26106564A; Wed, 27 Jul 2011 05:34:52 +0000 (UTC) (envelope-from pawel@dawidek.net) Received: from mail.dawidek.net (60.wheelsystems.com [83.12.187.60]) by mx1.freebsd.org (Postfix) with ESMTP id C630C8FC0A; Wed, 27 Jul 2011 05:34:51 +0000 (UTC) Received: from localhost (89-73-195-149.dynamic.chello.pl [89.73.195.149]) by mail.dawidek.net (Postfix) with ESMTPSA id 9CD6892B; Wed, 27 Jul 2011 07:34:49 +0200 (CEST) Date: Wed, 27 Jul 2011 07:34:46 +0200 From: Pawel Jakub Dawidek To: Martin Matuska Message-ID: <20110727053445.GA4808@garage.freebsd.pl> References: <4E2DD17C.60606@delphij.net> <4E2F29AF.5030905@FreeBSD.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="XsQoSWH+UP9D9v3l" Content-Disposition: inline In-Reply-To: <4E2F29AF.5030905@FreeBSD.org> X-OS: FreeBSD 9.0-CURRENT amd64 User-Agent: Mutt/1.5.21 (2010-09-15) Cc: zfs-devel@FreeBSD.ORG, d@delphij.net Subject: Re: [PATCH] Consistently use zfs_ioctl() X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 27 Jul 2011 05:34:52 -0000 --XsQoSWH+UP9D9v3l Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Tue, Jul 26, 2011 at 10:55:11PM +0200, Martin Matuska wrote: > I have chatted with delphij@ and he said his problem was in zfs jail / > unjail. >=20 > The problem is in zfs_jail(), cmd passed to ioctl() is initialized as > int instead of unsigned long. > Patch suggested for fixing this problem is attached. Looks good to me. > Index: cddl/contrib/opensolaris/lib/libzfs/common/libzfs_dataset.c > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > --- cddl/contrib/opensolaris/lib/libzfs/common/libzfs_dataset.c (revision= 224409) > +++ cddl/contrib/opensolaris/lib/libzfs/common/libzfs_dataset.c (working = copy) > @@ -4289,7 +4289,8 @@ > libzfs_handle_t *hdl =3D zhp->zfs_hdl; > zfs_cmd_t zc =3D { 0 }; > char errbuf[1024]; > - int cmd, ret; > + unsigned long cmd; > + int ret; > =20 > if (attach) { > (void) snprintf(errbuf, sizeof (errbuf), --=20 Pawel Jakub Dawidek http://www.wheelsystems.com FreeBSD committer http://www.FreeBSD.org Am I Evil? Yes, I Am! http://yomoli.com --XsQoSWH+UP9D9v3l Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (FreeBSD) iEYEARECAAYFAk4vo3UACgkQForvXbEpPzStaACg7WF3sxP735kTAQXQWgHk/PY4 mQgAoJTRmoFT8iHi3LFmtETpkN0vrOJJ =/L3p -----END PGP SIGNATURE----- --XsQoSWH+UP9D9v3l-- From owner-zfs-devel@FreeBSD.ORG Wed Jul 27 09:09:23 2011 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6711B106564A; Wed, 27 Jul 2011 09:09:23 +0000 (UTC) (envelope-from pawel@dawidek.net) Received: from mail.dawidek.net (60.wheelsystems.com [83.12.187.60]) by mx1.freebsd.org (Postfix) with ESMTP id AB7208FC0A; Wed, 27 Jul 2011 09:09:22 +0000 (UTC) Received: from localhost (58.wheelsystems.com [83.12.187.58]) by mail.dawidek.net (Postfix) with ESMTPSA id 1DD6B9B2; Wed, 27 Jul 2011 11:09:21 +0200 (CEST) Date: Wed, 27 Jul 2011 11:09:18 +0200 From: Pawel Jakub Dawidek To: Martin Matuska Message-ID: <20110727090918.GB1642@garage.freebsd.pl> References: <4E2EB3A4.60007@FreeBSD.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="+pHx0qQiF2pBVqBT" Content-Disposition: inline In-Reply-To: <4E2EB3A4.60007@FreeBSD.org> X-OS: FreeBSD 9.0-CURRENT amd64 User-Agent: Mutt/1.5.21 (2010-09-15) Cc: zfs-devel@FreeBSD.org, Xin Li Subject: Re: [PATCH] Illumos #883 ZIL reuse during remount can lead to data corruption X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 27 Jul 2011 09:09:23 -0000 --+pHx0qQiF2pBVqBT Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Tue, Jul 26, 2011 at 02:31:32PM +0200, Martin Matuska wrote: > Illumos developers have fixed a problem in ZIL that may lead to data > corruption during remount. >=20 > A detailed description of this problem is available at: > https://www.illumos.org/issues/883 >=20 > In my opinion this issue applies to FreeBSD, too, and we should fix this > ASAP. > Please review my patch (1:1 merge from illumos, no changes). >=20 > References: > illumos-gate revision: 13380:161b964a0e10 > https://github.com/illumos/illumos-gate/commit/c9ba2a43cb76c223d115e021fd= abd2c066e020ed This looks like a very important fix. There were several reports of people not being able to import their pools because of space map corruption and it seems like this will fix the underlying problem. I'd really like to see this committed and merged to stable/8. > Index: cddl/contrib/opensolaris/cmd/ztest/ztest.c > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > --- cddl/contrib/opensolaris/cmd/ztest/ztest.c (revision 224409) > +++ cddl/contrib/opensolaris/cmd/ztest/ztest.c (working copy) > @@ -205,6 +205,7 @@ > */ > typedef struct ztest_ds { > objset_t *zd_os; > + rwlock_t zd_zilog_lock; > zilog_t *zd_zilog; > uint64_t zd_seq; > ztest_od_t *zd_od; /* debugging aid */ > @@ -238,6 +239,7 @@ > ztest_func_t ztest_zap; > ztest_func_t ztest_zap_parallel; > ztest_func_t ztest_zil_commit; > +ztest_func_t ztest_zil_remount; > ztest_func_t ztest_dmu_read_write_zcopy; > ztest_func_t ztest_dmu_objset_create_destroy; > ztest_func_t ztest_dmu_prealloc; > @@ -273,6 +275,7 @@ > { ztest_zap_parallel, 100, &zopt_always }, > { ztest_split_pool, 1, &zopt_always }, > { ztest_zil_commit, 1, &zopt_incessant }, > + { ztest_zil_remount, 1, &zopt_sometimes }, > { ztest_dmu_read_write_zcopy, 1, &zopt_often }, > { ztest_dmu_objset_create_destroy, 1, &zopt_often }, > { ztest_dsl_prop_get_set, 1, &zopt_often }, > @@ -986,6 +989,7 @@ > zd->zd_seq =3D 0; > dmu_objset_name(os, zd->zd_name); > =20 > + VERIFY(rwlock_init(&zd->zd_zilog_lock, USYNC_THREAD, NULL) =3D=3D 0); > VERIFY(_mutex_init(&zd->zd_dirobj_lock, USYNC_THREAD, NULL) =3D=3D 0); > =20 > for (int l =3D 0; l < ZTEST_OBJECT_LOCKS; l++) > @@ -1965,6 +1969,8 @@ > if (ztest_random(2) =3D=3D 0) > io_type =3D ZTEST_IO_WRITE_TAG; > =20 > + (void) rw_rdlock(&zd->zd_zilog_lock); > + > switch (io_type) { > =20 > case ZTEST_IO_WRITE_TAG: > @@ -2000,6 +2006,8 @@ > break; > } > =20 > + (void) rw_unlock(&zd->zd_zilog_lock); > + > umem_free(data, blocksize); > } > =20 > @@ -2054,6 +2062,8 @@ > { > zilog_t *zilog =3D zd->zd_zilog; > =20 > + (void) rw_rdlock(&zd->zd_zilog_lock); > + > zil_commit(zilog, ztest_random(ZTEST_OBJECTS)); > =20 > /* > @@ -2065,9 +2075,34 @@ > ASSERT(zd->zd_seq <=3D zilog->zl_commit_lr_seq); > zd->zd_seq =3D zilog->zl_commit_lr_seq; > mutex_exit(&zilog->zl_lock); > + > + (void) rw_unlock(&zd->zd_zilog_lock); > } > =20 > /* > + * This function is designed to simulate the operations that occur durin= g a > + * mount/unmount operation. We hold the dataset across these operations= in an > + * attempt to expose any implicit assumptions about ZIL management. > + */ > +/* ARGSUSED */ > +void > +ztest_zil_remount(ztest_ds_t *zd, uint64_t id) > +{ > + objset_t *os =3D zd->zd_os; > + > + (void) rw_wrlock(&zd->zd_zilog_lock); > + > + /* zfsvfs_teardown() */ > + zil_close(zd->zd_zilog); > + > + /* zfsvfs_setup() */ > + VERIFY(zil_open(os, ztest_get_data) =3D=3D zd->zd_zilog); > + zil_replay(os, zd, ztest_replay_vector); > + > + (void) rw_unlock(&zd->zd_zilog_lock); > +} > + > +/* > * Verify that we can't destroy an active pool, create an existing pool, > * or create a pool with a bad vdev spec. > */ > Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zil.c > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zil.c (revision 224409) > +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zil.c (working copy) > @@ -20,6 +20,7 @@ > */ > /* > * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights re= served. > + * Copyright (c) 2011 by Delphix. All rights reserved. > */ > =20 > /* Portions Copyright 2010 Robert Milkowski */ > @@ -567,7 +568,7 @@ > =20 > if (!list_is_empty(&zilog->zl_lwb_list)) { > ASSERT(zh->zh_claim_txg =3D=3D 0); > - ASSERT(!keep_first); > + VERIFY(!keep_first); > while ((lwb =3D list_head(&zilog->zl_lwb_list)) !=3D NULL) { > list_remove(&zilog->zl_lwb_list, lwb); > if (lwb->lwb_buf !=3D NULL) > @@ -1668,20 +1669,9 @@ > void > zil_free(zilog_t *zilog) > { > - lwb_t *head_lwb; > - > zilog->zl_stop_sync =3D 1; > =20 > - /* > - * After zil_close() there should only be one lwb with a buffer. > - */ > - head_lwb =3D list_head(&zilog->zl_lwb_list); > - if (head_lwb) { > - ASSERT(head_lwb =3D=3D list_tail(&zilog->zl_lwb_list)); > - list_remove(&zilog->zl_lwb_list, head_lwb); > - zio_buf_free(head_lwb->lwb_buf, head_lwb->lwb_sz); > - kmem_cache_free(zil_lwb_cache, head_lwb); > - } > + ASSERT(list_is_empty(&zilog->zl_lwb_list)); > list_destroy(&zilog->zl_lwb_list); > =20 > avl_destroy(&zilog->zl_vdev_tree); > @@ -1721,6 +1711,10 @@ > { > zilog_t *zilog =3D dmu_objset_zil(os); > =20 > + ASSERT(zilog->zl_clean_taskq =3D=3D NULL); > + ASSERT(zilog->zl_get_data =3D=3D NULL); > + ASSERT(list_is_empty(&zilog->zl_lwb_list)); > + > zilog->zl_get_data =3D get_data; > zilog->zl_clean_taskq =3D taskq_create("zil_clean", 1, minclsyspri, > 2, 2, TASKQ_PREPOPULATE); > @@ -1734,7 +1728,7 @@ > void > zil_close(zilog_t *zilog) > { > - lwb_t *tail_lwb; > + lwb_t *lwb; > uint64_t txg =3D 0; > =20 > zil_commit(zilog, 0); /* commit all itx */ > @@ -1746,9 +1740,9 @@ > * destroy the zl_clean_taskq. > */ > mutex_enter(&zilog->zl_lock); > - tail_lwb =3D list_tail(&zilog->zl_lwb_list); > - if (tail_lwb !=3D NULL) > - txg =3D tail_lwb->lwb_max_txg; > + lwb =3D list_tail(&zilog->zl_lwb_list); > + if (lwb !=3D NULL) > + txg =3D lwb->lwb_max_txg; > mutex_exit(&zilog->zl_lock); > if (txg) > txg_wait_synced(zilog->zl_dmu_pool, txg); > @@ -1756,6 +1750,19 @@ > taskq_destroy(zilog->zl_clean_taskq); > zilog->zl_clean_taskq =3D NULL; > zilog->zl_get_data =3D NULL; > + > + /* > + * We should have only one LWB left on the list; remove it now. > + */ > + mutex_enter(&zilog->zl_lock); > + lwb =3D list_head(&zilog->zl_lwb_list); > + if (lwb !=3D NULL) { > + ASSERT(lwb =3D=3D list_tail(&zilog->zl_lwb_list)); > + list_remove(&zilog->zl_lwb_list, lwb); > + zio_buf_free(lwb->lwb_buf, lwb->lwb_sz); > + kmem_cache_free(zil_lwb_cache, lwb); > + } > + mutex_exit(&zilog->zl_lock); > } > =20 > /* --=20 Pawel Jakub Dawidek http://www.wheelsystems.com FreeBSD committer http://www.FreeBSD.org Am I Evil? Yes, I Am! http://yomoli.com --+pHx0qQiF2pBVqBT Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (FreeBSD) iEYEARECAAYFAk4v1b4ACgkQForvXbEpPzSvIgCg2tAl6I84bMPr/Gb+/6OFCI0O 910An3kcKh0RgFKtj+EnWkiCdxVF5LvR =b2ZW -----END PGP SIGNATURE----- --+pHx0qQiF2pBVqBT-- From owner-zfs-devel@FreeBSD.ORG Wed Jul 27 17:07:45 2011 Return-Path: Delivered-To: zfs-devel@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id ED62F106566C; Wed, 27 Jul 2011 17:07:44 +0000 (UTC) (envelope-from delphij@delphij.net) Received: from anubis.delphij.net (anubis.delphij.net [IPv6:2001:470:1:117::25]) by mx1.freebsd.org (Postfix) with ESMTP id D46EE8FC0C; Wed, 27 Jul 2011 17:07:44 +0000 (UTC) Received: from delta.delphij.net (drawbridge.ixsystems.com [206.40.55.65]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by anubis.delphij.net (Postfix) with ESMTPSA id A2ECF1496D; Wed, 27 Jul 2011 10:07:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=delphij.net; s=anubis; t=1311786464; bh=GiOHvCl09OcXBfsjppKVVap81mmQ26r9iz7OnBeQeYA=; h=Message-ID:Date:From:Reply-To:MIME-Version:To:CC:Subject: References:In-Reply-To:Content-Type:Content-Transfer-Encoding; b=JVbfaRenxGc4oQuYkdDATQ3ma0OU98OdDxzP2Fq+GHTn5bnlPCjsLep3OdsA5sM+s TwUuwEjc5pKkUv6shCY+u/zo3b08Ayr7OHP6snjqy40IMrgWKVGLVNTLtreqP8cICE wzb9KCVSBwoyopN7XyPPd8w2O8/4EkN9J+2s+fB8= Message-ID: <4E3045E0.7030108@delphij.net> Date: Wed, 27 Jul 2011 10:07:44 -0700 From: Xin LI Organization: The FreeBSD Project MIME-Version: 1.0 To: Martin Matuska References: <4E2DD17C.60606@delphij.net> <4E2F29AF.5030905@FreeBSD.org> In-Reply-To: <4E2F29AF.5030905@FreeBSD.org> OpenPGP: id=3FCA37C1; url=http://www.delphij.net/delphij.asc Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: zfs-devel@FreeBSD.ORG, d@delphij.net Subject: Re: [PATCH] Consistently use zfs_ioctl() X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: d@delphij.net List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 27 Jul 2011 17:07:45 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 On 07/26/11 13:55, Martin Matuska wrote: > I have chatted with delphij@ and he said his problem was in zfs jail > / unjail. > > The problem is in zfs_jail(), cmd passed to ioctl() is initialized > as int instead of unsigned long. Patch suggested for fixing this > problem is attached. Sorry for the delay and this solves the problem more neatly. Would you please propose your version to re@? Cheers, - -- Xin LI https://www.delphij.net/ FreeBSD - The Power to Serve! Live free or die -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.17 (FreeBSD) iQEcBAEBCAAGBQJOMEXfAAoJEATO+BI/yjfBf5EH/R12+1eMmB55+/zmLBOOimRf FA7XdBKhs556zP2iBw/2PWLON/2KANFTpPZnBZY9J2vO74CFYXP7e35UFNd9M7WS yYjkTu+E48k5GPqCk1OQw5Upx2ASrQQTwp2L0TzVlR1/O6APs4NaYUoODjGsWy48 IGpRkUO4NGUpNR8oCGRAiuY6nH/9OuT91r1EPBB3zg4sr0zxl15gcYqDxFYAU8fE 1yr+Vp64s9UA1l8Ast0/fjs/b1dOC2sZ+ErCFRdWpPlte014Uplh/dCCprZsk+dk PBcMbOa6IlzIbtSYIWKzIN6Q4iEjldjOnyy5Xdzt1iRSMRrzvociQF4qaRD67D0= =Pycb -----END PGP SIGNATURE----- From owner-zfs-devel@FreeBSD.ORG Wed Jul 27 20:08:47 2011 Return-Path: Delivered-To: zfs-devel@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8C982106566C; Wed, 27 Jul 2011 20:08:47 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from mail.vx.sk (mail.vx.sk [IPv6:2a01:4f8:100:1043::3]) by mx1.freebsd.org (Postfix) with ESMTP id 107BB8FC13; Wed, 27 Jul 2011 20:08:47 +0000 (UTC) Received: from core.vx.sk (localhost [127.0.0.1]) by mail.vx.sk (Postfix) with ESMTP id D2DB9167161; Wed, 27 Jul 2011 22:08:45 +0200 (CEST) X-Virus-Scanned: amavisd-new at mail.vx.sk Received: from mail.vx.sk ([127.0.0.1]) by core.vx.sk (mail.vx.sk [127.0.0.1]) (amavisd-new, port 10024) with LMTP id GrKtm9UJsmNz; Wed, 27 Jul 2011 22:08:43 +0200 (CEST) Received: from [10.9.8.3] (chello085216231078.chello.sk [85.216.231.78]) by mail.vx.sk (Postfix) with ESMTPSA id D7ABF167149; Wed, 27 Jul 2011 22:08:42 +0200 (CEST) Message-ID: <4E30704C.9070606@FreeBSD.org> Date: Wed, 27 Jul 2011 22:08:44 +0200 From: Martin Matuska User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:5.0) Gecko/20110624 Thunderbird/5.0 MIME-Version: 1.0 To: Pawel Jakub Dawidek , Xin LI X-Enigmail-Version: 1.2 Content-Type: multipart/mixed; boundary="------------000705020007040007080007" Cc: zfs-devel@FreeBSD.ORG, Kostik Belousov Subject: [CFR] [PATCH] jail mount/unmount patch X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 27 Jul 2011 20:08:47 -0000 This is a multi-part message in MIME format. --------------000705020007040007080007 Content-Type: text/plain; charset=windows-1250 Content-Transfer-Encoding: 7bit Please review my attached patch. The patch fixes mount/unmount inside a jail for filesystems with the VFCF_JAIL flag security.jail.mount_allowed=1 is required. For "enforce_statfs == 2" it makes no sense to allow mount/unmount inside a jail so enforce_statfs == 2 implies mount_allowed = 0 The filesystems mounted inside a jail have now a corect f_mntonname. Tested with: zfs - works! (both enforce_statfs=0 and enforce_statfs=1) nullfs (with added VFCF_JAIL flag) - works ! (both enforce_statfs=0 and enforce_statfs=1) tmpfs (with added VFCF_JAIL flag) - works ! (both enforce_statfs=0 and enforce_statfs=1) I assume other filesystems are going to work correctly, too (e.g nfs). Fore the future, I suggest a option to allow mounting specific filesystems in a jail (e.g. zfs, nullfs, tmpfs). I consider nullfs mounts harmless inside a jail. With jailed nullfs and tmpfs we can run tinderbox in a jail! Cheers, mm -- Martin Matuska FreeBSD committer http://blog.vx.sk --------------000705020007040007080007 Content-Type: text/plain; name="jail_mount.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="jail_mount.patch" Index: src/sys/kern/kern_jail.c =================================================================== --- src/sys/kern/kern_jail.c (revision 224297) +++ src/sys/kern/kern_jail.c (working copy) @@ -3858,7 +3858,8 @@ case PRIV_VFS_UNMOUNT: case PRIV_VFS_MOUNT_NONUSER: case PRIV_VFS_MOUNT_OWNER: - if (cred->cr_prison->pr_allow & PR_ALLOW_MOUNT) + if (cred->cr_prison->pr_allow & PR_ALLOW_MOUNT && + cred->cr_prison->pr_enforce_statfs != 2) return (0); else return (EPERM); Index: src/sys/kern/vfs_mount.c =================================================================== --- src/sys/kern/vfs_mount.c (revision 224297) +++ src/sys/kern/vfs_mount.c (working copy) @@ -1007,6 +1007,7 @@ struct vfsconf *vfsp; struct nameidata nd; struct vnode *vp; + char realfspath[MNAMELEN]; int error; /* @@ -1023,6 +1024,21 @@ } /* + * If we are jailed, construct real filesystem path + */ + if (jailed(td->td_ucred) && + strcmp(td->td_ucred->cr_prison->pr_path, "/") != 0) { + if (strlen(td->td_ucred->cr_prison->pr_path) + + strlen(fspath) >= MNAMELEN) + return (ENAMETOOLONG); + strlcpy(realfspath, td->td_ucred->cr_prison->pr_path, + sizeof(realfspath)); + strlcat(realfspath, fspath, sizeof(realfspath)); + } else { + strlcpy(realfspath, fspath, sizeof(realfspath)); + } + + /* * Do not allow NFS export or MNT_SUIDDIR by unprivileged users. */ if (fsflags & MNT_EXPORTED) { @@ -1070,7 +1086,7 @@ NDFREE(&nd, NDF_ONLY_PNBUF); vp = nd.ni_vp; if ((fsflags & MNT_UPDATE) == 0) { - error = vfs_domount_first(td, vfsp, fspath, vp, fsflags, + error = vfs_domount_first(td, vfsp, realfspath, vp, fsflags, optlist); } else { error = vfs_domount_update(td, vp, fsflags, optlist); @@ -1107,6 +1123,7 @@ struct mount *mp; char *pathbuf; int error, id0, id1; + char realfspath[MNAMELEN]; AUDIT_ARG_VALUE(uap->flags); if (jailed(td->td_ucred) || usermount == 0) { @@ -1139,10 +1156,23 @@ } mtx_unlock(&mountlist_mtx); } else { - AUDIT_ARG_UPATH1(td, pathbuf); + /* + * If we are jailed and enforce_statfs=1 + * construct real filesystem path + */ + if (jailed(td->td_ucred) && + td->td_ucred->cr_prison->pr_enforce_statfs == 1 && + strcmp(td->td_ucred->cr_prison->pr_path, "/") != 0) { + strlcpy(realfspath, td->td_ucred->cr_prison->pr_path, + sizeof(realfspath)); + strlcat(realfspath, pathbuf, sizeof(realfspath)); + } else { + strlcpy(realfspath, pathbuf, sizeof(realfspath)); + } + AUDIT_ARG_UPATH1(td, realfspath); mtx_lock(&mountlist_mtx); TAILQ_FOREACH_REVERSE(mp, &mountlist, mntlist, mnt_list) { - if (strcmp(mp->mnt_stat.f_mntonname, pathbuf) == 0) + if (strcmp(mp->mnt_stat.f_mntonname, realfspath) == 0) break; } mtx_unlock(&mountlist_mtx); --------------000705020007040007080007-- From owner-zfs-devel@FreeBSD.ORG Thu Jul 28 14:11:42 2011 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8FDCB1065672; Thu, 28 Jul 2011 14:11:42 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from mail.vx.sk (mail.vx.sk [IPv6:2a01:4f8:100:1043::3]) by mx1.freebsd.org (Postfix) with ESMTP id E4E1A8FC15; Thu, 28 Jul 2011 14:11:41 +0000 (UTC) Received: from core.vx.sk (localhost [127.0.0.1]) by mail.vx.sk (Postfix) with ESMTP id CE024167157; Thu, 28 Jul 2011 16:11:40 +0200 (CEST) X-Virus-Scanned: amavisd-new at mail.vx.sk Received: from mail.vx.sk ([127.0.0.1]) by core.vx.sk (mail.vx.sk [127.0.0.1]) (amavisd-new, port 10024) with LMTP id 1Ye+jXhIbmzV; Thu, 28 Jul 2011 16:11:38 +0200 (CEST) Received: from [10.0.3.3] (188-167-66-148.dynamic.chello.sk [188.167.66.148]) by mail.vx.sk (Postfix) with ESMTPSA id E3EF916714E; Thu, 28 Jul 2011 16:11:37 +0200 (CEST) Message-ID: <4E316E19.9040309@FreeBSD.org> Date: Thu, 28 Jul 2011 16:11:37 +0200 From: Martin Matuska User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:5.0) Gecko/20110627 Thunderbird/5.0 MIME-Version: 1.0 To: freebsd-current@FreeBSD.org X-Enigmail-Version: 1.2pre Content-Type: multipart/mixed; boundary="------------000605040303080100060003" X-Mailman-Approved-At: Thu, 28 Jul 2011 14:39:21 +0000 Cc: Subject: [PATCH] updated /etc/rc.d/jail and added ZFS support X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Jul 2011 14:11:42 -0000 This is a multi-part message in MIME format. --------------000605040303080100060003 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit The attached patch allows better fine-tuning of jails started via /etc/rc.d, uses the new jail(8) flags (-c -m), the persist parameter and adds ZFS support. Patch is fully backward compatible. Please review, comment and/or test my attached patch. Cheers, mm -- Martin Matuska FreeBSD committer http://blog.vx.sk --------------000605040303080100060003 Content-Type: text/x-patch; name="etc_jail.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="etc_jail.patch" Index: etc/rc.d/jail =================================================================== --- etc/rc.d/jail (revision 224471) +++ etc/rc.d/jail (working copy) @@ -43,6 +43,7 @@ eval _ip=\"\$jail_${_j}_ip\" eval _interface=\"\${jail_${_j}_interface:-${jail_interface}}\" eval _exec=\"\$jail_${_j}_exec\" + eval _params=\"\$jail_${_j}_params\" i=0 while : ; do @@ -83,6 +84,8 @@ i=$((i + 1)) done + eval _zfs=\"\${jail_${_j}_zfs:-}\" + if [ -n "${_exec}" ]; then # simple/backward-compatible execution _exec_start="${_exec}" @@ -98,6 +101,9 @@ fi # The default jail ruleset will be used by rc.subr if none is specified. + if [ -n "jail_devfs_ruleset" -a -n "_zfs" ]; then + jail_devfs_ruleset="devfsrules_jail_zfs" + fi eval _ruleset=\"\${jail_${_j}_devfs_ruleset:-${jail_devfs_ruleset}}\" eval _devfs=\"\${jail_${_j}_devfs_enable:-${jail_devfs_enable}}\" [ -z "${_devfs}" ] && _devfs="NO" @@ -200,6 +206,58 @@ if [ -z "${_rootdir}" ]; then err 3 "$name: No root directory has been defined for ${_j}" fi + + # Security-related parameters + eval _enforce_statfs=\"\$jail_${_j}_enforce_statfs\" + eval _allow_set_hostname=\"\$jail_${_j}_allow_set_hostname\" + eval _allow_sysvipc=\"\$jail_${_j}_allow_sysvipc\" + eval _allow_raw_sockets=\"\$jail_${_j}_allow_raw_sockets\" + eval _allow_chflags=\"\$jail_${_j}_allow_chflags\" + eval _allow_mount=\"\$jail_${_j}_allow_mount\" + eval _allow_socket_af=\"\$jail_${_j}_allow_socket_af\" + eval _allow_quotas=\"\$jail_${_j}_allow_quotas:-0\" + + if [ -z "${_enforce_statfs}" ]; then + _enforce_statfs=`${SYSCTL} -n security.jail.enforce_statfs` + fi + + if [ -z "${_allow_set_hostname}" ]; then + _allow_set_hostname=`${SYSCTL} -n security.jail.set_hostname_allowed` + fi + + if [ -z "${_allow_sysvipc}" ]; then + _allow_sysvipc=`${SYSCTL} -n security.jail.sysvipc_allowed` + fi + + if [ -z "${_allow_raw_sockets}" ]; then + _allow_raw_sockets=`${SYSCTL} -n security.jail.allow_raw_sockets` + fi + + if [ -z "${_allow_chflags}" ]; then + _allow_chflags=`${SYSCTL} -n security.jail.chflags_allowed` + fi + + if [ -z "${_allow_mount}" ]; then + _allow_mount=`${SYSCTL} -n security.jail.mount_allowed` + fi + + if [ -z "${_allow_socket_af}" ]; then + _tmpval=`${SYSCTL} -n security.jail.socket_unixiproute_only` + if [ "${_tmpval}" = "0" ]; then + _allow_socket_af=1 + else + _allow_socket_af=0 + fi + fi + + _security_params="enforce_statfs=${_enforce_statfs} \ + allow.set_hostname=${_allow_set_hostname} \ + allow.sysvipc=${_allow_sysvipc} \ + allow.raw_sockets=${_allow_raw_sockets} \ + allow.chflags=${_allow_chflags} \ + allow.mount=${_allow_mount} \ + allow.socket_af=${_allow_socket_af} \ + allow.quotas=${allow_quotas}" } # set_sysctl rc_knob mib msg @@ -345,6 +403,36 @@ mount -a -F "${_fstab}" } +# jail_zfs_jailin +# Make zfs datasets manageable from inside a jail +# the "jailed" dataset property must be set to "on" +jail_zfs_jailin() +{ + if [ -n "${_zfs}" ]; then + for _ds in ${_zfs}; do + _jailed=`zfs get -H jailed ${_ds} 2>/dev/null | awk '{ print $3 }'` + if [ "$_jailed" = "on" ]; then + zfs jail "${_jail_id}" ${_ds} 2>/dev/null + fi + done + fi +} + +# jail_zfs_jailout +# Unjail zfs datasets +# the "jailed" dataset property must be set to "on" +jail_zfs_jailout() +{ + if [ -n "${_zfs}" ]; then + for _ds in ${_zfs}; do + _jailed=`zfs get -H jailed ${_ds} 2>/dev/null | awk '{ print $3 }'` + if [ "$_jailed" = "on" ]; then + zfs unjail "${_jail_id}" ${_ds} 2>/dev/null + fi + done + fi +} + # jail_show_addresses jail # Debug print the input for the given _multi aliases # for a jail for init_variables(). @@ -483,10 +571,27 @@ *) ;; esac - # Append address to list of addresses for the jail command. - case "${_addrl}" in - "") _addrl="${_addr}" ;; - *) _addrl="${_addrl},${_addr}" ;; + case "${_type}" in + inet) + # Append address to list of ipv4 addresses for the + # jail command. + case "${_addrl}" in + "") _addrl="${_addr}" ;; + *) _addrl="${_addrl},${_addr}" ;; + esac + ;; + inet6) + # Append address to list of ipv6 addresses for the + # jail command. + case "${_addrl6}" in + "") _addrl6="${_addr}" ;; + *) _addrl6="${_addrl6},${_addr}" ;; + esac + ;; + *) warn "Could not determine address family. Not going" \ + "to set address '${_addr}' for ${_jail}." + continue + ;; esac # Configure interface alias if requested by a given interface @@ -494,14 +599,7 @@ case "${_iface}" in "") continue ;; esac - case "${_type}" in - inet) ;; - inet6) ;; - *) warn "Could not determine address family. Not going" \ - "to ${_action} address '${_addr}' for ${_jail}." - continue - ;; - esac + case "${_action}" in add) ifconfig ${_iface} ${_type} ${_addr}${_mask} alias ;; @@ -576,6 +674,7 @@ continue; fi _addrl="" + _addrl6="" jail_ips "add" if [ -n "${_fib}" ]; then _setfib="setfib -F '${_fib}'" @@ -644,42 +743,54 @@ i=$((i + 1)) done - eval ${_setfib} jail ${_flags} -i ${_rootdir} ${_hostname} \ - \"${_addrl}\" ${_exec_start} > ${_tmp_jail} 2>&1 \ - ${_tmp_jail} 2>&1 \ + ${_consolelog} - echo ${_jail_id} > /var/run/jail_${_jail}.id + echo -n " $_hostname" + tail +2 ${_tmp_jail} >${_consolelog} + echo ${_jail_id} > /var/run/jail_${_jail}.id - i=0 - while : ; do - eval out=\"\${_exec_poststart${i}:-''}\" - [ -z "$out" ] && break - ${out} - i=$((i + 1)) - done - else - jail_umount_fs - jail_ips "del" - echo " cannot start jail \"${_jail}\": " - tail +2 ${_tmp_jail} + i=0 + while : ; do + eval out=\"\${_exec_poststart${i}:-''}\" + [ -z "$out" ] && break + ${out} + i=$((i + 1)) + done + else + jail_zfs_jailout + jail -m jid="${_jail_id}" persist=0 + jail_umount_fs + jail_ips "del" + echo " cannot start jail \"${_jail}\": " + tail +2 ${_tmp_jail} + fi + rm -f ${_tmp_jail} fi - rm -f ${_tmp_jail} done rmdir ${_tmp_dir} echo '.' @@ -707,6 +818,7 @@ eval env -i /usr/sbin/jexec ${_jail_id} ${_exec_stop} \ >> ${_consolelog} 2>&1 fi + jail_zfs_jailout killall -j ${_jail_id} -TERM > /dev/null 2>&1 sleep 1 killall -j ${_jail_id} -KILL > /dev/null 2>&1 Index: etc/defaults/devfs.rules =================================================================== --- etc/defaults/devfs.rules (revision 224471) +++ etc/defaults/devfs.rules (working copy) @@ -83,3 +83,9 @@ add include $devfsrules_hide_all add include $devfsrules_unhide_basic add include $devfsrules_unhide_login + +# Jail with zfs support +# +[devfsrules_jail_zfs=5] +add include $devfsrules_jail +add path zfs unhide Index: etc/defaults/rc.conf =================================================================== --- etc/defaults/rc.conf (revision 224471) +++ etc/defaults/rc.conf (working copy) @@ -695,6 +695,21 @@ #jail_example_mount_enable="NO" # mount/umount jail's fs #jail_example_fstab="" # fstab(5) for mount/umount #jail_example_flags="-l -U root" # flags for jail(8) +#jail_example_params="" # additional parameters for jail(8) +#jail_example_enforce_statfs="" # jail(8) enforce_statfs parameter +#jail_example_allow_set_hostname="" # jail(8) allow.set_hostname parameter +#jail_example_allow_sysvipc="" # jail(8) allow.sysvipc parameter +#jail_example_allow_raw_sockets="" # jail(8) allow.raw_sockets parameter +#jail_example_allow_chflags="" # jail(8) allow.chflags parameter +#jail_example_allow_mount="" # jail(8) allow.mount parameter +#jail_example_allow_socket_af="" # jail(8) allow.socket_af parameter +#jail_example_allow_quotas="" # jail(8) allow.quotas parameter +#jail_example_zfs="" # Space-separated list of ZFS datasets to be + # managed from this jail. For proper operation, + # allow_mount must be defined and enforce_statfs + # must be lower than 2. The "jailed" property + # must be set to "on" on these datasets before starting + # the jail. ############################################################## ### Define source_rc_confs, the mechanism used by /etc/rc.* ## --------------000605040303080100060003-- From owner-zfs-devel@FreeBSD.ORG Thu Jul 28 16:07:58 2011 Return-Path: Delivered-To: zfs-devel@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 17B83106564A; Thu, 28 Jul 2011 16:07:58 +0000 (UTC) (envelope-from pawel@dawidek.net) Received: from mail.dawidek.net (60.wheelsystems.com [83.12.187.60]) by mx1.freebsd.org (Postfix) with ESMTP id C18448FC15; Thu, 28 Jul 2011 16:07:57 +0000 (UTC) Received: from localhost (89-73-195-149.dynamic.chello.pl [89.73.195.149]) by mail.dawidek.net (Postfix) with ESMTPSA id 7730CA3; Thu, 28 Jul 2011 18:07:55 +0200 (CEST) Date: Thu, 28 Jul 2011 18:07:52 +0200 From: Pawel Jakub Dawidek To: Martin Matuska Message-ID: <20110728160751.GA1704@garage.freebsd.pl> References: <4E30704C.9070606@FreeBSD.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="2oS5YaxWCcQjTEyO" Content-Disposition: inline In-Reply-To: <4E30704C.9070606@FreeBSD.org> X-OS: FreeBSD 9.0-CURRENT amd64 User-Agent: Mutt/1.5.21 (2010-09-15) Cc: Kostik Belousov , zfs-devel@FreeBSD.ORG Subject: Re: [CFR] [PATCH] jail mount/unmount patch X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Jul 2011 16:07:58 -0000 --2oS5YaxWCcQjTEyO Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, Jul 27, 2011 at 10:08:44PM +0200, Martin Matuska wrote: > Please review my attached patch. >=20 > The patch fixes mount/unmount inside a jail for filesystems with the > VFCF_JAIL flag > security.jail.mount_allowed=3D1 is required. > For "enforce_statfs =3D=3D 2" it makes no sense to allow mount/unmount > inside a jail so enforce_statfs =3D=3D 2 implies mount_allowed =3D 0 > The filesystems mounted inside a jail have now a corect f_mntonname. >=20 > Tested with: > zfs - works! (both enforce_statfs=3D0 and enforce_statfs=3D1) > nullfs (with added VFCF_JAIL flag) - works ! (both enforce_statfs=3D0 and > enforce_statfs=3D1) > tmpfs (with added VFCF_JAIL flag) - works ! (both enforce_statfs=3D0 and > enforce_statfs=3D1) >=20 > I assume other filesystems are going to work correctly, too (e.g nfs). > Fore the future, I suggest a option to allow mounting specific > filesystems in a jail (e.g. zfs, nullfs, tmpfs). >=20 > I consider nullfs mounts harmless inside a jail. >=20 > With jailed nullfs and tmpfs we can run tinderbox in a jail! In you patch you depend on fact that full path to mount directory is passed to the nmount(2) system call. This doesn't have to be true. I changed mount(8) to call realpath(3) in mount directory, but I see no reason someone calling nmount(2) directly with "./foo" mount dir. I think the proper way is to build full path from within the kernel using vn_fullpath_global(). --=20 Pawel Jakub Dawidek http://www.wheelsystems.com FreeBSD committer http://www.FreeBSD.org Am I Evil? Yes, I Am! http://yomoli.com --2oS5YaxWCcQjTEyO Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (FreeBSD) iEYEARECAAYFAk4xiVcACgkQForvXbEpPzRfGwCdE4+FvceZi2LZvA+nFBEl/nHJ OeEAnRwVfIED/tFkkWBN3DLqU3pFfUMQ =COGe -----END PGP SIGNATURE----- --2oS5YaxWCcQjTEyO-- From owner-zfs-devel@FreeBSD.ORG Thu Jul 28 17:38:16 2011 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9DEDB106564A; Thu, 28 Jul 2011 17:38:16 +0000 (UTC) (envelope-from pawel@dawidek.net) Received: from mail.dawidek.net (60.wheelsystems.com [83.12.187.60]) by mx1.freebsd.org (Postfix) with ESMTP id 531A48FC18; Thu, 28 Jul 2011 17:38:16 +0000 (UTC) Received: from localhost (89-73-195-149.dynamic.chello.pl [89.73.195.149]) by mail.dawidek.net (Postfix) with ESMTPSA id 9CEBFE0; Thu, 28 Jul 2011 19:38:14 +0200 (CEST) Date: Thu, 28 Jul 2011 19:38:11 +0200 From: Pawel Jakub Dawidek To: Kostik Belousov Message-ID: <20110728173811.GB1704@garage.freebsd.pl> References: <4E30704C.9070606@FreeBSD.org> <20110728160751.GA1704@garage.freebsd.pl> <20110728172539.GU17489@deviant.kiev.zoral.com.ua> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="LpQ9ahxlCli8rRTG" Content-Disposition: inline In-Reply-To: <20110728172539.GU17489@deviant.kiev.zoral.com.ua> X-OS: FreeBSD 9.0-CURRENT amd64 User-Agent: Mutt/1.5.21 (2010-09-15) Cc: zfs-devel@freebsd.org Subject: Re: [CFR] [PATCH] jail mount/unmount patch X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Jul 2011 17:38:16 -0000 --LpQ9ahxlCli8rRTG Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Thu, Jul 28, 2011 at 08:25:39PM +0300, Kostik Belousov wrote: > On Thu, Jul 28, 2011 at 06:07:52PM +0200, Pawel Jakub Dawidek wrote: > > In you patch you depend on fact that full path to mount directory is > > passed to the nmount(2) system call. This doesn't have to be true. > > I changed mount(8) to call realpath(3) in mount directory, but I see no > > reason someone calling nmount(2) directly with "./foo" mount dir. > >=20 > > I think the proper way is to build full path from within the kernel > > using vn_fullpath_global(). >=20 > It indeed may work if the supplied vnode is a directory, since it > will fall back to the dotdot lookup loop if the namecache is purged. Exactly. Mount directory by definition is always a directory, so it should be reliable. --=20 Pawel Jakub Dawidek http://www.wheelsystems.com FreeBSD committer http://www.FreeBSD.org Am I Evil? Yes, I Am! http://yomoli.com --LpQ9ahxlCli8rRTG Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (FreeBSD) iEYEARECAAYFAk4xnoIACgkQForvXbEpPzS3OACgyy0E1Ycqz1AbucDo0oBeP3wc CAsAn2x7PpreyfSoyUbYdhWCoElxZFCf =QuAm -----END PGP SIGNATURE----- --LpQ9ahxlCli8rRTG-- From owner-zfs-devel@FreeBSD.ORG Thu Jul 28 17:40:05 2011 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D795D106566C for ; Thu, 28 Jul 2011 17:40:05 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.zoral.com.ua (mx0.zoral.com.ua [91.193.166.200]) by mx1.freebsd.org (Postfix) with ESMTP id 6B06B8FC16 for ; Thu, 28 Jul 2011 17:40:05 +0000 (UTC) Received: from deviant.kiev.zoral.com.ua (root@deviant.kiev.zoral.com.ua [10.1.1.148]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id p6SHPd4B019140 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 28 Jul 2011 20:25:39 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: from deviant.kiev.zoral.com.ua (kostik@localhost [127.0.0.1]) by deviant.kiev.zoral.com.ua (8.14.4/8.14.4) with ESMTP id p6SHPd1S015954; Thu, 28 Jul 2011 20:25:39 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by deviant.kiev.zoral.com.ua (8.14.4/8.14.4/Submit) id p6SHPd21015953; Thu, 28 Jul 2011 20:25:39 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: deviant.kiev.zoral.com.ua: kostik set sender to kostikbel@gmail.com using -f Date: Thu, 28 Jul 2011 20:25:39 +0300 From: Kostik Belousov To: Pawel Jakub Dawidek Message-ID: <20110728172539.GU17489@deviant.kiev.zoral.com.ua> References: <4E30704C.9070606@FreeBSD.org> <20110728160751.GA1704@garage.freebsd.pl> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="xK0oco9ieugCOMaM" Content-Disposition: inline In-Reply-To: <20110728160751.GA1704@garage.freebsd.pl> User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: clamav-milter 0.95.2 at skuns.kiev.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-3.3 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00, DNS_FROM_OPENWHOIS autolearn=no version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on skuns.kiev.zoral.com.ua Cc: zfs-devel@freebsd.org Subject: Re: [CFR] [PATCH] jail mount/unmount patch X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Jul 2011 17:40:06 -0000 --xK0oco9ieugCOMaM Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Thu, Jul 28, 2011 at 06:07:52PM +0200, Pawel Jakub Dawidek wrote: > On Wed, Jul 27, 2011 at 10:08:44PM +0200, Martin Matuska wrote: > > Please review my attached patch. > >=20 > > The patch fixes mount/unmount inside a jail for filesystems with the > > VFCF_JAIL flag > > security.jail.mount_allowed=3D1 is required. > > For "enforce_statfs =3D=3D 2" it makes no sense to allow mount/unmount > > inside a jail so enforce_statfs =3D=3D 2 implies mount_allowed =3D 0 > > The filesystems mounted inside a jail have now a corect f_mntonname. > >=20 > > Tested with: > > zfs - works! (both enforce_statfs=3D0 and enforce_statfs=3D1) > > nullfs (with added VFCF_JAIL flag) - works ! (both enforce_statfs=3D0 a= nd > > enforce_statfs=3D1) > > tmpfs (with added VFCF_JAIL flag) - works ! (both enforce_statfs=3D0 and > > enforce_statfs=3D1) > >=20 > > I assume other filesystems are going to work correctly, too (e.g nfs). > > Fore the future, I suggest a option to allow mounting specific > > filesystems in a jail (e.g. zfs, nullfs, tmpfs). > >=20 > > I consider nullfs mounts harmless inside a jail. > >=20 > > With jailed nullfs and tmpfs we can run tinderbox in a jail! >=20 > In you patch you depend on fact that full path to mount directory is > passed to the nmount(2) system call. This doesn't have to be true. > I changed mount(8) to call realpath(3) in mount directory, but I see no > reason someone calling nmount(2) directly with "./foo" mount dir. >=20 > I think the proper way is to build full path from within the kernel > using vn_fullpath_global(). It indeed may work if the supplied vnode is a directory, since it will fall back to the dotdot lookup loop if the namecache is purged. --xK0oco9ieugCOMaM Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (FreeBSD) iEYEARECAAYFAk4xm5IACgkQC3+MBN1Mb4hVvgCgkf3/1L/oVW34LWyKmZrOx5AQ eSUAn3Y3Wsu8eeBlv50qSjttesfZr6Tc =wZk4 -----END PGP SIGNATURE----- --xK0oco9ieugCOMaM-- From owner-zfs-devel@FreeBSD.ORG Thu Jul 28 17:41:50 2011 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 50F2F106566C; Thu, 28 Jul 2011 17:41:50 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.zoral.com.ua (mx0.zoral.com.ua [91.193.166.200]) by mx1.freebsd.org (Postfix) with ESMTP id BE3198FC0A; Thu, 28 Jul 2011 17:41:49 +0000 (UTC) Received: from deviant.kiev.zoral.com.ua (root@deviant.kiev.zoral.com.ua [10.1.1.148]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id p6SHfgoW020892 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 28 Jul 2011 20:41:42 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: from deviant.kiev.zoral.com.ua (kostik@localhost [127.0.0.1]) by deviant.kiev.zoral.com.ua (8.14.4/8.14.4) with ESMTP id p6SHfg1r016562; Thu, 28 Jul 2011 20:41:42 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by deviant.kiev.zoral.com.ua (8.14.4/8.14.4/Submit) id p6SHfgaM016561; Thu, 28 Jul 2011 20:41:42 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: deviant.kiev.zoral.com.ua: kostik set sender to kostikbel@gmail.com using -f Date: Thu, 28 Jul 2011 20:41:42 +0300 From: Kostik Belousov To: Pawel Jakub Dawidek Message-ID: <20110728174142.GW17489@deviant.kiev.zoral.com.ua> References: <4E30704C.9070606@FreeBSD.org> <20110728160751.GA1704@garage.freebsd.pl> <20110728172539.GU17489@deviant.kiev.zoral.com.ua> <20110728173811.GB1704@garage.freebsd.pl> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="0N88lRoFd5qAfd1F" Content-Disposition: inline In-Reply-To: <20110728173811.GB1704@garage.freebsd.pl> User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: clamav-milter 0.95.2 at skuns.kiev.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-3.3 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00, DNS_FROM_OPENWHOIS autolearn=no version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on skuns.kiev.zoral.com.ua Cc: zfs-devel@FreeBSD.org, Martin Matuska Subject: Re: [CFR] [PATCH] jail mount/unmount patch X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Jul 2011 17:41:50 -0000 --0N88lRoFd5qAfd1F Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Thu, Jul 28, 2011 at 07:38:11PM +0200, Pawel Jakub Dawidek wrote: > On Thu, Jul 28, 2011 at 08:25:39PM +0300, Kostik Belousov wrote: > > On Thu, Jul 28, 2011 at 06:07:52PM +0200, Pawel Jakub Dawidek wrote: > > > In you patch you depend on fact that full path to mount directory is > > > passed to the nmount(2) system call. This doesn't have to be true. > > > I changed mount(8) to call realpath(3) in mount directory, but I see = no > > > reason someone calling nmount(2) directly with "./foo" mount dir. > > >=20 > > > I think the proper way is to build full path from within the kernel > > > using vn_fullpath_global(). > >=20 > > It indeed may work if the supplied vnode is a directory, since it > > will fall back to the dotdot lookup loop if the namecache is purged. >=20 > Exactly. Mount directory by definition is always a directory, so it > should be reliable. It is so on FreeBSD, but is not true for a random Unix in existence. E.g., Solaris does allow to mount over the file, and it is useful. --0N88lRoFd5qAfd1F Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (FreeBSD) iEYEARECAAYFAk4xn1UACgkQC3+MBN1Mb4hgAwCghciKG94NNKKnyHc/bbycCrwa adUAniMwbywpnJO16fpRpFFPXUeuX885 =+BhQ -----END PGP SIGNATURE----- --0N88lRoFd5qAfd1F-- From owner-zfs-devel@FreeBSD.ORG Fri Jul 29 10:30:33 2011 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3938C106566C; Fri, 29 Jul 2011 10:30:33 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from mail.vx.sk (mail.vx.sk [IPv6:2a01:4f8:100:1043::3]) by mx1.freebsd.org (Postfix) with ESMTP id 8D4CF8FC0C; Fri, 29 Jul 2011 10:30:32 +0000 (UTC) Received: from core.vx.sk (localhost [127.0.0.1]) by mail.vx.sk (Postfix) with ESMTP id AA870145301; Fri, 29 Jul 2011 12:30:31 +0200 (CEST) X-Virus-Scanned: amavisd-new at mail.vx.sk Received: from mail.vx.sk ([127.0.0.1]) by core.vx.sk (mail.vx.sk [127.0.0.1]) (amavisd-new, port 10024) with LMTP id oMZ6YuP4141m; Fri, 29 Jul 2011 12:30:29 +0200 (CEST) Received: from [10.9.8.1] (chello085216231078.chello.sk [85.216.231.78]) by mail.vx.sk (Postfix) with ESMTPSA id 3909C1452DB; Fri, 29 Jul 2011 12:30:26 +0200 (CEST) Message-ID: <4E328BC5.5030001@FreeBSD.org> Date: Fri, 29 Jul 2011 12:30:29 +0200 From: Martin Matuska User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:5.0) Gecko/20110624 Thunderbird/5.0 MIME-Version: 1.0 To: Pawel Jakub Dawidek References: <4E30704C.9070606@FreeBSD.org> <20110728160751.GA1704@garage.freebsd.pl> <20110728172539.GU17489@deviant.kiev.zoral.com.ua> <20110728173811.GB1704@garage.freebsd.pl> In-Reply-To: <20110728173811.GB1704@garage.freebsd.pl> X-Enigmail-Version: 1.2 Content-Type: multipart/mixed; boundary="------------020101090001040705070308" Cc: Kostik Belousov , zfs-devel@freebsd.org Subject: Re: [CFR] [PATCH] jail mount/unmount patch X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 29 Jul 2011 10:30:33 -0000 This is a multi-part message in MIME format. --------------020101090001040705070308 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Dňa 28. 7. 2011 19:38, Pawel Jakub Dawidek wrote / napísal(a): > On Thu, Jul 28, 2011 at 08:25:39PM +0300, Kostik Belousov wrote: >> On Thu, Jul 28, 2011 at 06:07:52PM +0200, Pawel Jakub Dawidek wrote: >>> In you patch you depend on fact that full path to mount directory is >>> passed to the nmount(2) system call. This doesn't have to be true. >>> I changed mount(8) to call realpath(3) in mount directory, but I see no >>> reason someone calling nmount(2) directly with "./foo" mount dir. >>> >>> I think the proper way is to build full path from within the kernel >>> using vn_fullpath_global(). >> It indeed may work if the supplied vnode is a directory, since it >> will fall back to the dotdot lookup loop if the namecache is purged. > Exactly. Mount directory by definition is always a directory, so it > should be reliable. > I have reworked and tested the patch to use vn_fullpath_global(), patch attached. The vp to global path translation is done in vfs_domount_first() (static function) that doesn't take a fspath argument anymore. Please review, comment and/or test my attached patch. -- Martin Matuska FreeBSD committer http://blog.vx.sk --------------020101090001040705070308 Content-Type: text/plain; name="jail_mount.patch" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="jail_mount.patch" SW5kZXg6IHN5cy9rZXJuL2tlcm5famFpbC5jCj09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT0KLS0tIHN5cy9rZXJu L2tlcm5famFpbC5jCShyZXZpc2lvbiAyMjQ0NzEpCisrKyBzeXMva2Vybi9rZXJuX2phaWwu Ywkod29ya2luZyBjb3B5KQpAQCAtMzg1OCw3ICszODU4LDggQEAKIAljYXNlIFBSSVZfVkZT X1VOTU9VTlQ6CiAJY2FzZSBQUklWX1ZGU19NT1VOVF9OT05VU0VSOgogCWNhc2UgUFJJVl9W RlNfTU9VTlRfT1dORVI6Ci0JCWlmIChjcmVkLT5jcl9wcmlzb24tPnByX2FsbG93ICYgUFJf QUxMT1dfTU9VTlQpCisJCWlmIChjcmVkLT5jcl9wcmlzb24tPnByX2FsbG93ICYgUFJfQUxM T1dfTU9VTlQgJiYKKwkJCWNyZWQtPmNyX3ByaXNvbi0+cHJfZW5mb3JjZV9zdGF0ZnMgIT0g MikKIAkJCXJldHVybiAoMCk7CiAJCWVsc2UKIAkJCXJldHVybiAoRVBFUk0pOwpJbmRleDog c3lzL2tlcm4vdmZzX21vdW50LmMKPT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PQotLS0gc3lzL2tlcm4vdmZzX21v dW50LmMJKHJldmlzaW9uIDIyNDQ3MSkKKysrIHN5cy9rZXJuL3Zmc19tb3VudC5jCSh3b3Jr aW5nIGNvcHkpCkBAIC03NDUsNyArNzQ1LDYgQEAKIHZmc19kb21vdW50X2ZpcnN0KAogCXN0 cnVjdCB0aHJlYWQgKnRkLAkJLyogQ2FsbGluZyB0aHJlYWQuICovCiAJc3RydWN0IHZmc2Nv bmYgKnZmc3AsCQkvKiBGaWxlIHN5c3RlbSB0eXBlLiAqLwotCWNoYXIgKmZzcGF0aCwJCQkv KiBNb3VudCBwYXRoLiAqLwogCXN0cnVjdCB2bm9kZSAqdnAsCQkvKiBWbm9kZSB0byBiZSBj b3ZlcmVkLiAqLwogCWludCBmc2ZsYWdzLAkJCS8qIEZsYWdzIGNvbW1vbiB0byBhbGwgZmls ZXN5c3RlbXMuICovCiAJc3RydWN0IHZmc29wdGxpc3QgKipvcHRsaXN0CS8qIE9wdGlvbnMg bG9jYWwgdG8gdGhlIGZpbGVzeXN0ZW0uICovCkBAIC03NTUsMTEgKzc1NCwyMyBAQAogCXN0 cnVjdCBtb3VudCAqbXA7CiAJc3RydWN0IHZub2RlICpuZXdkcDsKIAlpbnQgZXJyb3I7CisJ Y2hhciAqZnNwYXRoLCAqZmJ1ZjsKIAogCW10eF9hc3NlcnQoJkdpYW50LCBNQV9PV05FRCk7 CiAJQVNTRVJUX1ZPUF9FTE9DS0VEKHZwLCBfX2Z1bmNfXyk7CiAJS0FTU0VSVCgoZnNmbGFn cyAmIE1OVF9VUERBVEUpID09IDAsICgiTU5UX1VQREFURSBzaG91bGRuJ3QgYmUgaGVyZSIp KTsKIAorCS8qIENvbnN0cnVjdCBnbG9iYWwgZmlsZXN5c3RlbSBwYXRoIGZyb20gdm5vZGUg Ki8KKwllcnJvciA9IHZuX2Z1bGxwYXRoX2dsb2JhbCh0ZCwgdnAsICZmc3BhdGgsICZmYnVm KTsKKwlpZiAoZXJyb3IgPT0gMCAmJiBzdHJsZW4oZnNwYXRoKSA+PSBNTkFNRUxFTikKKwkJ ZXJyb3IgPSBFTkFNRVRPT0xPTkc7CisJaWYgKGVycm9yICE9IDApIHsKKwkJdnB1dCh2cCk7 CisJCWlmIChmYnVmICE9IE5VTEwpCisJCQlmcmVlKGZidWYsIE1fVEVNUCk7CisJCXJldHVy bihlcnJvcik7CisJfQorCiAJLyoKIAkgKiBJZiB0aGUgdXNlciBpcyBub3Qgcm9vdCwgZW5z dXJlIHRoYXQgdGhleSBvd24gdGhlIGRpcmVjdG9yeQogCSAqIG9udG8gd2hpY2ggd2UgYXJl IGF0dGVtcHRpbmcgdG8gbW91bnQuCkBAIC03ODEsMTIgKzc5MiwxNCBAQAogCX0KIAlpZiAo ZXJyb3IgIT0gMCkgewogCQl2cHV0KHZwKTsKKwkJZnJlZShmYnVmLCBNX1RFTVApOwogCQly ZXR1cm4gKGVycm9yKTsKIAl9CiAJVk9QX1VOTE9DSyh2cCwgMCk7CiAKIAkvKiBBbGxvY2F0 ZSBhbmQgaW5pdGlhbGl6ZSB0aGUgZmlsZXN5c3RlbS4gKi8KIAltcCA9IHZmc19tb3VudF9h bGxvYyh2cCwgdmZzcCwgZnNwYXRoLCB0ZC0+dGRfdWNyZWQpOworCWZyZWUoZmJ1ZiwgTV9U RU1QKTsKIAkvKiBYWFhNQUM6IHBhc3MgdG8gdmZzX21vdW50X2FsbG9jPyAqLwogCW1wLT5t bnRfb3B0bmV3ID0gKm9wdGxpc3Q7CiAJLyogU2V0IHRoZSBtb3VudCBsZXZlbCBmbGFncy4g Ki8KQEAgLTEwNzAsNyArMTA4Myw3IEBACiAJTkRGUkVFKCZuZCwgTkRGX09OTFlfUE5CVUYp OwogCXZwID0gbmQubmlfdnA7CiAJaWYgKChmc2ZsYWdzICYgTU5UX1VQREFURSkgPT0gMCkg ewotCQllcnJvciA9IHZmc19kb21vdW50X2ZpcnN0KHRkLCB2ZnNwLCBmc3BhdGgsIHZwLCBm c2ZsYWdzLAorCQllcnJvciA9IHZmc19kb21vdW50X2ZpcnN0KHRkLCB2ZnNwLCB2cCwgZnNm bGFncywKIAkJICAgIG9wdGxpc3QpOwogCX0gZWxzZSB7CiAJCWVycm9yID0gdmZzX2RvbW91 bnRfdXBkYXRlKHRkLCB2cCwgZnNmbGFncywgb3B0bGlzdCk7CkBAIC0xMTA1LDcgKzExMTgs NyBAQAogCX0gKi8gKnVhcDsKIHsKIAlzdHJ1Y3QgbW91bnQgKm1wOwotCWNoYXIgKnBhdGhi dWY7CisJY2hhciAqcGF0aGJ1ZiwgKnJwYXRoYnVmOwogCWludCBlcnJvciwgaWQwLCBpZDE7 CiAKIAlBVURJVF9BUkdfVkFMVUUodWFwLT5mbGFncyk7CkBAIC0xMTM5LDEzICsxMTUyLDI4 IEBACiAJCX0KIAkJbXR4X3VubG9jaygmbW91bnRsaXN0X210eCk7CiAJfSBlbHNlIHsKLQkJ QVVESVRfQVJHX1VQQVRIMSh0ZCwgcGF0aGJ1Zik7CisJCS8qCisJCSAqIElmIHdlIGFyZSBq YWlsZWQgYW5kIGVuZm9yY2Vfc3RhdGZzID09IDEKKwkJICogY29uc3RydWN0IHJlYWwgZmls ZXN5c3RlbSBwYXRoCisJCSAqLworCQlycGF0aGJ1ZiA9IG1hbGxvYyhNTkFNRUxFTiwgTV9U RU1QLCBNX1dBSVRPSyk7CisJCWlmIChqYWlsZWQodGQtPnRkX3VjcmVkKSAmJgorCQkgICAg dGQtPnRkX3VjcmVkLT5jcl9wcmlzb24tPnByX2VuZm9yY2Vfc3RhdGZzID09IDEgJiYKKwkJ ICAgIHN0cmNtcCh0ZC0+dGRfdWNyZWQtPmNyX3ByaXNvbi0+cHJfcGF0aCwgIi8iKSAhPSAw KSB7CisJCQlzdHJsY3B5KHJwYXRoYnVmLCB0ZC0+dGRfdWNyZWQtPmNyX3ByaXNvbi0+cHJf cGF0aCwKKwkJCSAgICBNTkFNRUxFTik7CisJCQlzdHJsY2F0KHJwYXRoYnVmLCBwYXRoYnVm LCBNTkFNRUxFTik7CisJCX0gZWxzZSB7CisJCQlzdHJsY3B5KHJwYXRoYnVmLCBwYXRoYnVm LCBNTkFNRUxFTik7CisJCX0KKwkJQVVESVRfQVJHX1VQQVRIMSh0ZCwgcnBhdGhidWYpOwog CQltdHhfbG9jaygmbW91bnRsaXN0X210eCk7CiAJCVRBSUxRX0ZPUkVBQ0hfUkVWRVJTRSht cCwgJm1vdW50bGlzdCwgbW50bGlzdCwgbW50X2xpc3QpIHsKLQkJCWlmIChzdHJjbXAobXAt Pm1udF9zdGF0LmZfbW50b25uYW1lLCBwYXRoYnVmKSA9PSAwKQorCQkJaWYgKHN0cmNtcCht cC0+bW50X3N0YXQuZl9tbnRvbm5hbWUsIHJwYXRoYnVmKSA9PSAwKQogCQkJCWJyZWFrOwog CQl9CiAJCW10eF91bmxvY2soJm1vdW50bGlzdF9tdHgpOworCQlmcmVlKHJwYXRoYnVmLCBN X1RFTVApOwogCX0KIAlmcmVlKHBhdGhidWYsIE1fVEVNUCk7CiAJaWYgKG1wID09IE5VTEwp IHsK --------------020101090001040705070308-- From owner-zfs-devel@FreeBSD.ORG Fri Jul 29 18:32:41 2011 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1FEAB106566C for ; Fri, 29 Jul 2011 18:32:41 +0000 (UTC) (envelope-from will@firepipe.net) Received: from mail-wy0-f182.google.com (mail-wy0-f182.google.com [74.125.82.182]) by mx1.freebsd.org (Postfix) with ESMTP id B48588FC08 for ; Fri, 29 Jul 2011 18:32:40 +0000 (UTC) Received: by wyg24 with SMTP id 24so516515wyg.13 for ; Fri, 29 Jul 2011 11:32:39 -0700 (PDT) MIME-Version: 1.0 Received: by 10.227.203.10 with SMTP id fg10mr2248017wbb.102.1311963049826; Fri, 29 Jul 2011 11:10:49 -0700 (PDT) Received: by 10.216.45.77 with HTTP; Fri, 29 Jul 2011 11:10:49 -0700 (PDT) Date: Fri, 29 Jul 2011 12:10:49 -0600 Message-ID: From: Will Andrews To: zfs-devel@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 Subject: ZFS sharenfs mangling X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 29 Jul 2011 18:32:41 -0000 Hi, Why do the FreeBSD fsshare compatibility routines mangle the sharenfs value? As best as I can tell, this behavior only exists so that people don't have to quote their sharenfs value when setting it via 'zfs set'. However, this feature breaks using hyphens anywhere but in options. An user reported this problem last month: http://lists.freebsd.org/pipermail/freebsd-fs/2011-June/011784.html Removing translate_opts() (and the call from fsshare_main()) from cddl/compat/opensolaris/misc/fsshare.c fixed the problem for me, and had the desired result. The function could be tweaked to fix this problem, but I think it would be better to simply remove it. --Will. From owner-zfs-devel@FreeBSD.ORG Sun Jul 31 22:35:38 2011 Return-Path: Delivered-To: zfs-devel@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2718B106566C; Sun, 31 Jul 2011 22:35:38 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from mail.vx.sk (mail.vx.sk [IPv6:2a01:4f8:100:1043::3]) by mx1.freebsd.org (Postfix) with ESMTP id DC34F8FC0A; Sun, 31 Jul 2011 22:35:37 +0000 (UTC) Received: from core.vx.sk (localhost [127.0.0.1]) by mail.vx.sk (Postfix) with ESMTP id 905DB1686D0; Mon, 1 Aug 2011 00:35:36 +0200 (CEST) X-Virus-Scanned: amavisd-new at mail.vx.sk Received: from mail.vx.sk ([127.0.0.1]) by core.vx.sk (mail.vx.sk [127.0.0.1]) (amavisd-new, port 10024) with LMTP id jI70UDYBgP-r; Mon, 1 Aug 2011 00:35:34 +0200 (CEST) Received: from [10.9.8.3] (chello085216231078.chello.sk [85.216.231.78]) by mail.vx.sk (Postfix) with ESMTPSA id 116941686C0; Mon, 1 Aug 2011 00:35:34 +0200 (CEST) Message-ID: <4E35D8B5.5080100@FreeBSD.org> Date: Mon, 01 Aug 2011 00:35:33 +0200 From: Martin Matuska User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:5.0) Gecko/20110624 Thunderbird/5.0 MIME-Version: 1.0 To: Pawel Jakub Dawidek X-Enigmail-Version: 1.2 Content-Type: multipart/mixed; boundary="------------090600050800050909060601" Cc: zfs-devel@FreeBSD.ORG Subject: [PATCH] fix integer overflow in txg_delay() X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 31 Jul 2011 22:35:38 -0000 This is a multi-part message in MIME format. --------------090600050800050909060601 Content-Type: text/plain; charset=windows-1250 Content-Transfer-Encoding: 7bit The txg_delay() function in txg.c uses the following initialization: int timeout = ddi_get_lbolt() + ticks; Later, we have: while (ddi_get_lbolt() < timeout && tx->tx_syncing_txg < txg-1 && !txg_stalled(dp)) (void) cv_timedwait(&tx->tx_quiesce_more_cv, &tx->tx_sync_lock, timeout - ddi_get_lbolt()); The function txg_delay() is called from: dsl_pool_tempreserve_space() and dsl_dir_tempreserve_space() In 24.855 days ddi_get_lbolt will be never smaller than timeout. Please review and/or comment the attached patch. -- Martin Matuska FreeBSD committer http://blog.vx.sk --------------090600050800050909060601 Content-Type: text/plain; name="txg.c.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="txg.c.patch" Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/txg.c =================================================================== --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/txg.c (revision 224527) +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/txg.c (working copy) @@ -488,7 +488,7 @@ txg_delay(dsl_pool_t *dp, uint64_t txg, int ticks) { tx_state_t *tx = &dp->dp_tx; - int timeout = ddi_get_lbolt() + ticks; + clock_t timeout = ddi_get_lbolt() + ticks; /* don't delay if this txg could transition to quiesing immediately */ if (tx->tx_open_txg > txg || --------------090600050800050909060601-- From owner-zfs-devel@FreeBSD.ORG Mon Aug 1 13:35:57 2011 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 44647106564A for ; Mon, 1 Aug 2011 13:35:57 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 947808FC14 for ; Mon, 1 Aug 2011 13:35:56 +0000 (UTC) Received: from odyssey.starpoint.kiev.ua (alpha-e.starpoint.kiev.ua [212.40.38.101]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id QAA26807; Mon, 01 Aug 2011 16:20:15 +0300 (EEST) (envelope-from avg@FreeBSD.org) Message-ID: <4E36A80E.8080909@FreeBSD.org> Date: Mon, 01 Aug 2011 16:20:14 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:5.0) Gecko/20110705 Thunderbird/5.0 MIME-Version: 1.0 To: Martin Matuska References: <4E35D8B5.5080100@FreeBSD.org> In-Reply-To: <4E35D8B5.5080100@FreeBSD.org> X-Enigmail-Version: 1.2pre Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Cc: zfs-devel@FreeBSD.org, Pawel Jakub Dawidek Subject: Re: [PATCH] fix integer overflow in txg_delay() X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Aug 2011 13:35:57 -0000 on 01/08/2011 01:35 Martin Matuska said the following: > The txg_delay() function in txg.c uses the following initialization: > int timeout = ddi_get_lbolt() + ticks; > > Later, we have: > while (ddi_get_lbolt() < timeout && > tx->tx_syncing_txg < txg-1 && !txg_stalled(dp)) > (void) cv_timedwait(&tx->tx_quiesce_more_cv, > &tx->tx_sync_lock, > timeout - ddi_get_lbolt()); > > The function txg_delay() is called from: > dsl_pool_tempreserve_space() and dsl_dir_tempreserve_space() > > In 24.855 days ddi_get_lbolt will be never smaller than timeout. > > Please review and/or comment the attached patch. > I agree with the patch - thank you for catching this bug! -- Andriy Gapon From owner-zfs-devel@FreeBSD.ORG Tue Aug 2 05:33:34 2011 Return-Path: Delivered-To: zfs-devel@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B4CB0106564A; Tue, 2 Aug 2011 05:33:34 +0000 (UTC) (envelope-from pawel@dawidek.net) Received: from mail.dawidek.net (60.wheelsystems.com [83.12.187.60]) by mx1.freebsd.org (Postfix) with ESMTP id 67C208FC08; Tue, 2 Aug 2011 05:33:34 +0000 (UTC) Received: from localhost (89-73-195-149.dynamic.chello.pl [89.73.195.149]) by mail.dawidek.net (Postfix) with ESMTPSA id 93CA0F81; Tue, 2 Aug 2011 07:33:31 +0200 (CEST) Date: Tue, 2 Aug 2011 07:33:24 +0200 From: Pawel Jakub Dawidek To: Martin Matuska Message-ID: <20110802053324.GG1685@garage.freebsd.pl> References: <4E35D8B5.5080100@FreeBSD.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="rV8arf8D5Dod9UkK" Content-Disposition: inline In-Reply-To: <4E35D8B5.5080100@FreeBSD.org> X-OS: FreeBSD 9.0-CURRENT amd64 User-Agent: Mutt/1.5.21 (2010-09-15) Cc: zfs-devel@FreeBSD.ORG Subject: Re: [PATCH] fix integer overflow in txg_delay() X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Aug 2011 05:33:34 -0000 --rV8arf8D5Dod9UkK Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Mon, Aug 01, 2011 at 12:35:33AM +0200, Martin Matuska wrote: > The txg_delay() function in txg.c uses the following initialization: > int timeout =3D ddi_get_lbolt() + ticks; >=20 > Later, we have: > while (ddi_get_lbolt() < timeout && > tx->tx_syncing_txg < txg-1 && !txg_stalled(dp)) > (void) cv_timedwait(&tx->tx_quiesce_more_cv, > &tx->tx_sync_lock, > timeout - ddi_get_lbolt()); >=20 > The function txg_delay() is called from: > dsl_pool_tempreserve_space() and dsl_dir_tempreserve_space() >=20 > In 24.855 days ddi_get_lbolt will be never smaller than timeout. >=20 > Please review and/or comment the attached patch. Looks good to me. Can you elaborate a bit on consequences of such overflow? How the problem manifests itself? BTW. Is this something that affects IllumOS as well? If so, it would be nice to share with them. > Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/txg.c > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/txg.c (revision 224527) > +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/txg.c (working copy) > @@ -488,7 +488,7 @@ > txg_delay(dsl_pool_t *dp, uint64_t txg, int ticks) > { > tx_state_t *tx =3D &dp->dp_tx; > - int timeout =3D ddi_get_lbolt() + ticks; > + clock_t timeout =3D ddi_get_lbolt() + ticks; > =20 > /* don't delay if this txg could transition to quiesing immediately */ > if (tx->tx_open_txg > txg || --=20 Pawel Jakub Dawidek http://www.wheelsystems.com FreeBSD committer http://www.FreeBSD.org Am I Evil? Yes, I Am! http://yomoli.com --rV8arf8D5Dod9UkK Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (FreeBSD) iEYEARECAAYFAk43jCMACgkQForvXbEpPzQLeQCg3acNcUSKsXeyCcBQboKhvrf0 nzkAnjvW7+AAoxKN+wcC4D46PKjYoseq =6STl -----END PGP SIGNATURE----- --rV8arf8D5Dod9UkK-- From owner-zfs-devel@FreeBSD.ORG Tue Aug 2 06:06:20 2011 Return-Path: Delivered-To: zfs-devel@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id F365E106564A; Tue, 2 Aug 2011 06:06:19 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from mail.vx.sk (mail.vx.sk [IPv6:2a01:4f8:100:1043::3]) by mx1.freebsd.org (Postfix) with ESMTP id 8DD628FC12; Tue, 2 Aug 2011 06:06:19 +0000 (UTC) Received: from core.vx.sk (localhost [127.0.0.1]) by mail.vx.sk (Postfix) with ESMTP id B012B14BF3A; Tue, 2 Aug 2011 08:06:17 +0200 (CEST) X-Virus-Scanned: amavisd-new at mail.vx.sk Received: from mail.vx.sk ([127.0.0.1]) by core.vx.sk (mail.vx.sk [127.0.0.1]) (amavisd-new, port 10024) with LMTP id WaZ+e4TDf84E; Tue, 2 Aug 2011 08:06:15 +0200 (CEST) Received: from [10.9.8.1] (chello085216231078.chello.sk [85.216.231.78]) by mail.vx.sk (Postfix) with ESMTPSA id 5D4D514BF2B; Tue, 2 Aug 2011 08:06:15 +0200 (CEST) Message-ID: <4E3793DA.7090506@FreeBSD.org> Date: Tue, 02 Aug 2011 08:06:18 +0200 From: Martin Matuska User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:5.0) Gecko/20110624 Thunderbird/5.0 MIME-Version: 1.0 To: Pawel Jakub Dawidek References: <4E35D8B5.5080100@FreeBSD.org> <20110802053324.GG1685@garage.freebsd.pl> In-Reply-To: <20110802053324.GG1685@garage.freebsd.pl> X-Enigmail-Version: 1.2 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: zfs-devel@FreeBSD.ORG Subject: Re: [PATCH] fix integer overflow in txg_delay() X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Aug 2011 06:06:20 -0000 Dňa 2. 8. 2011 7:33, Pawel Jakub Dawidek wrote / napísal(a): > On Mon, Aug 01, 2011 at 12:35:33AM +0200, Martin Matuska wrote: >> The txg_delay() function in txg.c uses the following initialization: >> int timeout = ddi_get_lbolt() + ticks; >> >> Later, we have: >> while (ddi_get_lbolt() < timeout && >> tx->tx_syncing_txg < txg-1 && !txg_stalled(dp)) >> (void) cv_timedwait(&tx->tx_quiesce_more_cv, >> &tx->tx_sync_lock, >> timeout - ddi_get_lbolt()); >> >> The function txg_delay() is called from: >> dsl_pool_tempreserve_space() and dsl_dir_tempreserve_space() >> >> In 24.855 days ddi_get_lbolt will be never smaller than timeout. >> >> Please review and/or comment the attached patch. > Looks good to me. Can you elaborate a bit on consequences of such > overflow? How the problem manifests itself? > > BTW. Is this something that affects IllumOS as well? If so, it would be > nice to share with them. > >> Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/txg.c >> =================================================================== >> --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/txg.c (revision 224527) >> +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/txg.c (working copy) >> @@ -488,7 +488,7 @@ >> txg_delay(dsl_pool_t *dp, uint64_t txg, int ticks) >> { >> tx_state_t *tx = &dp->dp_tx; >> - int timeout = ddi_get_lbolt() + ticks; >> + clock_t timeout = ddi_get_lbolt() + ticks; >> >> /* don't delay if this txg could transition to quiesing immediately */ >> if (tx->tx_open_txg > txg || The overflow causes txg_delay() not delaying txg threads anymore. It is called from dsl_pool_tempreserve_space() in dsl_pool.c and from dsl_dir_tempreserve_space() in dsl_dir.c from busy transaction groups: dsl_pool.c: /* * If this transaction group is over 7/8ths capacity, delay * the caller 1 clock tick. This will slow down the "fill" * rate until the sync process can catch up with us. */ if (reserved && reserved > (write_limit - (write_limit >> 3))) txg_delay(dp, tx->tx_txg, 1); dsl_dir.c: err = arc_tempreserve_space(lsize, tx->tx_txg); if (err == 0) { struct tempreserve *tr; tr = kmem_zalloc(sizeof (struct tempreserve), KM_SLEEP); tr->tr_size = lsize; list_insert_tail(tr_list, tr); err = dsl_pool_tempreserve_space(dd->dd_pool, asize, tx); } else { if (err == EAGAIN) { txg_delay(dd->dd_pool, tx->tx_txg, 1); err = ERESTART; } dsl_pool_memory_pressure(dd->dd_pool); } In my interpretation, this overflow will increase thread concurrency and memory pressure on the pool and therefore slow down write speed. I have filed this yesterday as Illumos bug #1313. https://www.illumos.org/issues/1313 -- Martin Matuska FreeBSD committer http://blog.vx.sk From owner-zfs-devel@FreeBSD.ORG Tue Aug 2 06:30:15 2011 Return-Path: Delivered-To: zfs-devel@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id EA7B0106564A; Tue, 2 Aug 2011 06:30:14 +0000 (UTC) (envelope-from pawel@dawidek.net) Received: from mail.dawidek.net (60.wheelsystems.com [83.12.187.60]) by mx1.freebsd.org (Postfix) with ESMTP id 684948FC13; Tue, 2 Aug 2011 06:30:14 +0000 (UTC) Received: from localhost (58.wheelsystems.com [83.12.187.58]) by mail.dawidek.net (Postfix) with ESMTPSA id D5E43FA3; Tue, 2 Aug 2011 08:30:12 +0200 (CEST) Date: Tue, 2 Aug 2011 08:30:07 +0200 From: Pawel Jakub Dawidek To: Martin Matuska Message-ID: <20110802063007.GB2789@garage.freebsd.pl> References: <4E35D8B5.5080100@FreeBSD.org> <20110802053324.GG1685@garage.freebsd.pl> <4E3793DA.7090506@FreeBSD.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="hHWLQfXTYDoKhP50" Content-Disposition: inline In-Reply-To: <4E3793DA.7090506@FreeBSD.org> X-OS: FreeBSD 9.0-CURRENT amd64 User-Agent: Mutt/1.5.21 (2010-09-15) Cc: zfs-devel@FreeBSD.ORG Subject: Re: [PATCH] fix integer overflow in txg_delay() X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Aug 2011 06:30:15 -0000 --hHWLQfXTYDoKhP50 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Tue, Aug 02, 2011 at 08:06:18AM +0200, Martin Matuska wrote: > D=C5=88a 2. 8. 2011 7:33, Pawel Jakub Dawidek wrote / nap=C3=ADsal(a): > > On Mon, Aug 01, 2011 at 12:35:33AM +0200, Martin Matuska wrote: > >> The txg_delay() function in txg.c uses the following initialization: > >> int timeout =3D ddi_get_lbolt() + ticks; > >> > >> Later, we have: > >> while (ddi_get_lbolt() < timeout && > >> tx->tx_syncing_txg < txg-1 && !txg_stalled(dp)) > >> (void) cv_timedwait(&tx->tx_quiesce_more_cv, > >> &tx->tx_sync_lock, > >> timeout - ddi_get_lbolt()); > >> > >> The function txg_delay() is called from: > >> dsl_pool_tempreserve_space() and dsl_dir_tempreserve_space() > >> > >> In 24.855 days ddi_get_lbolt will be never smaller than timeout. > >> > >> Please review and/or comment the attached patch. > > Looks good to me. Can you elaborate a bit on consequences of such > > overflow? How the problem manifests itself? > > > > BTW. Is this something that affects IllumOS as well? If so, it would be > > nice to share with them. > > > >> Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/txg.c > >> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > >> --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/txg.c (revision 224= 527) > >> +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/txg.c (working copy) > >> @@ -488,7 +488,7 @@ > >> txg_delay(dsl_pool_t *dp, uint64_t txg, int ticks) > >> { > >> tx_state_t *tx =3D &dp->dp_tx; > >> - int timeout =3D ddi_get_lbolt() + ticks; > >> + clock_t timeout =3D ddi_get_lbolt() + ticks; > >> =20 > >> /* don't delay if this txg could transition to quiesing immediately = */ > >> if (tx->tx_open_txg > txg || > The overflow causes txg_delay() not delaying txg threads anymore. > It is called from dsl_pool_tempreserve_space() in dsl_pool.c and from > dsl_dir_tempreserve_space() in dsl_dir.c from busy transaction groups: >=20 > dsl_pool.c: > /* > * If this transaction group is over 7/8ths capacity, delay > * the caller 1 clock tick. This will slow down the "fill" > * rate until the sync process can catch up with us. > */ > if (reserved && reserved > (write_limit - (write_limit >> 3))) > txg_delay(dp, tx->tx_txg, 1); >=20 > dsl_dir.c: > err =3D arc_tempreserve_space(lsize, tx->tx_txg); > if (err =3D=3D 0) { > struct tempreserve *tr; >=20 > tr =3D kmem_zalloc(sizeof (struct tempreserve), KM_SLEEP); > tr->tr_size =3D lsize; > list_insert_tail(tr_list, tr); >=20 > err =3D dsl_pool_tempreserve_space(dd->dd_pool, asize, tx); > } else { > if (err =3D=3D EAGAIN) { > txg_delay(dd->dd_pool, tx->tx_txg, 1); > err =3D ERESTART; > } > dsl_pool_memory_pressure(dd->dd_pool); > } >=20 > In my interpretation, this overflow will increase thread concurrency and > memory pressure on the pool and therefore slow down write speed. >=20 > I have filed this yesterday as Illumos bug #1313. > https://www.illumos.org/issues/1313 Ok, thanks. --=20 Pawel Jakub Dawidek http://www.wheelsystems.com FreeBSD committer http://www.FreeBSD.org Am I Evil? Yes, I Am! http://yomoli.com --hHWLQfXTYDoKhP50 Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (FreeBSD) iEYEARECAAYFAk43mW8ACgkQForvXbEpPzSlqACgpZuBy5otS4BHXvp315Z3nx75 VY4An05rl8HVk7heeEDrMqde6XqA8puW =maWI -----END PGP SIGNATURE----- --hHWLQfXTYDoKhP50-- From owner-zfs-devel@FreeBSD.ORG Tue Aug 2 06:39:06 2011 Return-Path: Delivered-To: zfs-devel@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 44ACD1065670; Tue, 2 Aug 2011 06:39:06 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from mail.vx.sk (mail.vx.sk [IPv6:2a01:4f8:100:1043::3]) by mx1.freebsd.org (Postfix) with ESMTP id B979B8FC12; Tue, 2 Aug 2011 06:39:05 +0000 (UTC) Received: from core.vx.sk (localhost [127.0.0.1]) by mail.vx.sk (Postfix) with ESMTP id 2A95114DF6F; Tue, 2 Aug 2011 08:39:05 +0200 (CEST) X-Virus-Scanned: amavisd-new at mail.vx.sk Received: from mail.vx.sk ([127.0.0.1]) by core.vx.sk (mail.vx.sk [127.0.0.1]) (amavisd-new, port 10024) with LMTP id mQD208TISvo9; Tue, 2 Aug 2011 08:39:02 +0200 (CEST) Received: from [10.9.8.1] (chello085216231078.chello.sk [85.216.231.78]) by mail.vx.sk (Postfix) with ESMTPSA id 94C8C14DF4E; Tue, 2 Aug 2011 08:39:02 +0200 (CEST) Message-ID: <4E379B89.9040409@FreeBSD.org> Date: Tue, 02 Aug 2011 08:39:05 +0200 From: Martin Matuska User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:5.0) Gecko/20110624 Thunderbird/5.0 MIME-Version: 1.0 To: Pawel Jakub Dawidek , zfs-devel@FreeBSD.ORG References: <20110802044718.GA8838@gw.zagrebin.ru> In-Reply-To: <20110802044718.GA8838@gw.zagrebin.ru> X-Enigmail-Version: 1.2 X-Forwarded-Message-Id: <20110802044718.GA8838@gw.zagrebin.ru> Content-Type: multipart/mixed; boundary="------------000101040805060109030304" X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: Subject: Fwd: ZFS v28: kernel panics while reading an extended attribute X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Aug 2011 06:39:06 -0000 This is a multi-part message in MIME format. --------------000101040805060109030304 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit I can confirm the bug found by Alexander Zagrebin in ZFS v28, he suggests the attached bugfix. Is the patch acceptable? How to reproduce the bug: setextattr user testattr 1 testfile zfs snapshot test@s1 getextattr user testattr .zfs/snapshot/s1/testfile PANIC ----- Original message ----- Subject: ZFS v28: kernel panics while reading an extended attribute Date: Tue, 2 Aug 2011 08:47:19 +0400 From: Alexander Zagrebin To: freebsd-fs@freebsd.org, freebsd-stable@freebsd.org Cc: freebsd-current@freebsd.org Hi! It seems, I've found a bug in the ZFS v28 on the latest stable: if we have a snapshot with some files having an extended attributes, then attempt to read an extended attributes's value leads to a well reproducible kernel panic. The part of backtrace follows: #6 0xffffffff804bbe44 in calltrap () at /usr/src/sys/amd64/amd64/exception.S:228 #7 0xffffffff80950ea7 in zil_commit (zilog=0x0, foid=5795917) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zil.c:1497 #8 0xffffffff80979e6b in zfs_freebsd_read (ap=Variable "ap" is not available.) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c:622 #9 0xffffffff80979750 in zfs_getextattr (ap=0xffffff80dd5d8820) at vnode_if.h:384 #10 0xffffffff8038921b in extattr_get_vp (vp=0xffffff0056a01588, attrnamespace=1, attrname=0xffffff80dd5d89a0 "DOSATTRIB", data=Variable "data" is not available.) at vnode_if.h:1332 It seems that ZIL isn't available for snapshots, but zfs_freebsd_read doesn't check this when calling zil_commit. The attached patch fixes this issue. Can anybody confirm this? -- Alexander Zagrebin --------------000101040805060109030304 Content-Type: text/x-diff; name="patch-zfs_vnops.c" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="patch-zfs_vnops.c" --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c.orig 2011-08-01 23:04:07.358173627 +0400 +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c 2011-08-02 00:10:02.674585604 +0400 @@ -618,7 +618,8 @@ zfs_read(vnode_t *vp, uio_t *uio, int io /* * If we're in FRSYNC mode, sync out this znode before reading it. */ - if (ioflag & FRSYNC || zfsvfs->z_os->os_sync == ZFS_SYNC_ALWAYS) + if (zfsvfs->z_log && + (ioflag & FRSYNC || zfsvfs->z_os->os_sync == ZFS_SYNC_ALWAYS)) zil_commit(zfsvfs->z_log, zp->z_id); /* --------------000101040805060109030304 Content-Type: text/plain; name="=?windows-1250?Q?=C8as=9D_prilo=9Eenej_spr=E1vy?=" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0*=windows-1250''%C8%61%73%9D%20%70%72%69%6C%6F%9E%65%6E%65%6A%20; filename*1*=%73%70%72%E1%76%79 _______________________________________________ freebsd-fs@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-fs To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" --------------000101040805060109030304-- From owner-zfs-devel@FreeBSD.ORG Tue Aug 2 07:52:21 2011 Return-Path: Delivered-To: zfs-devel@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id AF8C5106568C; Tue, 2 Aug 2011 07:52:21 +0000 (UTC) (envelope-from pawel@dawidek.net) Received: from mail.dawidek.net (60.wheelsystems.com [83.12.187.60]) by mx1.freebsd.org (Postfix) with ESMTP id 461E08FC1D; Tue, 2 Aug 2011 07:52:21 +0000 (UTC) Received: from localhost (58.wheelsystems.com [83.12.187.58]) by mail.dawidek.net (Postfix) with ESMTPSA id EEB9EFD5; Tue, 2 Aug 2011 09:52:18 +0200 (CEST) Date: Tue, 2 Aug 2011 09:52:13 +0200 From: Pawel Jakub Dawidek To: Martin Matuska Message-ID: <20110802075211.GC2789@garage.freebsd.pl> References: <20110802044718.GA8838@gw.zagrebin.ru> <4E379B89.9040409@FreeBSD.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="ZmUaFz6apKcXQszQ" Content-Disposition: inline In-Reply-To: <4E379B89.9040409@FreeBSD.org> X-OS: FreeBSD 9.0-CURRENT amd64 User-Agent: Mutt/1.5.21 (2010-09-15) Cc: zfs-devel@FreeBSD.ORG Subject: Re: Fwd: ZFS v28: kernel panics while reading an extended attribute X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Aug 2011 07:52:23 -0000 --ZmUaFz6apKcXQszQ Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Tue, Aug 02, 2011 at 08:39:05AM +0200, Martin Matuska wrote: > I can confirm the bug found by Alexander Zagrebin in ZFS v28, he > suggests the attached bugfix. > Is the patch acceptable? >=20 > How to reproduce the bug: > setextattr user testattr 1 testfile > zfs snapshot test@s1 > getextattr user testattr .zfs/snapshot/s1/testfile > PANIC This is because zfs_getextattr() calls VOP_READ(9) with IO_SYNC flag. I was wondering if we could trigger this panic from regular read(2), but regular read never passes IO_SYNC flag. So I think this fix is correct, but I'd also remove IO_SYNC flag from VOP_READ() call inside zfs_getextattr(), as I don't see the point in having it there. --=20 Pawel Jakub Dawidek http://www.wheelsystems.com FreeBSD committer http://www.FreeBSD.org Am I Evil? Yes, I Am! http://yomoli.com --ZmUaFz6apKcXQszQ Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (FreeBSD) iEYEARECAAYFAk43rKsACgkQForvXbEpPzSoYQCfVMtC23JY/xqeoq0WT8cdgmkL oecAn3SGSEOQsCaxAyx5khIQnuHuwPmT =I4uj -----END PGP SIGNATURE----- --ZmUaFz6apKcXQszQ-- From owner-zfs-devel@FreeBSD.ORG Thu Aug 4 07:03:51 2011 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3B1F51065670 for ; Thu, 4 Aug 2011 07:03:51 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from mail.vx.sk (mail.vx.sk [IPv6:2a01:4f8:100:1043::3]) by mx1.freebsd.org (Postfix) with ESMTP id 381038FC12 for ; Thu, 4 Aug 2011 07:03:50 +0000 (UTC) Received: from core.vx.sk (localhost [127.0.0.1]) by mail.vx.sk (Postfix) with ESMTP id 5B3EB18C97B for ; Thu, 4 Aug 2011 09:03:49 +0200 (CEST) X-Virus-Scanned: amavisd-new at mail.vx.sk Received: from mail.vx.sk ([127.0.0.1]) by core.vx.sk (mail.vx.sk [127.0.0.1]) (amavisd-new, port 10024) with LMTP id EXhXDNlJcXUH for ; Thu, 4 Aug 2011 09:03:47 +0200 (CEST) Received: from [10.0.3.3] (188-167-66-148.dynamic.chello.sk [188.167.66.148]) by mail.vx.sk (Postfix) with ESMTPSA id 3D3DA18C961 for ; Thu, 4 Aug 2011 09:03:45 +0200 (CEST) Message-ID: <4E3A4450.5090100@FreeBSD.org> Date: Thu, 04 Aug 2011 09:03:44 +0200 From: Martin Matuska User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:5.0) Gecko/20110627 Thunderbird/5.0 MIME-Version: 1.0 To: zfs-devel@FreeBSD.org X-Enigmail-Version: 1.2pre Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: Subject: New ZFS changes in Illumos (13314) X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Aug 2011 07:03:51 -0000 Illumos has made another change to ZFS, this time dealing with two of their isuses: 734 taskq_dispatch_prealloc() desired 943 zio_interrupt ends up calling taskq_dispatch with TQ_SLEEP New code (changeset 13414): http://hg.openindiana.org/illumos-gate/rev/b42c1f0432b6 Would be implementing this change to our code of value? Issue 734: https://www.illumos.org/issues/734 Description: We would like an in-kernel interface for non-dynamic taskq's where the caller supplies the storage for the taskq_ent_t. The reason for this is to avoid hitting the kmem_alloc()/kmem_free() calls on very hot code paths. The other motivation (perhaps bigger) is that if we can avoid the allocation, we can avoid having to deal with scheduling failures on hot code paths, or in situations where it might not be easy to recover (e.g. interrupt context) -- by providing the storage before hand, the taskq_dispatch_prealloc() can degenerate to mutex_enter, twiddle list pointers, cv_broadcast/signal, mutex_exit. Easy peasey. Issue 943: https://www.illumos.org/issues/943 Description: While digging through bug reports and such looking for hints toward an end-user deadlock, I happened upon a report suggesting that ZFS was calling taskq_dispatch with TQ_SLEEP while in interrupt context. Code inspection appears to bear this out. The call chain ends up being: sdintr --> ... --> biodone --> vdev_disk_io_intr --> zio_interrupt --> zio_taskq_dispatch --> taskq_dispatch, where zio_taskq_dispatch explicitly ors TQ_SLEEP into any flags argument: source:usr/src/uts/common/fs/zfs/zio.c#L1056 -- Martin Matuska FreeBSD committer http://blog.vx.sk From owner-zfs-devel@FreeBSD.ORG Sun Aug 7 08:52:55 2011 Return-Path: Delivered-To: zfs-devel@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6D1A6106566C; Sun, 7 Aug 2011 08:52:55 +0000 (UTC) (envelope-from delphij@delphij.net) Received: from anubis.delphij.net (anubis.delphij.net [IPv6:2001:470:1:117::25]) by mx1.freebsd.org (Postfix) with ESMTP id 5212D8FC0C; Sun, 7 Aug 2011 08:52:55 +0000 (UTC) Received: from delta.delphij.net (unknown [IPv6:2001:470:83bf:0:221:5cff:fe6a:37bb]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by anubis.delphij.net (Postfix) with ESMTPSA id 1A5B81593A; Sun, 7 Aug 2011 01:52:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=delphij.net; s=anubis; t=1312707175; bh=reIeV/7rzxiSTk5lCdUdAGb2Zc36cv4OsBb9yvKu2+M=; h=Message-ID:Date:From:Reply-To:MIME-Version:To:CC:Subject: Content-Type; b=5HNJUIqKhsBg5+5W6tuCabd+BmeEXSAYUOYz6QCq7N49pQELFzMS+CWLnQIKtq9Wb 5H9BPCfy9u15Y3hX3I3gOXsp6MirgQmsMiRdiHhyLANlAe8y3oeWOYaPpMSDfww7Qq Rj2smTU4By41fsIPWsQf96w6A2+jWgLmwkkrGmvw= Message-ID: <4E3E525D.1040906@delphij.net> Date: Sun, 07 Aug 2011 01:52:45 -0700 From: Xin LI Organization: The FreeBSD Project MIME-Version: 1.0 To: zfs-devel@FreeBSD.ORG OpenPGP: id=3FCA37C1; url=http://www.delphij.net/delphij.asc Content-Type: multipart/mixed; boundary="------------080503040005000208080600" Cc: Josh Paetzel , Xin LI , Doug White Subject: [PATCH] L2ARC deadlock X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: d@delphij.net List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 07 Aug 2011 08:52:55 -0000 This is a multi-part message in MIME format. --------------080503040005000208080600 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 Hi, I have observed this on a test system which have both L2ARC and ZIL device installed, and several vdev groups. This can be triggered by removing the l2arc and zil device (detaching l2arc would fail), blocking at 'spa_namespace_lock' After reboot, the pool can not be imported. 'zpool import' shows the zpool command stuck with 'spa_namespace_lock'. I have a theory about the issue but the patch was not tested, maybe someone who is more familiar with the code can shed me some light? Thread 1 (zpool) Thread 2 (l2arc_feed_thread) Calls vdev_open() vdev_open() can now assert RW_WRITER on spa_config; [*1] vdev_open() calls vdev_geom_open() vdev_geom_open() called with spa_namespace_lock held, sets lock=1 and drops spa_namespace_lock, DROP_GIANT(), g_topology_lock() Races in, proceed to l2arc_dev_get_next Acquire &spa_namespace_lock; [*2] Found a device and then spa_config_enter(RW_READER) spa_config_enter blocks waiting writer to finish Tries to obtain spa_namespace_lock [*3] This creates a deadlock situation. My proposed solution is to use spa_config_tryenter() instead of spa_config_enter() in arc.c (see attachment). Comments? Cheers, - -- Xin LI https://www.delphij.net/ FreeBSD - The Power to Serve! Live free or die -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.18 (FreeBSD) iQEcBAEBCAAGBQJOPlJcAAoJEATO+BI/yjfB4UoH/j3pK5rQ4iF52HmUwzMzKKol UVXHmoUPbV1Ibhpajcz6LprSL7b/SI50a2yelIr+1ZjgC22Lrw6fFird90JfeXF7 YtQ1LyVVuDbMA5KfM6wD8linm3HYri88DQ2CDtUqZesZ1w7PH0XNYnRKRYcEGRem 1JG09BMl0EqGuvKSy+69UnE5dPTRnzrAQOe3xBB4LntZBeapwMy5F8gcBY5XP5Lh W2lBJFcdFX3Zh390fUi1dxY5uoXQLfO5gu+YCXF7Zie2PLl596ZUjAEA3CsE/9yw qWBSBGufMP4gK24j1EulxhmoIU993vGtqc3iZhs1TugfjGbHAlXiHQEcmJnaEk8= =zkTh -----END PGP SIGNATURE----- --------------080503040005000208080600 Content-Type: text/plain; name="arc.c.diff" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="arc.c.diff" SW5kZXg6IHN5cy9jZGRsL2NvbnRyaWIvb3BlbnNvbGFyaXMvdXRzL2NvbW1vbi9mcy96ZnMv YXJjLmMKPT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PQotLS0gc3lzL2NkZGwvY29udHJpYi9vcGVuc29sYXJpcy91 dHMvY29tbW9uL2ZzL3pmcy9hcmMuYwkocmV2aXNpb24gMjI0NjUyKQorKysgc3lzL2NkZGwv Y29udHJpYi9vcGVuc29sYXJpcy91dHMvY29tbW9uL2ZzL3pmcy9hcmMuYwkod29ya2luZyBj b3B5KQpAQCAtNDIyNCw4ICs0MjI0LDEwIEBAIG91dDoKIAkgKiBHcmFiIHRoZSBjb25maWcg bG9jayB0byBwcmV2ZW50IHRoZSAnbmV4dCcgZGV2aWNlIGZyb20gYmVpbmcKIAkgKiByZW1v dmVkIHdoaWxlIHdlIGFyZSB3cml0aW5nIHRvIGl0LgogCSAqLwotCWlmIChuZXh0ICE9IE5V TEwpCi0JCXNwYV9jb25maWdfZW50ZXIobmV4dC0+bDJhZF9zcGEsIFNDTF9MMkFSQywgbmV4 dCwgUldfUkVBREVSKTsKKwlpZiAobmV4dCAhPSBOVUxMKSB7CisJCWlmICghc3BhX2NvbmZp Z190cnllbnRlcihuZXh0LT5sMmFkX3NwYSwgU0NMX0wyQVJDLCBuZXh0LCBSV19SRUFERVIp KQorCQkJbmV4dCA9IE5VTEw7CisJfQogCW11dGV4X2V4aXQoJnNwYV9uYW1lc3BhY2VfbG9j ayk7CiAKIAlyZXR1cm4gKG5leHQpOwo= --------------080503040005000208080600-- From owner-zfs-devel@FreeBSD.ORG Mon Aug 8 13:28:48 2011 Return-Path: Delivered-To: zfs-devel@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 68559106564A; Mon, 8 Aug 2011 13:28:48 +0000 (UTC) (envelope-from pawel@dawidek.net) Received: from mail.dawidek.net (60.wheelsystems.com [83.12.187.60]) by mx1.freebsd.org (Postfix) with ESMTP id 12CE78FC16; Mon, 8 Aug 2011 13:28:47 +0000 (UTC) Received: from localhost (58.wheelsystems.com [83.12.187.58]) by mail.dawidek.net (Postfix) with ESMTPSA id 9F5E17D8; Mon, 8 Aug 2011 15:28:46 +0200 (CEST) Date: Mon, 8 Aug 2011 15:28:37 +0200 From: Pawel Jakub Dawidek To: d@delphij.net Message-ID: <20110808132837.GH1640@garage.freebsd.pl> References: <4E3E525D.1040906@delphij.net> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="fCcDWlUEdh43YKr8" Content-Disposition: inline In-Reply-To: <4E3E525D.1040906@delphij.net> X-OS: FreeBSD 9.0-CURRENT amd64 User-Agent: Mutt/1.5.21 (2010-09-15) Cc: Josh Paetzel , zfs-devel@FreeBSD.ORG, gibbs@FreeBSD.org, Xin LI , Doug White Subject: Re: [PATCH] L2ARC deadlock X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Aug 2011 13:28:48 -0000 --fCcDWlUEdh43YKr8 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sun, Aug 07, 2011 at 01:52:45AM -0700, Xin LI wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA256 >=20 > Hi, >=20 > I have observed this on a test system which have both L2ARC and ZIL > device installed, and several vdev groups. This can be triggered by > removing the l2arc and zil device (detaching l2arc would fail), blocking > at 'spa_namespace_lock' >=20 > After reboot, the pool can not be imported. 'zpool import' shows the > zpool command stuck with 'spa_namespace_lock'. >=20 > I have a theory about the issue but the patch was not tested, maybe > someone who is more familiar with the code can shed me some light? Yes, your theory is correct, but the problem is wider (there are other cases it might deadlock as we discussed with Justin (CCed)). The reason for it is that we have two GEOM classes in ZFS: ZVOL and VDEV_GEOM. ZVOL calls from GEOM into ZFS and VDEV_GEOM calls from ZFS into GEOM. This creates lock order reversals. While working on ZFSv28 I cleaned up locking, so that locks are always acquired in the following order: zfsdev_state_lock g_topology_lock spa_namespace_lock Unfortunately I missed spa_config lock, which is special and not handled by WITNESS and the problem you are seeing is indeed related to that lock (reversal between spa_namespace_lock and spa_config lock). I just tried to simplify the locking by eliminating the zfsdev_state_lock altogether and replacing it with the spa_namespace_lock. Xin, could you try the following patch: http://people.freebsd.org/~pjd/patches/zfsdev_state_lock.patch Justin, what do you think? --=20 Pawel Jakub Dawidek http://www.wheelsystems.com FreeBSD committer http://www.FreeBSD.org Am I Evil? Yes, I Am! http://yomoli.com --fCcDWlUEdh43YKr8 Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (FreeBSD) iEYEARECAAYFAk4/5IUACgkQForvXbEpPzQzkwCg7v4w+z88LdCjwVlYnPdpmsYZ vjoAoIfbGMnUkGY9wM9ipjJYYEGETnvN =eKTo -----END PGP SIGNATURE----- --fCcDWlUEdh43YKr8-- From owner-zfs-devel@FreeBSD.ORG Tue Aug 9 21:15:35 2011 Return-Path: Delivered-To: zfs-devel@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0BDA5106566B for ; Tue, 9 Aug 2011 21:15:35 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from mail.vx.sk (mail.vx.sk [IPv6:2a01:4f8:100:1043::3]) by mx1.freebsd.org (Postfix) with ESMTP id 9BFB28FC0A for ; Tue, 9 Aug 2011 21:15:34 +0000 (UTC) Received: from core.vx.sk (localhost [127.0.0.1]) by mail.vx.sk (Postfix) with ESMTP id C41A41914EA for ; Tue, 9 Aug 2011 23:15:33 +0200 (CEST) X-Virus-Scanned: amavisd-new at mail.vx.sk Received: from mail.vx.sk ([127.0.0.1]) by core.vx.sk (mail.vx.sk [127.0.0.1]) (amavisd-new, port 10024) with LMTP id Hs-5gOokOacU for ; Tue, 9 Aug 2011 23:15:31 +0200 (CEST) Received: from [10.9.8.1] (chello085216231078.chello.sk [85.216.231.78]) by mail.vx.sk (Postfix) with ESMTPSA id 4F9B31914E4 for ; Tue, 9 Aug 2011 23:15:31 +0200 (CEST) Message-ID: <4E41A377.8060103@FreeBSD.org> Date: Tue, 09 Aug 2011 23:15:35 +0200 From: Martin Matuska User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:5.0) Gecko/20110624 Thunderbird/5.0 MIME-Version: 1.0 To: zfs-devel@FreeBSD.ORG X-Enigmail-Version: 1.2 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: Subject: [BUG] cache corruption on incremental snapshot receive X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 09 Aug 2011 21:15:35 -0000 Hi, a user has wrongly described a bug report (kern/156933) but has contacted me recently. I wrote a script that reproduces the bug, it has nothing to do with readonly=on. The following script corrupts cache on snapshot receive (not reproducible on Solaris): #!/bin/sh zpool destroy tpool dd if=/dev/zero of=/tmp/poolfile bs=1m count=64 zpool create tpool /tmp/poolfile zfs create tpool/ds1 zfs create tpool/ds2 echo "Test line 1" > /tpool/ds1/test.txt zfs snapshot tpool/ds1@s1 zfs send tpool/ds1@s1 | zfs recv -F tpool/ds2 echo "Test line 2" >> /tpool/ds1/test.txt zfs snapshot tpool/ds1@s2 # This causes corrupted FS cache after 2nd recv tail /tpool/ds2/test.txt # zfs send -i @s1 tpool/ds1@s2 | zfs recv -F tpool/ds2 md5 /tpool/ds1/test.txt md5 /tpool/ds2/test.txt If you umount + mount tpool/ds2, the file is correct again. -- Martin Matuska FreeBSD committer http://blog.vx.sk From owner-zfs-devel@FreeBSD.ORG Wed Aug 10 18:05:13 2011 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0DD7A106564A for ; Wed, 10 Aug 2011 18:05:13 +0000 (UTC) (envelope-from martin@matuska.org) Received: from mail.vx.sk (mail.vx.sk [IPv6:2a01:4f8:100:1043::3]) by mx1.freebsd.org (Postfix) with ESMTP id 47D6C8FC14 for ; Wed, 10 Aug 2011 18:05:12 +0000 (UTC) Received: from core.vx.sk (localhost [127.0.0.1]) by mail.vx.sk (Postfix) with ESMTP id 5864B170322 for ; Wed, 10 Aug 2011 20:05:11 +0200 (CEST) X-Virus-Scanned: amavisd-new at mail.vx.sk Received: from mail.vx.sk ([127.0.0.1]) by core.vx.sk (mail.vx.sk [127.0.0.1]) (amavisd-new, port 10024) with LMTP id mRuzTQSH5lU5 for ; Wed, 10 Aug 2011 20:05:09 +0200 (CEST) Received: from [10.9.8.1] (chello085216231078.chello.sk [85.216.231.78]) by mail.vx.sk (Postfix) with ESMTPSA id F412017030D for ; Wed, 10 Aug 2011 20:05:08 +0200 (CEST) Message-ID: <4E42C859.5000909@matuska.org> Date: Wed, 10 Aug 2011 20:05:13 +0200 From: Martin Matuska User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:5.0) Gecko/20110624 Thunderbird/5.0 MIME-Version: 1.0 To: zfs-devel@FreeBSD.org Content-Type: multipart/mixed; boundary="------------020407000507000703080808" Cc: Subject: [PATCH] fix ZFS prefetch code X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 10 Aug 2011 18:05:13 -0000 This is a multi-part message in MIME format. --------------020407000507000703080808 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit The attached patch is a solution for kern/157728. The new zfs send/recv code doesn't play well with prefetching temporary clones. In Illumos, the prefetch code in zfs_ioc_dataset_list_next() is different from our code: /* * Pre-fetch the datasets. dmu_objset_prefetch() always returns 0 * but is not declared void because its called by dmu_objset_find(). */ if (zc->zc_cookie == 0) { uint64_t cookie = 0; int len = sizeof (zc->zc_name) - (p - zc->zc_name); while (dmu_dir_list_next(os, len, p, NULL, &cookie) == 0) (void) dmu_objset_prefetch(p NULL); } Compared to our code: /* * Pre-fetch the datasets. dmu_objset_prefetch() always returns 0 * but is not declared void because its called by dmu_objset_find(). */ if (zc->zc_cookie == 0) { uint64_t cookie = 0; int len = sizeof (zc->zc_name) - (p - zc->zc_name); while (dmu_dir_list_next(os, len, p, NULL, &cookie) == 0) (void) dmu_objset_prefetch(zc->zc_name, NULL); } I have done some debugging and Pawel's change is right, with "p" it doesn't work, dmu_objset_prefetch() rejects all entries on dmu_objset_hold() and everything is just skipped because names are incorrect) -> Solaris and Illumos do not prefetch datasets. How does this race happen? At the end of dmu_recv_existing_end() we destroy the temporary clone without checking any result code: (void) dsl_dataset_destroy(drc->drc_real_ds, dmu_recv_tag, B_FALSE); dsl_dataset_destroy() fails if there is an extra hold on the dataset - and dmu_objset_prefetch() is the function that places this hold. Because the prefetch code in Solaris doesn't work (is unfixed), the don't have this race. Now we have this code and comments in dmu_objset.c (dmu_objset_snapshot_one()): /* * If the objset starts with a '%', then ignore it unless it was * explicitly named (ie, not recursive). These hidden datasets * are always inconsistent, and by not opening them here, we can * avoid a race with dsl_dir_destroy_check(). */ cp = strrchr(name, '/'); if (cp && cp[1] == '%' && sn->recursive) return (0); (void) strcpy(sn->failed, name); /* * Check permissions if we are doing a recursive snapshot. The * permission checks for the starting dataset have already been * performed in zfs_secpolicy_snapshot() */ if (sn->recursive && (err = zfs_secpolicy_snapshot_perms(name, CRED()))) return (err); err = dmu_objset_hold(name, sn, &os); if (err != 0) return (err); /* * If the objset is in an inconsistent state (eg, in the process * of being destroyed), don't snapshot it. As with %hidden * datasets, we return EBUSY if this name was explicitly * requested (ie, not recursive), and otherwise ignore it. */ if (os->os_dsl_dataset->ds_phys->ds_flags & DS_FLAG_INCONSISTENT) { dmu_objset_rele(os, sn); return (sn->recursive ? 0 : EBUSY); } This is exactly the same race I am talking about. So when these datasets count as allways inconsistent, we have to do the same in dmu_objset_prefetch() and I suggest we don't prefetch inconsistent datasets. I am submitting this to Illumos as well. Please review attached patch. -- Martin Matuska FreeBSD committer http://blog.vx.sk --------------020407000507000703080808 Content-Type: text/plain; name="dmu_objset.c.patch" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="dmu_objset.c.patch" SW5kZXg6IHN5cy9jZGRsL2NvbnRyaWIvb3BlbnNvbGFyaXMvdXRzL2NvbW1vbi9mcy96ZnMv ZG11X29ianNldC5jCj09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT0KLS0tIHN5cy9jZGRsL2NvbnRyaWIvb3BlbnNv bGFyaXMvdXRzL2NvbW1vbi9mcy96ZnMvZG11X29ianNldC5jCShyZXZpc2lvbiAyMjQ3NjAp CisrKyBzeXMvY2RkbC9jb250cmliL29wZW5zb2xhcmlzL3V0cy9jb21tb24vZnMvemZzL2Rt dV9vYmpzZXQuYwkod29ya2luZyBjb3B5KQpAQCAtMTc2MCwxMCArMTc2MCwyOSBAQAogZG11 X29ianNldF9wcmVmZXRjaChjb25zdCBjaGFyICpuYW1lLCB2b2lkICphcmcpCiB7CiAJZHNs X2RhdGFzZXRfdCAqZHM7CisJY2hhciAqY3A7CiAKKwkvKgorCSAqIElmIHRoZSBvYmpzZXQg c3RhcnRzIHdpdGggYSAnJScsIHRoZW4gaWdub3JlIGl0LgorCSAqIFRoZXNlIGhpZGRlbiBk YXRhc2V0cyBhcmUgYWx3YXlzIGluY29uc2lzdGVudCBhbmQgYnkgbm90IG9wZW5pbmcKKwkg KiB0aGVtIGhlcmUsIHdlIGNhbiBhdm9pZCBhIHJhY2Ugd2l0aCBkc2xfZGlyX2Rlc3Ryb3lf Y2hlY2soKS4KKwkgKi8KKwljcCA9IHN0cnJjaHIobmFtZSwgJy8nKTsKKwlpZiAoY3AgJiYg Y3BbMV0gPT0gJyUnKQorCQlyZXR1cm4gKDApOworCiAJaWYgKGRzbF9kYXRhc2V0X2hvbGQo bmFtZSwgRlRBRywgJmRzKSkKIAkJcmV0dXJuICgwKTsKIAorCS8qCisJICogSWYgdGhlIG9i anNldCBpcyBpbiBhbiBpbmNvbnNpc3RlbnQgc3RhdGUgKGVnLCBpbiB0aGUgcHJvY2Vzcwor CSAqIG9mIGJlaW5nIGRlc3Ryb3llZCksIGRvbid0IHByZWZldGNoIGl0LgorCSAqLworCWlm IChkcy0+ZHNfcGh5cy0+ZHNfZmxhZ3MgJiBEU19GTEFHX0lOQ09OU0lTVEVOVCkgeworCQlk c2xfZGF0YXNldF9yZWxlKGRzLCBGVEFHKTsKKwkJcmV0dXJuICgwKTsKKwl9CisKIAlpZiAo IUJQX0lTX0hPTEUoJmRzLT5kc19waHlzLT5kc19icCkpIHsKIAkJbXV0ZXhfZW50ZXIoJmRz LT5kc19vcGVuaW5nX2xvY2spOwogCQlpZiAoZHMtPmRzX29ianNldCA9PSBOVUxMKSB7Cg== --------------020407000507000703080808-- From owner-zfs-devel@FreeBSD.ORG Wed Aug 10 19:49:36 2011 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 88C15106564A; Wed, 10 Aug 2011 19:49:36 +0000 (UTC) (envelope-from delphij@gmail.com) Received: from mail-vx0-f182.google.com (mail-vx0-f182.google.com [209.85.220.182]) by mx1.freebsd.org (Postfix) with ESMTP id 12F948FC0A; Wed, 10 Aug 2011 19:49:35 +0000 (UTC) Received: by vxh11 with SMTP id 11so1532550vxh.13 for ; Wed, 10 Aug 2011 12:49:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=JYAfSnHXd0+P82z/vPT9miQVbJ3grgbjP1tYh3+zEgQ=; b=N4zcv9OPcY58PJFHd4dzBDAlwXlFVXlXSzrRq0mKWUjy9/2KSPUcexUxiB3uih4LE+ OE3n1eeJ09k3aVaax8BU7Swpr0ijINNKY6/zX/S0mKtZwKO6poKpVXZvLT7/eskHWsju Y2psupvatvo7KMRbmI9RdDRc8e5rDvLpvGCjc= MIME-Version: 1.0 Received: by 10.52.90.135 with SMTP id bw7mr9052681vdb.279.1313004405764; Wed, 10 Aug 2011 12:26:45 -0700 (PDT) Received: by 10.52.110.166 with HTTP; Wed, 10 Aug 2011 12:26:45 -0700 (PDT) In-Reply-To: <20110808132837.GH1640@garage.freebsd.pl> References: <4E3E525D.1040906@delphij.net> <20110808132837.GH1640@garage.freebsd.pl> Date: Wed, 10 Aug 2011 12:26:45 -0700 Message-ID: From: Xin LI To: Pawel Jakub Dawidek Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: Josh Paetzel , zfs-devel@freebsd.org, Xin LI , Doug White Subject: Re: [PATCH] L2ARC deadlock X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 10 Aug 2011 19:49:36 -0000 Hi, Pawel, > Xin, could you try the following patch: > > =C2=A0 =C2=A0 =C2=A0 =C2=A0http://people.freebsd.org/~pjd/patches/zfsdev_= state_lock.patch For the record, our tests can no longer reproduce the issue I've mentioned before, with the patch applied. Could you please commit this patch against -HEAD? Cheers, --=20 Xin LI https://www.delphij.net/ FreeBSD - The Power to Serve! Live free or die From owner-zfs-devel@FreeBSD.ORG Fri Aug 12 14:29:04 2011 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 480F8106566B; Fri, 12 Aug 2011 14:29:04 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from mail.vx.sk (mail.vx.sk [IPv6:2a01:4f8:100:1043::3]) by mx1.freebsd.org (Postfix) with ESMTP id C16E38FC08; Fri, 12 Aug 2011 14:29:03 +0000 (UTC) Received: from core.vx.sk (localhost [127.0.0.1]) by mail.vx.sk (Postfix) with ESMTP id 21D4F190E1E; Fri, 12 Aug 2011 16:29:03 +0200 (CEST) X-Virus-Scanned: amavisd-new at mail.vx.sk Received: from mail.vx.sk ([127.0.0.1]) by core.vx.sk (mail.vx.sk [127.0.0.1]) (amavisd-new, port 10024) with LMTP id ZJq4MevRWFXd; Fri, 12 Aug 2011 16:29:00 +0200 (CEST) Received: from [10.9.8.1] (chello085216231078.chello.sk [85.216.231.78]) by mail.vx.sk (Postfix) with ESMTPSA id 48289190E04; Fri, 12 Aug 2011 16:28:59 +0200 (CEST) Message-ID: <4E4538B0.8080508@FreeBSD.org> Date: Fri, 12 Aug 2011 16:29:04 +0200 From: Martin Matuska User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:5.0) Gecko/20110624 Thunderbird/5.0 MIME-Version: 1.0 To: Pawel Jakub Dawidek , Xin Li , Andriy Gapon X-Enigmail-Version: 1.2 Content-Type: multipart/mixed; boundary="------------040508030707020806090701" Cc: zfs-devel@freebsd.org Subject: [PATCH] zfs prefetch patch X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 12 Aug 2011 14:29:04 -0000 This is a multi-part message in MIME format. --------------040508030707020806090701 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Please review my attached patch, I have decided to go for the minimalistic version for 9.0-RELEASE. Since upgrade to zfs v28 I have been experiencing temporary clones left over (not deleted) by incremental zfs receive. This made the parent snapshots undeletable. This has been also reported by several users in mailing lists. I have investigated this issue, and the cause is a race between dmu_objset_prefetch() invoked from zfs_ioc_dataset_list_next() and dsl_dir_destroy_check() indirectly invoked from dmu_recv_existing_end() via dsl_dataset_destroy(). In addition, the calling of dmu_objset_prefetch() from with internal datasets, temporary clones and invisible datasets is useless as it either fails (internal datsets) or we are not going to process these datasets later (they are skipped in the following code). Last, fix calling of dmu_objset_prefetch() from zvol_create_minors() by using absolute instead of relative dataset name. PR: kern/157728 (reported on Jun 9, 2011) Filed as Illumos bug #1346 https://www.illumos.org/issues/1346 Thank you very much. -- Martin Matuska FreeBSD committer http://blog.vx.sk --------------040508030707020806090701 Content-Type: text/plain; name="zfs_prefetch.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="zfs_prefetch.patch" Index: zfs_ioctl.c =================================================================== --- zfs_ioctl.c (revision 224794) +++ zfs_ioctl.c (working copy) @@ -1964,7 +1964,8 @@ zfs_ioc_dataset_list_next(zfs_cmd_t *zc) int len = sizeof (zc->zc_name) - (p - zc->zc_name); while (dmu_dir_list_next(os, len, p, NULL, &cookie) == 0) - (void) dmu_objset_prefetch(zc->zc_name, NULL); + if (dataset_name_hidden(zc->zc_name) == B_FALSE) + (void) dmu_objset_prefetch(zc->zc_name, NULL); } do { Index: zvol.c =================================================================== --- zvol.c (revision 224794) +++ zvol.c (working copy) @@ -2197,12 +2197,9 @@ zvol_create_minors(const char *name) p = osname + strlen(osname); len = MAXPATHLEN - (p - osname); - if (strchr(name, '/') == NULL) { - /* Prefetch only for pool name. */ - cookie = 0; - while (dmu_dir_list_next(os, len, p, NULL, &cookie) == 0) - (void) dmu_objset_prefetch(p, NULL); - } + /* Prefetch only for pool name. */ + if (strchr(name, '/') == NULL) + (void) dmu_objset_prefetch(name, NULL); cookie = 0; while (dmu_dir_list_next(os, MAXPATHLEN - (p - osname), p, NULL, --------------040508030707020806090701 Content-Type: text/plain; name="zfs_prefetch.txt" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="zfs_prefetch.txt" Rml4IHJhY2UgYmV0d2VlbiBkbXVfb2Jqc2V0X3ByZWZldGNoKCkgaW52b2tlZCBmcm9tCnpm c19pb2NfZGF0YXNldF9saXN0X25leHQoKSBhbmQgZHNsX2Rpcl9kZXN0cm95X2NoZWNrKCkg aW5kaXJlY3RseQppbnZva2VkIGZyb20gZG11X3JlY3ZfZXhpc3RpbmdfZW5kKCkgdmlhIGRz bF9kYXRhc2V0X2Rlc3Ryb3koKSBieSBub3QKcHJlZmV0Y2hpbmcgdGVtcG9yYXJ5IGNsb25l cywgYXMgdGhlc2UgY291bnQgYXMgYWx3YXlzIGluY29uc2lzdGVudC4KSW4gYWRkaXRpb24s IGRvIG5vdCBwcmVmZXRjaCBoaWRkZW4gZGF0YXNldHMgYXQgYWxsIGFzIHdlIGFyZSBub3QK Z29pbmcgdG8gcHJvY2VzcyB0aGVzZSBsYXRlci4gWzFdCgpGaXggY2FsbGluZyBvZiBkbXVf b2Jqc2V0X3ByZWZldGNoKCkgZnJvbSB6dm9sX2NyZWF0ZV9taW5vcnMoKQpieSB1c2luZyBh YnNvbHV0ZSBpbnN0ZWFkIG9mIHJlbGF0aXZlIGRhdGFzZXQgbmFtZS4KCkZpbGVkIGFzIEls bHVtb3MgQnVnIDEzNDYgWzFdCgpQUjoJCQkJa2Vybi8xNTc3MjggWzFdClRlc3RlZCBieToJ CUJvcmphIE1hcmNvcyA8Ym9yamFtQHNhcmVuZXQuZXM+IFsxXSwgbW0KQXBwcm92ZWQgYnk6 CXJlICgpCk1GQyBhZnRlcjoJCTEgd2VlayA= --------------040508030707020806090701-- From owner-zfs-devel@FreeBSD.ORG Wed Aug 24 13:03:15 2011 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8846E1065673; Wed, 24 Aug 2011 13:03:15 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from mail.vx.sk (mail.vx.sk [IPv6:2a01:4f8:100:1043::3]) by mx1.freebsd.org (Postfix) with ESMTP id 21A088FC14; Wed, 24 Aug 2011 13:03:15 +0000 (UTC) Received: from core.vx.sk (localhost [127.0.0.1]) by mail.vx.sk (Postfix) with ESMTP id 3EF6C197CA3; Wed, 24 Aug 2011 15:03:14 +0200 (CEST) X-Virus-Scanned: amavisd-new at mail.vx.sk Received: from mail.vx.sk ([127.0.0.1]) by core.vx.sk (mail.vx.sk [127.0.0.1]) (amavisd-new, port 10024) with LMTP id CHw1yIMx1S6U; Wed, 24 Aug 2011 15:03:11 +0200 (CEST) Received: from [10.0.3.3] (188-167-66-148.dynamic.chello.sk [188.167.66.148]) by mail.vx.sk (Postfix) with ESMTPSA id 74BBA197C97; Wed, 24 Aug 2011 15:03:09 +0200 (CEST) Message-ID: <4E54F689.4010600@FreeBSD.org> Date: Wed, 24 Aug 2011 15:03:05 +0200 From: Martin Matuska User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:6.0) Gecko/20110812 Thunderbird/6.0 MIME-Version: 1.0 To: Pawel Jakub Dawidek X-Enigmail-Version: 1.3.1 Content-Type: multipart/mixed; boundary="------------090805040003050802020905" Cc: zfs-devel@FreeBSD.org, Andriy Gapon Subject: [PATCH] fix for kern/160035 for review X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Aug 2011 13:03:15 -0000 This is a multi-part message in MIME format. --------------090805040003050802020905 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit The attached patch fixes kern/160035 and kern/156933. http://www.freebsd.org/cgi/query-pr.cgi?pr=160035 The function zfs_rezget() is called as part of zfs_resume_fs(). Suspending and resuming a ZFS filesystem is used in three cases: 1.) zfs rollback 2.) snapshot receive on a mounted fs (zfs recv -F) 3.) zfs upgrade To make sure we don't have mmaped wrong data we have to invalidate buffers for all vnodes. -- Martin Matuska FreeBSD committer http://blog.vx.sk --------------090805040003050802020905 Content-Type: text/plain; name="zfs_znode.c.patch" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="zfs_znode.c.patch" SW5kZXg6IHN5cy9jZGRsL2NvbnRyaWIvb3BlbnNvbGFyaXMvdXRzL2NvbW1vbi9mcy96ZnMv emZzX3pub2RlLmMKPT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PQotLS0gc3lzL2NkZGwvY29udHJpYi9vcGVuc29s YXJpcy91dHMvY29tbW9uL2ZzL3pmcy96ZnNfem5vZGUuYwkocmV2aXNpb24gMjI1MTQwKQor Kysgc3lzL2NkZGwvY29udHJpYi9vcGVuc29sYXJpcy91dHMvY29tbW9uL2ZzL3pmcy96ZnNf em5vZGUuYwkod29ya2luZyBjb3B5KQpAQCAtMTM0Myw2ICsxMzQzLDkgQEAgemZzX3Jlemdl dCh6bm9kZV90ICp6cCkKIAogCXpwLT56X3VubGlua2VkID0gKHpwLT56X2xpbmtzID09IDAp OwogCXpwLT56X2Jsa3N6ID0gZG9pLmRvaV9kYXRhX2Jsb2NrX3NpemU7CisKKwlpZiAodmlu dmFsYnVmKFpUT1YoenApLCBWX1NBVkUsIDAsIDApICE9IDApCisJCXZpbnZhbGJ1ZihaVE9W KHpwKSwgMCwgMCwgMCk7CiAJaWYgKHpwLT56X3NpemUgIT0gc2l6ZSAmJiBaVE9WKHpwKSAh PSBOVUxMKQogCQl2bm9kZV9wYWdlcl9zZXRzaXplKFpUT1YoenApLCB6cC0+el9zaXplKTsK IAo= --------------090805040003050802020905-- From owner-zfs-devel@FreeBSD.ORG Wed Aug 24 17:38:03 2011 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3B17B1065673; Wed, 24 Aug 2011 17:38:03 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from mail.vx.sk (mail.vx.sk [IPv6:2a01:4f8:100:1043::3]) by mx1.freebsd.org (Postfix) with ESMTP id C44E68FC0C; Wed, 24 Aug 2011 17:38:02 +0000 (UTC) Received: from core.vx.sk (localhost [127.0.0.1]) by mail.vx.sk (Postfix) with ESMTP id E5B53197341; Wed, 24 Aug 2011 19:38:01 +0200 (CEST) X-Virus-Scanned: amavisd-new at mail.vx.sk Received: from mail.vx.sk ([127.0.0.1]) by core.vx.sk (mail.vx.sk [127.0.0.1]) (amavisd-new, port 10024) with LMTP id DufxESP22zL3; Wed, 24 Aug 2011 19:38:00 +0200 (CEST) Received: from [10.9.8.1] (188-167-78-15.dynamic.chello.sk [188.167.78.15]) by mail.vx.sk (Postfix) with ESMTPSA id B1FE019732C; Wed, 24 Aug 2011 19:37:58 +0200 (CEST) Message-ID: <4E5536F6.9020200@FreeBSD.org> Date: Wed, 24 Aug 2011 19:37:58 +0200 From: Martin Matuska User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:6.0) Gecko/20110812 Thunderbird/6.0 MIME-Version: 1.0 To: Kostik Belousov References: <4E54F947.8060506@FreeBSD.org> <20110824134514.GR17489@deviant.kiev.zoral.com.ua> In-Reply-To: <20110824134514.GR17489@deviant.kiev.zoral.com.ua> X-Enigmail-Version: 1.3.1 Content-Type: multipart/mixed; boundary="------------000207000804020509030203" Cc: zfs-devel@freebsd.org, Pawel Jakub Dawidek Subject: Re: Patch review reguest: zfs_znode.c X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Aug 2011 17:38:03 -0000 This is a multi-part message in MIME format. --------------000207000804020509030203 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit On 24. 8. 2011 15:45, Kostik Belousov wrote: > On Wed, Aug 24, 2011 at 03:14:47PM +0200, Martin Matuska wrote: >> Hi Kostik, >> >> could you please review the attached patch? >> >> It fixes kern/160035 and kern/156933. >> http://www.freebsd.org/cgi/query-pr.cgi?pr=160035 >> >> The function zfs_rezget() is called as part of zfs_resume_fs(). >> Suspending and resuming a ZFS filesystem is used in three cases: >> 1.) zfs rollback >> 2.) snapshot receive on a mounted fs (zfs recv -F) >> 3.) zfs upgrade >> >> In case 1.) and 2.) the filesystem usually changes (changed/deleted/new files/directories). >> >> To make sure we don't have mmaped wrong data we have to invalidate >> buffers for all vnodes of the resuming fs. >> >> Thank you very much, >> Martin >> > Uh. Does ZFS use the buffer cache at all ? > > Could it be that all you need is a call to vm_object_page_remove(vp->v_object, > 0, 0) ? Yes, that seems to be enough. Updated patch attached. -- Martin Matuska FreeBSD committer http://blog.vx.sk --------------000207000804020509030203 Content-Type: text/plain; name="zfs_znode.c.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="zfs_znode.c.patch" Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_znode.c =================================================================== --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_znode.c (revision 225140) +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_znode.c (working copy) @@ -1343,6 +1343,12 @@ zp->z_unlinked = (zp->z_links == 0); zp->z_blksz = doi.doi_data_block_size; + + if (ZTOV(zp)->v_object != NULL) { + VM_OBJECT_LOCK(ZTOV(zp)->v_object); + vm_object_page_remove(ZTOV(zp)->v_object, 0, 0, 0); + VM_OBJECT_UNLOCK(ZTOV(zp)->v_object); + } if (zp->z_size != size && ZTOV(zp) != NULL) vnode_pager_setsize(ZTOV(zp), zp->z_size); --------------000207000804020509030203-- From owner-zfs-devel@FreeBSD.ORG Wed Aug 24 17:49:18 2011 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 893981065673; Wed, 24 Aug 2011 17:49:18 +0000 (UTC) (envelope-from pawel@dawidek.net) Received: from mail.dawidek.net (60.wheelsystems.com [83.12.187.60]) by mx1.freebsd.org (Postfix) with ESMTP id 3D58E8FC08; Wed, 24 Aug 2011 17:49:18 +0000 (UTC) Received: from localhost (89-73-195-149.dynamic.chello.pl [89.73.195.149]) by mail.dawidek.net (Postfix) with ESMTPSA id B85FDF7C; Wed, 24 Aug 2011 19:49:15 +0200 (CEST) Date: Wed, 24 Aug 2011 19:48:58 +0200 From: Pawel Jakub Dawidek To: Martin Matuska Message-ID: <20110824174858.GB1688@garage.freebsd.pl> References: <4E54F947.8060506@FreeBSD.org> <20110824134514.GR17489@deviant.kiev.zoral.com.ua> <4E5536F6.9020200@FreeBSD.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="wq9mPyueHGvFACwf" Content-Disposition: inline In-Reply-To: <4E5536F6.9020200@FreeBSD.org> X-OS: FreeBSD 9.0-CURRENT amd64 User-Agent: Mutt/1.5.21 (2010-09-15) Cc: Kostik Belousov , zfs-devel@freebsd.org Subject: Re: Patch review reguest: zfs_znode.c X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Aug 2011 17:49:18 -0000 --wq9mPyueHGvFACwf Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, Aug 24, 2011 at 07:37:58PM +0200, Martin Matuska wrote: > On 24. 8. 2011 15:45, Kostik Belousov wrote: > > Uh. Does ZFS use the buffer cache at all ? > > > > Could it be that all you need is a call to vm_object_page_remove(vp->v_= object, > > 0, 0) ? > Yes, that seems to be enough. > Updated patch attached. > Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_znode.c > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_znode.c (revision = 225140) > +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_znode.c (working c= opy) > @@ -1343,6 +1343,12 @@ > =20 > zp->z_unlinked =3D (zp->z_links =3D=3D 0); > zp->z_blksz =3D doi.doi_data_block_size; > + > + if (ZTOV(zp)->v_object !=3D NULL) { > + VM_OBJECT_LOCK(ZTOV(zp)->v_object); > + vm_object_page_remove(ZTOV(zp)->v_object, 0, 0, 0); > + VM_OBJECT_UNLOCK(ZTOV(zp)->v_object); > + } > if (zp->z_size !=3D size && ZTOV(zp) !=3D NULL) > vnode_pager_setsize(ZTOV(zp), zp->z_size); As you can see after the code you added we check for ZTOV(zp) not being NULL. If it is possible it can be NULL, you need to check it also before referencing it. --=20 Pawel Jakub Dawidek http://www.wheelsystems.com FreeBSD committer http://www.FreeBSD.org Am I Evil? Yes, I Am! http://yomoli.com --wq9mPyueHGvFACwf Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (FreeBSD) iEYEARECAAYFAk5VOYkACgkQForvXbEpPzTiSACghZCeeHEi/kXR3t/W7Ukl++4J 0rQAoJsJqga+w6Qi9BZrb/R1JxlNzOnQ =316C -----END PGP SIGNATURE----- --wq9mPyueHGvFACwf-- From owner-zfs-devel@FreeBSD.ORG Wed Aug 24 17:51:31 2011 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 15A591065673; Wed, 24 Aug 2011 17:51:31 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.zoral.com.ua (mx0.zoral.com.ua [91.193.166.200]) by mx1.freebsd.org (Postfix) with ESMTP id A26D68FC18; Wed, 24 Aug 2011 17:51:30 +0000 (UTC) Received: from deviant.kiev.zoral.com.ua (root@deviant.kiev.zoral.com.ua [10.1.1.148]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id p7OHpPn6067339 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 24 Aug 2011 20:51:25 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: from deviant.kiev.zoral.com.ua (kostik@localhost [127.0.0.1]) by deviant.kiev.zoral.com.ua (8.14.4/8.14.4) with ESMTP id p7OHpPDp075876; Wed, 24 Aug 2011 20:51:25 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by deviant.kiev.zoral.com.ua (8.14.4/8.14.4/Submit) id p7OHpPYX075875; Wed, 24 Aug 2011 20:51:25 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: deviant.kiev.zoral.com.ua: kostik set sender to kostikbel@gmail.com using -f Date: Wed, 24 Aug 2011 20:51:25 +0300 From: Kostik Belousov To: Martin Matuska Message-ID: <20110824175125.GV17489@deviant.kiev.zoral.com.ua> References: <4E54F947.8060506@FreeBSD.org> <20110824134514.GR17489@deviant.kiev.zoral.com.ua> <4E5536F6.9020200@FreeBSD.org> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="XzfsYBK/mU6LvfOy" Content-Disposition: inline In-Reply-To: <4E5536F6.9020200@FreeBSD.org> User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: clamav-milter 0.95.2 at skuns.kiev.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-3.3 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00, DNS_FROM_OPENWHOIS autolearn=no version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on skuns.kiev.zoral.com.ua Cc: zfs-devel@freebsd.org Subject: Re: Patch review reguest: zfs_znode.c X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Aug 2011 17:51:32 -0000 --XzfsYBK/mU6LvfOy Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, Aug 24, 2011 at 07:37:58PM +0200, Martin Matuska wrote: > On 24. 8. 2011 15:45, Kostik Belousov wrote: > > On Wed, Aug 24, 2011 at 03:14:47PM +0200, Martin Matuska wrote: > >> Hi Kostik, > >> > >> could you please review the attached patch? > >> > >> It fixes kern/160035 and kern/156933. > >> http://www.freebsd.org/cgi/query-pr.cgi?pr=3D160035 > >> > >> The function zfs_rezget() is called as part of zfs_resume_fs(). > >> Suspending and resuming a ZFS filesystem is used in three cases: > >> 1.) zfs rollback > >> 2.) snapshot receive on a mounted fs (zfs recv -F) > >> 3.) zfs upgrade > >> > >> In case 1.) and 2.) the filesystem usually changes (changed/deleted/ne= w files/directories). > >> > >> To make sure we don't have mmaped wrong data we have to invalidate > >> buffers for all vnodes of the resuming fs. > >> > >> Thank you very much, > >> Martin=20 > >> > > Uh. Does ZFS use the buffer cache at all ? > > > > Could it be that all you need is a call to vm_object_page_remove(vp->v_= object, > > 0, 0) ? > Yes, that seems to be enough. > Updated patch attached. >=20 > --=20 > Martin Matuska > FreeBSD committer > http://blog.vx.sk >=20 > Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_znode.c > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_znode.c (revision = 225140) > +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_znode.c (working c= opy) > @@ -1343,6 +1343,12 @@ > =20 > zp->z_unlinked =3D (zp->z_links =3D=3D 0); > zp->z_blksz =3D doi.doi_data_block_size; > + > + if (ZTOV(zp)->v_object !=3D NULL) { > + VM_OBJECT_LOCK(ZTOV(zp)->v_object); > + vm_object_page_remove(ZTOV(zp)->v_object, 0, 0, 0); > + VM_OBJECT_UNLOCK(ZTOV(zp)->v_object); > + } > if (zp->z_size !=3D size && ZTOV(zp) !=3D NULL) > vnode_pager_setsize(ZTOV(zp), zp->z_size); > =20 I recommend you to create a local variable of the type vm_object_t and use it, instead of dereferencing the long chain of pointers. Also, consider moving ffs_pages_remove out from ffs_inode() into vfs_vnops.= c, calling it e.g. vn_pages_remove(), and use it instead. --XzfsYBK/mU6LvfOy Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (FreeBSD) iEYEARECAAYFAk5VOh0ACgkQC3+MBN1Mb4jz4gCfdJNbpYqOeMk6Fdj6OlvnD0bK hocAoJUD2+1D9YXLv/kkiMClzZht1CJs =kF+I -----END PGP SIGNATURE----- --XzfsYBK/mU6LvfOy-- From owner-zfs-devel@FreeBSD.ORG Wed Aug 24 19:14:15 2011 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8E5B1106567F; Wed, 24 Aug 2011 19:14:15 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from mail.vx.sk (mail.vx.sk [IPv6:2a01:4f8:100:1043::3]) by mx1.freebsd.org (Postfix) with ESMTP id 4980B8FC20; Wed, 24 Aug 2011 19:14:15 +0000 (UTC) Received: from core.vx.sk (localhost [127.0.0.1]) by mail.vx.sk (Postfix) with ESMTP id 490B019625A; Wed, 24 Aug 2011 21:14:13 +0200 (CEST) X-Virus-Scanned: amavisd-new at mail.vx.sk Received: from mail.vx.sk ([127.0.0.1]) by core.vx.sk (mail.vx.sk [127.0.0.1]) (amavisd-new, port 10024) with LMTP id 2X3CHf7wgtIP; Wed, 24 Aug 2011 21:14:11 +0200 (CEST) Received: from [10.9.8.1] (188-167-78-15.dynamic.chello.sk [188.167.78.15]) by mail.vx.sk (Postfix) with ESMTPSA id D1ACD19624A; Wed, 24 Aug 2011 21:14:10 +0200 (CEST) Message-ID: <4E554D82.805@FreeBSD.org> Date: Wed, 24 Aug 2011 21:14:10 +0200 From: Martin Matuska User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:6.0) Gecko/20110812 Thunderbird/6.0 MIME-Version: 1.0 To: Kostik Belousov References: <4E54F947.8060506@FreeBSD.org> <20110824134514.GR17489@deviant.kiev.zoral.com.ua> <4E5536F6.9020200@FreeBSD.org> <20110824175125.GV17489@deviant.kiev.zoral.com.ua> In-Reply-To: <20110824175125.GV17489@deviant.kiev.zoral.com.ua> X-Enigmail-Version: 1.3.1 Content-Type: multipart/mixed; boundary="------------010201060300020505080503" Cc: zfs-devel@freebsd.org Subject: Re: Patch review reguest: zfs_znode.c X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Aug 2011 19:14:15 -0000 This is a multi-part message in MIME format. --------------010201060300020505080503 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit On 24. 8. 2011 19:51, Kostik Belousov wrote: > I recommend you to create a local variable of the type vm_object_t > and use it, instead of dereferencing the long chain of pointers. > > Also, consider moving ffs_pages_remove out from ffs_inode() into vfs_vnops.c, > calling it e.g. vn_pages_remove(), and use it instead. Thank you very much for your comments. A new version with function moved to zfs_vnops.c and considering pjd's remark is attached. The function is thematically placed after static update_pages() and before static mappedread_sf(). -- Martin Matuska FreeBSD committer http://blog.vx.sk --------------010201060300020505080503 Content-Type: text/plain; name="zfs_remove_pages.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="zfs_remove_pages.patch" Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_znode.c =================================================================== --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_znode.c (revision 225140) +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_znode.c (working copy) @@ -1253,6 +1253,8 @@ again: return (err); } +extern void zfs_remove_pages(vnode_t *vp); + int zfs_rezget(znode_t *zp) { @@ -1343,8 +1345,11 @@ zfs_rezget(znode_t *zp) zp->z_unlinked = (zp->z_links == 0); zp->z_blksz = doi.doi_data_block_size; - if (zp->z_size != size && ZTOV(zp) != NULL) - vnode_pager_setsize(ZTOV(zp), zp->z_size); + if (ZTOV(zp) != NULL) { + zfs_remove_pages(ZTOV(zp)); + if (zp->z_size != size) + vnode_pager_setsize(ZTOV(zp), zp->z_size); + } ZFS_OBJ_HOLD_EXIT(zfsvfs, obj_num); Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c =================================================================== --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c (revision 225140) +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c (working copy) @@ -420,6 +420,21 @@ update_pages(vnode_t *vp, int64_t start, int len, } /* + * Remove all mapped pages from a vnode + */ +void +zfs_remove_pages(vnode_t *vp) +{ + vm_object_t obj = vp->v_object; + + if (obj != NULL) { + VM_OBJECT_LOCK(obj); + vm_object_page_remove(obj, 0, 0, 0); + VM_OBJECT_UNLOCK(obj); + } +} + +/* * Read with UIO_NOCOPY flag means that sendfile(2) requests * ZFS to populate a range of page cache pages with data. * --------------010201060300020505080503-- From owner-zfs-devel@FreeBSD.ORG Wed Aug 24 21:07:50 2011 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CF9681065672; Wed, 24 Aug 2011 21:07:50 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from mail.vx.sk (mail.vx.sk [IPv6:2a01:4f8:100:1043::3]) by mx1.freebsd.org (Postfix) with ESMTP id 3AA1B8FC18; Wed, 24 Aug 2011 21:07:50 +0000 (UTC) Received: from core.vx.sk (localhost [127.0.0.1]) by mail.vx.sk (Postfix) with ESMTP id EC2A6197625; Wed, 24 Aug 2011 23:07:48 +0200 (CEST) X-Virus-Scanned: amavisd-new at mail.vx.sk Received: from mail.vx.sk ([127.0.0.1]) by core.vx.sk (mail.vx.sk [127.0.0.1]) (amavisd-new, port 10024) with LMTP id W2GpVcgUpJ8U; Wed, 24 Aug 2011 23:07:46 +0200 (CEST) Received: from [10.9.8.1] (188-167-78-15.dynamic.chello.sk [188.167.78.15]) by mail.vx.sk (Postfix) with ESMTPSA id 5313819761A; Wed, 24 Aug 2011 23:07:46 +0200 (CEST) Message-ID: <4E556822.1030004@FreeBSD.org> Date: Wed, 24 Aug 2011 23:07:46 +0200 From: Martin Matuska User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:6.0) Gecko/20110812 Thunderbird/6.0 MIME-Version: 1.0 To: Kostik Belousov References: <4E54F947.8060506@FreeBSD.org> <20110824134514.GR17489@deviant.kiev.zoral.com.ua> <4E5536F6.9020200@FreeBSD.org> <20110824175125.GV17489@deviant.kiev.zoral.com.ua> In-Reply-To: <20110824175125.GV17489@deviant.kiev.zoral.com.ua> X-Enigmail-Version: 1.3.1 Content-Type: multipart/mixed; boundary="------------070308010107030404050803" Cc: zfs-devel@freebsd.org Subject: Patch review reguest: vn_pages_remove() X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Aug 2011 21:07:50 -0000 This is a multi-part message in MIME format. --------------070308010107030404050803 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit A generalization of vn_pages_remove() is attached + ZTOV(vp) suggested change by pjd. Is this we have been talking about? Thanks, mm -- Martin Matuska FreeBSD committer http://blog.vx.sk --------------070308010107030404050803 Content-Type: text/plain; name="vn_pages_remove.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="vn_pages_remove.patch" Index: sys/ufs/ffs/ffs_softdep.c =================================================================== --- sys/ufs/ffs/ffs_softdep.c (revision 225140) +++ sys/ufs/ffs/ffs_softdep.c (working copy) @@ -6541,7 +6541,7 @@ trunc_pages(ip, length, extblocks, flags) fs = ip->i_fs; extend = OFF_TO_IDX(lblktosize(fs, -extblocks)); if ((flags & IO_EXT) != 0) - ffs_pages_remove(vp, extend, 0); + vn_pages_remove(vp, extend, 0); if ((flags & IO_NORMAL) == 0) return; BO_LOCK(&vp->v_bufobj); @@ -6567,7 +6567,7 @@ trunc_pages(ip, length, extblocks, flags) end = OFF_TO_IDX(lblktosize(fs, lbn)); } else end = extend; - ffs_pages_remove(vp, OFF_TO_IDX(OFF_MAX), end); + vn_pages_remove(vp, OFF_TO_IDX(OFF_MAX), end); } /* Index: sys/ufs/ffs/ffs_extern.h =================================================================== --- sys/ufs/ffs/ffs_extern.h (revision 225140) +++ sys/ufs/ffs/ffs_extern.h (working copy) @@ -79,7 +79,6 @@ int ffs_isfreeblock(struct fs *, u_char *, ufs1_da void ffs_load_inode(struct buf *, struct inode *, struct fs *, ino_t); int ffs_mountroot(void); void ffs_oldfscompat_write(struct fs *, struct ufsmount *); -void ffs_pages_remove(struct vnode *vp, vm_pindex_t start, vm_pindex_t end); int ffs_reallocblks(struct vop_reallocblks_args *); int ffs_realloccg(struct inode *, ufs2_daddr_t, ufs2_daddr_t, ufs2_daddr_t, int, int, int, struct ucred *, struct buf **); Index: sys/ufs/ffs/ffs_inode.c =================================================================== --- sys/ufs/ffs/ffs_inode.c (revision 225140) +++ sys/ufs/ffs/ffs_inode.c (working copy) @@ -120,18 +120,6 @@ ffs_update(vp, waitfor) } } -void -ffs_pages_remove(struct vnode *vp, vm_pindex_t start, vm_pindex_t end) -{ - vm_object_t object; - - if ((object = vp->v_object) == NULL) - return; - VM_OBJECT_LOCK(object); - vm_object_page_remove(object, start, end, 0); - VM_OBJECT_UNLOCK(object); -} - #define SINGLE 0 /* index of single indirect block */ #define DOUBLE 1 /* index of double indirect block */ #define TRIPLE 2 /* index of triple indirect block */ @@ -219,7 +207,7 @@ ffs_truncate(vp, length, flags, cred, td) (void) chkdq(ip, -extblocks, NOCRED, 0); #endif vinvalbuf(vp, V_ALT, 0, 0); - ffs_pages_remove(vp, + vn_pages_remove(vp, OFF_TO_IDX(lblktosize(fs, -extblocks)), 0); osize = ip->i_din2->di_extsize; ip->i_din2->di_blocks -= extblocks; Index: sys/kern/vfs_vnops.c =================================================================== --- sys/kern/vfs_vnops.c (revision 225152) +++ sys/kern/vfs_vnops.c (working copy) @@ -64,6 +64,9 @@ __FBSDID("$FreeBSD$"); #include #include +#include +#include + static fo_rdwr_t vn_read; static fo_rdwr_t vn_write; static fo_truncate_t vn_truncate; @@ -1398,3 +1401,15 @@ vn_chown(struct file *fp, uid_t uid, gid_t gid, st VFS_UNLOCK_GIANT(vfslocked); return (error); } + +void +vn_pages_remove(struct vnode *vp, vm_pindex_t start, vm_pindex_t end) +{ + vm_object_t object; + + if ((object = vp->v_object) == NULL) + return; + VM_OBJECT_LOCK(object); + vm_object_page_remove(object, start, end, 0); + VM_OBJECT_UNLOCK(object); +} Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_znode.c =================================================================== --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_znode.c (revision 225140) +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_znode.c (working copy) @@ -1259,6 +1259,7 @@ zfs_rezget(znode_t *zp) zfsvfs_t *zfsvfs = zp->z_zfsvfs; dmu_object_info_t doi; dmu_buf_t *db; + vnode_t *vp = ZTOV(zp); uint64_t obj_num = zp->z_id; uint64_t mode, size; sa_bulk_attr_t bulk[8]; @@ -1334,8 +1335,8 @@ zfs_rezget(znode_t *zp) * that for example regular file was replaced with directory * which has the same object number. */ - if (ZTOV(zp) != NULL && - ZTOV(zp)->v_type != IFTOVT((mode_t)zp->z_mode)) { + if (vp != NULL && + vp->v_type != IFTOVT((mode_t)zp->z_mode)) { zfs_znode_dmu_fini(zp); ZFS_OBJ_HOLD_EXIT(zfsvfs, obj_num); return (EIO); @@ -1343,8 +1344,11 @@ zfs_rezget(znode_t *zp) zp->z_unlinked = (zp->z_links == 0); zp->z_blksz = doi.doi_data_block_size; - if (zp->z_size != size && ZTOV(zp) != NULL) - vnode_pager_setsize(ZTOV(zp), zp->z_size); + if (vp != NULL) { + vn_pages_remove(vp, 0, 0); + if (zp->z_size != size) + vnode_pager_setsize(vp, zp->z_size); + } ZFS_OBJ_HOLD_EXIT(zfsvfs, obj_num); Index: sys/sys/param.h =================================================================== --- sys/sys/param.h (revision 225140) +++ sys/sys/param.h (working copy) @@ -58,7 +58,7 @@ * in the range 5 to 9. */ #undef __FreeBSD_version -#define __FreeBSD_version 900041 /* Master, propagated to newvers */ +#define __FreeBSD_version 900042 /* Master, propagated to newvers */ #ifdef _KERNEL #define P_OSREL_SIGSEGV 700004 Index: sys/sys/vnode.h =================================================================== --- sys/sys/vnode.h (revision 225140) +++ sys/sys/vnode.h (working copy) @@ -640,6 +640,7 @@ int _vn_lock(struct vnode *vp, int flags, char *fi int vn_open(struct nameidata *ndp, int *flagp, int cmode, struct file *fp); int vn_open_cred(struct nameidata *ndp, int *flagp, int cmode, u_int vn_open_flags, struct ucred *cred, struct file *fp); +void vn_pages_remove(struct vnode *vp, vm_pindex_t start, vm_pindex_t end); int vn_pollrecord(struct vnode *vp, struct thread *p, int events); int vn_rdwr(enum uio_rw rw, struct vnode *vp, void *base, int len, off_t offset, enum uio_seg segflg, int ioflg, --------------070308010107030404050803-- From owner-zfs-devel@FreeBSD.ORG Wed Aug 24 21:26:47 2011 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 512801065673; Wed, 24 Aug 2011 21:26:47 +0000 (UTC) (envelope-from pawel@dawidek.net) Received: from mail.dawidek.net (60.wheelsystems.com [83.12.187.60]) by mx1.freebsd.org (Postfix) with ESMTP id 03F8A8FC0C; Wed, 24 Aug 2011 21:26:46 +0000 (UTC) Received: from localhost (89-73-195-149.dynamic.chello.pl [89.73.195.149]) by mail.dawidek.net (Postfix) with ESMTPSA id CBD87C3; Wed, 24 Aug 2011 23:26:44 +0200 (CEST) Date: Wed, 24 Aug 2011 23:26:27 +0200 From: Pawel Jakub Dawidek To: Martin Matuska Message-ID: <20110824212626.GJ1688@garage.freebsd.pl> References: <4E54F947.8060506@FreeBSD.org> <20110824134514.GR17489@deviant.kiev.zoral.com.ua> <4E5536F6.9020200@FreeBSD.org> <20110824175125.GV17489@deviant.kiev.zoral.com.ua> <4E556822.1030004@FreeBSD.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="BEa57a89OpeoUzGD" Content-Disposition: inline In-Reply-To: <4E556822.1030004@FreeBSD.org> X-OS: FreeBSD 9.0-CURRENT amd64 User-Agent: Mutt/1.5.21 (2010-09-15) Cc: Kostik Belousov , zfs-devel@freebsd.org Subject: Re: Patch review reguest: vn_pages_remove() X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Aug 2011 21:26:47 -0000 --BEa57a89OpeoUzGD Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, Aug 24, 2011 at 11:07:46PM +0200, Martin Matuska wrote: > A generalization of vn_pages_remove() is attached + ZTOV(vp) suggested > change by pjd. > Is this we have been talking about? [...] > --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_znode.c (revision = 225140) > +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_znode.c (working c= opy) > @@ -1259,6 +1259,7 @@ zfs_rezget(znode_t *zp) > zfsvfs_t *zfsvfs =3D zp->z_zfsvfs; > dmu_object_info_t doi; > dmu_buf_t *db; > + vnode_t *vp =3D ZTOV(zp); Could you move the assignment just before it is used? We put a hold on znode and I'd prefer not to obtain its vnode pointer before we do that. Just in case. > uint64_t obj_num =3D zp->z_id; > uint64_t mode, size; > sa_bulk_attr_t bulk[8]; > @@ -1334,8 +1335,8 @@ zfs_rezget(znode_t *zp) > * that for example regular file was replaced with directory > * which has the same object number. > */ I'll move it here: vp =3D ZTOV(zp); > - if (ZTOV(zp) !=3D NULL && > - ZTOV(zp)->v_type !=3D IFTOVT((mode_t)zp->z_mode)) { > + if (vp !=3D NULL && > + vp->v_type !=3D IFTOVT((mode_t)zp->z_mode)) { Those two lines should now fit into one. Other than that looks good. --=20 Pawel Jakub Dawidek http://www.wheelsystems.com FreeBSD committer http://www.FreeBSD.org Am I Evil? Yes, I Am! http://yomoli.com --BEa57a89OpeoUzGD Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (FreeBSD) iEYEARECAAYFAk5VbIIACgkQForvXbEpPzSd3wCeOn8hrtJqtp4Gq2HYhplbSYCC kjUAoLPoBpdk+99tguJ+N+iihBilecZ2 =VVUk -----END PGP SIGNATURE----- --BEa57a89OpeoUzGD-- From owner-zfs-devel@FreeBSD.ORG Tue Aug 30 08:56:11 2011 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 75C9A106564A for ; Tue, 30 Aug 2011 08:56:11 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from mail.vx.sk (mail.vx.sk [IPv6:2a01:4f8:100:1043::3]) by mx1.freebsd.org (Postfix) with ESMTP id 10CF98FC13 for ; Tue, 30 Aug 2011 08:56:11 +0000 (UTC) Received: from core.vx.sk (localhost [127.0.0.1]) by mail.vx.sk (Postfix) with ESMTP id 4085418D4C3 for ; Tue, 30 Aug 2011 10:56:10 +0200 (CEST) X-Virus-Scanned: amavisd-new at mail.vx.sk Received: from mail.vx.sk ([127.0.0.1]) by core.vx.sk (mail.vx.sk [127.0.0.1]) (amavisd-new, port 10024) with LMTP id jBccCGwa18Vp for ; Tue, 30 Aug 2011 10:56:04 +0200 (CEST) Received: from [10.0.3.3] (188-167-66-148.dynamic.chello.sk [188.167.66.148]) by mail.vx.sk (Postfix) with ESMTPSA id 9C1D518D4A3 for ; Tue, 30 Aug 2011 10:56:04 +0200 (CEST) Message-ID: <4E5CA59F.7010705@FreeBSD.org> Date: Tue, 30 Aug 2011 10:55:59 +0200 From: Martin Matuska User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:6.0) Gecko/20110812 Thunderbird/6.0 MIME-Version: 1.0 To: zfs-devel@FreeBSD.org X-Enigmail-Version: 1.3.1 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Cc: Subject: List of recently fixed Oracle bugs X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 Aug 2011 08:56:11 -0000 Oracle has upgraded ZFS in Solaris 10 to version 29 (still not including deduplication). Here is a list of fixed bugs that I was manage to get that have been not fixed in OpenSolaris or Illumos (e.g. disabled zfs vdev cache). Some of the bugs are not relevant to FreeBSD as we have already fixed the issues a different way. 6651136 zfs_link_destroy() should use reader vn_vfsrlock instead of writer vn_vfswlock http://wesunsolve.net/bugid/id/6651136 6998684 deadlock between zfs_inactive() and zfs_write()/as_fault() http://wesunsolve.net/bugid/id/6998684 6914204 zfs commands hang trying to get spa_errlog_lock owned by sync_thread stalled by ZIO error http://wesunsolve.net/bugid/id/6914204 6920295 possible deadlock between ZFS range lock and address space a_lock http://wesunsolve.net/bugid/id/6920295 6931697 ZFS should keep writing data to vdevs with an active hot spare http://wesunsolve.net/bugid/id/6931697 7019356 extra VN_RELE in zfs_setattr() if user has exceeded user quota and file has extended attributes http://wesunsolve.net/bugid/id/7019356 7019358 small race in zfs_sa_upgrade http://wesunsolve.net/bugid/id/7019358 7019370 zfs_aclset_common could cause creation of unnecessary SA layout http://wesunsolve.net/bugid/id/7019370 6700597 zfs send -R | zfs receive -d will fail if any of the file systems have non default mountpoints http://wesunsolve.net/bugid/id/6700597 6883722 want 'zfs recv -o prop=value' to set initial property values of received dataset http://wesunsolve.net/bugid/id/6883722 6886341 want a 'zfs send' option to ignore local property settings when restoring from a backup http://wesunsolve.net/bugid/id/6886341 6900937 ZFS hang waiting for exclusive access to the dataset http://wesunsolve.net/bugid/id/6900937 6947432 zfs list could accept "fs" as synonym for "filesystem" http://wesunsolve.net/bugid/id/6947432 6961707 ZFS rpool panic at zfs: allocating allocated segment http://wesunsolve.net/bugid/id/6961707 6986538 "zfs holds -r" doesn't work http://wesunsolve.net/bugid/id/6986538 6992148 zfs diff shouldn't insist on finding a .zfs/shares http://wesunsolve.net/bugid/id/6992148 7009010 unable to delete ZFS file on NFSv4 share once refquota has been reached http://wesunsolve.net/bugid/id/7009010 -- Martin Matuska FreeBSD committer http://blog.vx.sk From owner-zfs-devel@FreeBSD.ORG Wed Sep 28 08:59:14 2011 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1BE2C106564A; Wed, 28 Sep 2011 08:59:14 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from mail.vx.sk (mail.vx.sk [IPv6:2a01:4f8:100:1043::3]) by mx1.freebsd.org (Postfix) with ESMTP id 8A30A8FC12; Wed, 28 Sep 2011 08:59:10 +0000 (UTC) Received: from core.vx.sk (localhost [127.0.0.1]) by mail.vx.sk (Postfix) with ESMTP id ACEE21A5FFF; Wed, 28 Sep 2011 10:59:09 +0200 (CEST) X-Virus-Scanned: amavisd-new at mail.vx.sk Received: from mail.vx.sk ([127.0.0.1]) by core.vx.sk (mail.vx.sk [127.0.0.1]) (amavisd-new, port 10024) with LMTP id td0TImwpZpSF; Wed, 28 Sep 2011 10:59:07 +0200 (CEST) Received: from [10.0.3.3] (188-167-66-148.dynamic.chello.sk [188.167.66.148]) by mail.vx.sk (Postfix) with ESMTPSA id 6024C1A5FF6; Wed, 28 Sep 2011 10:59:07 +0200 (CEST) Message-ID: <4E82E1DA.1000009@FreeBSD.org> Date: Wed, 28 Sep 2011 10:59:06 +0200 From: Martin Matuska User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:6.0.2) Gecko/20110902 Thunderbird/6.0.2 MIME-Version: 1.0 To: bug-followup@FreeBSD.org X-Enigmail-Version: 1.3.2 Content-Type: multipart/mixed; boundary="------------020904070709010904030807" Cc: zfs-devel@FreeBSD.org Subject: [PATCH] Re: bin/160400: zfs(1): zfs rename dumps core X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 28 Sep 2011 08:59:14 -0000 This is a multi-part message in MIME format. --------------020904070709010904030807 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit The reproduction of this issue can be simplified and similiar case has already been reported on freebsd-fs@: http://lists.freebsd.org/pipermail/freebsd-fs/2010-September/009379.html http://lists.freebsd.org/pipermail/freebsd-fs/2011-August/012291.html It is important that the mountpoint of the dataset to be renamed is set to none or legacy, otherwise this assertion is not triggered. # zfs create -o mountpoint=none tank/a # zfs create tank/a/b # zfs rename tank/a tank/c Assertion failed: (!clp->cl_alldependents), file /usr/src/cddl/lib/libzfs/../../../cddl/contrib/opensolaris/lib/libzfs/common/libzfs_changelist.c, line 470. Abort (core dumped) As to my code examination, the assertion in libzfs_changelist.c: change_one() is inproper or invalid. The code path is called with clp->cl_sorted = B_FALSE and cl_alldependents = B_TRUE from changelist_gather() if mountpoint is legacy or none. This misbehaviour was introduced in this OpenSolaris commit: changeset: 10196:210962933dfd user: William Gorrell date: Wed Jul 29 08:49:33 2009 -0600 summary: 6612218 inherited zfs set mountpoint mounts children before parent https://github.com/illumos/illumos-gate/commit/3cc4a7920cf40de22a5c8c465a4676b2b7f620dd#usr/src/lib/libzfs/common/libzfs_changelist.c I am generally putting in question why that assertion was in this place at it concerns exclusively the case with mountpoint=none or mountpoint=legacy and it manages only the order of the datasets being processed. I suggest removing the assertion as we can safely try to unmount the dependent filesystems. Please review the attached patch. -- Martin Matuska FreeBSD committer http://blog.vx.sk --------------020904070709010904030807 Content-Type: text/plain; name="libzfs_changelist.c.patch" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="libzfs_changelist.c.patch" SW5kZXg6IGNkZGwvY29udHJpYi9vcGVuc29sYXJpcy9saWIvbGliemZzL2NvbW1vbi9saWJ6 ZnNfY2hhbmdlbGlzdC5jCj09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT0KLS0tIGNkZGwvY29udHJpYi9vcGVuc29s YXJpcy9saWIvbGliemZzL2NvbW1vbi9saWJ6ZnNfY2hhbmdlbGlzdC5jCShyZXZpc2lvbiAy MjU2ODkpCisrKyBjZGRsL2NvbnRyaWIvb3BlbnNvbGFyaXMvbGliL2xpYnpmcy9jb21tb24v bGliemZzX2NoYW5nZWxpc3QuYwkod29ya2luZyBjb3B5KQpAQCAtNDY3LDcgKzQ2Nyw2IEBA IGNoYW5nZV9vbmUoemZzX2hhbmRsZV90ICp6aHAsIHZvaWQgKmRhdGEpCiAJCQkgKiBUaGlz IGlzIG5lY2Vzc2FyeSB3aGVuIHRoZSBvcmlnaW5hbCBtb3VudHBvaW50CiAJCQkgKiBpcyBs ZWdhY3kgb3Igbm9uZS4KIAkJCSAqLwotCQkJQVNTRVJUKCFjbHAtPmNsX2FsbGRlcGVuZGVu dHMpOwogCQkJdmVyaWZ5KHV1X2xpc3RfaW5zZXJ0X2JlZm9yZShjbHAtPmNsX2xpc3QsCiAJ CQkgICAgdXVfbGlzdF9maXJzdChjbHAtPmNsX2xpc3QpLCBjbikgPT0gMCk7CiAJCX0K --------------020904070709010904030807-- From owner-zfs-devel@FreeBSD.ORG Wed Sep 28 10:42:47 2011 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 63FB91065673; Wed, 28 Sep 2011 10:42:47 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from mail.vx.sk (mail.vx.sk [IPv6:2a01:4f8:100:1043::3]) by mx1.freebsd.org (Postfix) with ESMTP id 135A48FC1E; Wed, 28 Sep 2011 10:42:47 +0000 (UTC) Received: from core.vx.sk (localhost [127.0.0.1]) by mail.vx.sk (Postfix) with ESMTP id 6A3D9150617; Wed, 28 Sep 2011 12:42:46 +0200 (CEST) X-Virus-Scanned: amavisd-new at mail.vx.sk Received: from mail.vx.sk ([127.0.0.1]) by core.vx.sk (mail.vx.sk [127.0.0.1]) (amavisd-new, port 10024) with LMTP id gl27T9Qr6xW1; Wed, 28 Sep 2011 12:42:44 +0200 (CEST) Received: from [10.0.3.3] (188-167-66-148.dynamic.chello.sk [188.167.66.148]) by mail.vx.sk (Postfix) with ESMTPSA id 23CFD150608; Wed, 28 Sep 2011 12:42:44 +0200 (CEST) Message-ID: <4E82FA22.5020406@FreeBSD.org> Date: Wed, 28 Sep 2011 12:42:42 +0200 From: Martin Matuska User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:6.0.2) Gecko/20110902 Thunderbird/6.0.2 MIME-Version: 1.0 To: zfs-devel@FreeBSD.org, Pawel Jakub Dawidek X-Enigmail-Version: 1.3.2 Content-Type: multipart/mixed; boundary="------------020405000101050706040408" Cc: Subject: [PATCH] Illumos changeset #13469 (dmu_tx.c) X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 28 Sep 2011 10:42:47 -0000 This is a multi-part message in MIME format. --------------020405000101050706040408 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Illumos has now provided a fix for the Issue #1475 regarding invalid spill blkptr: https://www.illumos.org/issues/1475 changeset: 13469:b8e89e5c4167 tag: tip user: Albert Lee date: Sun Sep 25 03:07:35 2011 -0400 summary: 1475 zfs spill block hold can access invalid spill blkptr https://github.com/illumos/illumos-gate/commit/9dccfd2a04cd1645e2616b7307b3a88041aba991 A partial fix by pjd was already in our code: http://p4db.freebsd.org/changeView.cgi?CH=185940 http://p4db.freebsd.org/changeView.cgi?CH=185942 I suggest importing the full fix to match the Illumos version. Please review and/or comment the attached patch. -- Martin Matuska FreeBSD committer http://blog.vx.sk --------------020405000101050706040408 Content-Type: text/plain; name="dmu_tx.c.patch" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="dmu_tx.c.patch" SW5kZXg6IHN5cy9jZGRsL2NvbnRyaWIvb3BlbnNvbGFyaXMvdXRzL2NvbW1vbi9mcy96ZnMv ZG11X3R4LmMKPT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PQotLS0gc3lzL2NkZGwvY29udHJpYi9vcGVuc29sYXJp cy91dHMvY29tbW9uL2ZzL3pmcy9kbXVfdHguYwkocmV2aXNpb24gMjI1Njg5KQorKysgc3lz L2NkZGwvY29udHJpYi9vcGVuc29sYXJpcy91dHMvY29tbW9uL2ZzL3pmcy9kbXVfdHguYwko d29ya2luZyBjb3B5KQpAQCAtMjEsNiArMjEsOSBAQAogLyoKICAqIENvcHlyaWdodCAoYykg MjAwNSwgMjAxMCwgT3JhY2xlIGFuZC9vciBpdHMgYWZmaWxpYXRlcy4gQWxsIHJpZ2h0cyBy ZXNlcnZlZC4KICAqLworLyoKKyAqIENvcHlyaWdodCAyMDExIE5leGVudGEgU3lzdGVtcywg SW5jLiAgQWxsIHJpZ2h0cyByZXNlcnZlZC4KKyAqLwogCiAjaW5jbHVkZSA8c3lzL2RtdS5o PgogI2luY2x1ZGUgPHN5cy9kbXVfaW1wbC5oPgpAQCAtNjc2LDYgKzY3OSw4IEBACiAJQVNT RVJUM1AoZG11X290W2RuLT5kbl90eXBlXS5vdF9ieXRlc3dhcCwgPT0sIHphcF9ieXRlc3dh cCk7CiAKIAlpZiAoZG4tPmRuX21heGJsa2lkID09IDAgJiYgIWFkZCkgeworCQlibGtwdHJf dCAqYnA7CisKIAkJLyoKIAkJICogSWYgdGhlcmUgaXMgb25seSBvbmUgYmxvY2sgIChpLmUu IHRoaXMgaXMgYSBtaWNyby16YXApCiAJCSAqIGFuZCB3ZSBhcmUgbm90IGFkZGluZyBhbnl0 aGluZywgdGhlIGFjY291bnRpbmcgaXMgc2ltcGxlLgpAQCAtNjkwLDE0ICs2OTUsMTMgQEAK IAkJICogVXNlIG1heCBibG9jayBzaXplIGhlcmUsIHNpbmNlIHdlIGRvbid0IGtub3cgaG93 IG11Y2gKIAkJICogdGhlIHNpemUgd2lsbCBjaGFuZ2UgYmV0d2VlbiBub3cgYW5kIHRoZSBk YnVmIGRpcnR5IGNhbGwuCiAJCSAqLworCQlicCA9ICZkbi0+ZG5fcGh5cy0+ZG5fYmxrcHRy WzBdOwogCQlpZiAoZHNsX2RhdGFzZXRfYmxvY2tfZnJlZWFibGUoZG4tPmRuX29ianNldC0+ b3NfZHNsX2RhdGFzZXQsCi0JCSAgICAmZG4tPmRuX3BoeXMtPmRuX2Jsa3B0clswXSwKLQkJ ICAgIGRuLT5kbl9waHlzLT5kbl9ibGtwdHJbMF0uYmxrX2JpcnRoKSkgeworCQkgICAgYnAs IGJwLT5ibGtfYmlydGgpKQogCQkJdHhoLT50eGhfc3BhY2VfdG9vdmVyd3JpdGUgKz0gU1BB X01BWEJMT0NLU0laRTsKLQkJfSBlbHNlIHsKKwkJZWxzZQogCQkJdHhoLT50eGhfc3BhY2Vf dG93cml0ZSArPSBTUEFfTUFYQkxPQ0tTSVpFOwotCQl9Ci0JCWlmIChkbi0+ZG5fcGh5cy0+ ZG5fYmxrcHRyWzBdLmJsa19iaXJ0aCkKKwkJaWYgKCFCUF9JU19IT0xFKGJwKSkKIAkJCXR4 aC0+dHhoX3NwYWNlX3RvdW5yZWYgKz0gU1BBX01BWEJMT0NLU0laRTsKIAkJcmV0dXJuOwog CX0KQEAgLTEyNzMsNyArMTI3Nyw2IEBACiB7CiAJZG5vZGVfdCAqZG47CiAJZG11X3R4X2hv bGRfdCAqdHhoOwotCWJsa3B0cl90ICpicDsKIAogCXR4aCA9IGRtdV90eF9ob2xkX29iamVj dF9pbXBsKHR4LCB0eC0+dHhfb2Jqc2V0LCBvYmplY3QsCiAJICAgIFRIVF9TUElMTCwgMCwg MCk7CkBAIC0xMjg2LDE1ICsxMjg5LDE2IEBACiAJLyogSWYgYmxrcHRyIGRvZXNuJ3QgZXhp c3QgdGhlbiBhZGQgc3BhY2UgdG8gdG93cml0ZSAqLwogCWlmICghKGRuLT5kbl9waHlzLT5k bl9mbGFncyAmIEROT0RFX0ZMQUdfU1BJTExfQkxLUFRSKSkgewogCQl0eGgtPnR4aF9zcGFj ZV90b3dyaXRlICs9IFNQQV9NQVhCTE9DS1NJWkU7Ci0JCXR4aC0+dHhoX3NwYWNlX3RvdW5y ZWYgPSAwOwogCX0gZWxzZSB7CisJCWJsa3B0cl90ICpicDsKKwogCQlicCA9ICZkbi0+ZG5f cGh5cy0+ZG5fc3BpbGw7CiAJCWlmIChkc2xfZGF0YXNldF9ibG9ja19mcmVlYWJsZShkbi0+ ZG5fb2Jqc2V0LT5vc19kc2xfZGF0YXNldCwKIAkJICAgIGJwLCBicC0+YmxrX2JpcnRoKSkK IAkJCXR4aC0+dHhoX3NwYWNlX3Rvb3ZlcndyaXRlICs9IFNQQV9NQVhCTE9DS1NJWkU7CiAJ CWVsc2UKIAkJCXR4aC0+dHhoX3NwYWNlX3Rvd3JpdGUgKz0gU1BBX01BWEJMT0NLU0laRTsK LQkJaWYgKGJwLT5ibGtfYmlydGgpCisJCWlmICghQlBfSVNfSE9MRShicCkpCiAJCQl0eGgt PnR4aF9zcGFjZV90b3VucmVmICs9IFNQQV9NQVhCTE9DS1NJWkU7CiAJfQogfQo= --------------020405000101050706040408-- From owner-zfs-devel@FreeBSD.ORG Thu Sep 29 12:48:04 2011 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 577A11065670 for ; Thu, 29 Sep 2011 12:48:04 +0000 (UTC) (envelope-from hellosamirdesai@gmail.com) Received: from mail-vx0-f182.google.com (mail-vx0-f182.google.com [209.85.220.182]) by mx1.freebsd.org (Postfix) with ESMTP id 08F268FC17 for ; Thu, 29 Sep 2011 12:48:03 +0000 (UTC) Received: by vcbf13 with SMTP id f13so558918vcb.13 for ; Thu, 29 Sep 2011 05:48:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:content-type; bh=Ckv/P5PBv7EhZzDy26txSA/Ha/47RhaVVy7MjgipsAw=; b=uqwc4HvCA5ebc2lAP4qAlcmj2NNAqbwCF+y09UEV6sxQKoY7FnYp/FnuoFxT2i1Fdt WFuwEjeJExqg+vxQmSySw0Hihd9kBhr1nlzSVmhbWBW6oJEg/MqPRquljnto5ZekMgEQ ji8g3zYk0UcP/I+zBcfR6yT/iFXojVp2fcuM4= MIME-Version: 1.0 Received: by 10.52.35.180 with SMTP id i20mr6033992vdj.198.1317299023421; Thu, 29 Sep 2011 05:23:43 -0700 (PDT) Received: by 10.52.182.1 with HTTP; Thu, 29 Sep 2011 05:23:43 -0700 (PDT) Date: Thu, 29 Sep 2011 17:53:43 +0530 Message-ID: From: Samir Desai To: zfs-devel@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: [BUG] zfs panic in dsl_dir_open_spa() because of NULL ptr X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 Sep 2011 12:48:04 -0000 hi We are seeing ZFS v28 panic on FreeBSD, which seems to be caused by a race condition between a thread loading spa and another trying to access it. The two thread stacks trace these paths 1. zfs_ioc_snapshot -> zfs_unmount_snap -> ...-> spa_load ->vdev_open ..... -> vdev_geom_open ......-> biowait 2. zfs_secpolicy_write_perms -> ...-> dmu_objset_snapshot -> ... ->dsl_dir_open_spa -> _sx_slock the two threads should be serialized by spa_namespace_lock, but apparently the first thread, which is initializing spa, drops the lock in vdev_geom_open(). This lets the second thread come in and see and uninitialized spa. The spa->spa_dsl_pool pointer is NULL. The second thread tries to take lock in dsl_dir_open_spa() dp = spa_get_dsl(spa); rw_enter(&dp->dp_config_rwlock, RW_READER); This panics because "dp" is NULL Should we be dropping the spa_namespace_lock in vdev_geom_open() ? What serializes the spa operations ? Some searching around the net showed up this thread, which has very similar problem http://freebsd.1045724.n5.nabble.com/ZFS-RAID-Z-panic-on-vdev-failure-subsequent-panics-and-hangs-td4032243.html Note that one of the devices was in UNAVAIL state because we had removed it on purpose Here are the stacktraces of the two threads -------------------------- thread 1 -------------------------------------------------------------- Thread 537 (Thread 100398): #0 sched_switch (td=0xffffff0002d118c0, newtd=0xffffff00023c38c0, flags=Variable "flags" is not available. ) at ../../../kern/sched_ule.c:1858 #1 0xffffffff805cb166 in mi_switch (flags=260, newtd=0x0) at ../../../kern/kern_synch.c:449 #2 0xffffffff805fe7b2 in sleepq_timedwait (wchan=0xffffff00027bb0e8, pri=76) at ../../../kern/subr_sleepqueue.c:644 #3 0xffffffff805cb6c1 in _sleep (ident=0xffffff00027bb0e8, lock=0xffffff800021a2b0, priority=Variable "priority" is not av ailable. ) at ../../../kern/kern_synch.c:230 #4 0xffffffff80638680 in biowait (bp=0xffffff00027bb0e8, wchan=0xffffffff80f09406 "vdev_geom_io") at ../../../kern/vfs_bio .c:3189 #5 0xffffffff80ea77d3 in vdev_geom_open_by_path (vd=0x1e07c0000, check_guid=262144) at /usr/src/sys/modules/zfs/../../cddl /contrib/opensolaris/uts/common/fs/zfs/vdev_geom.c:388 #6 0xffffffff80ea81f2 in vdev_geom_open (vd=0xffffff00027bce00, psize=0xffffff804d00a670, ashift=0xffffff804d00a668) at /u sr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_geom.c:362 #7 0xffffffff80e5e877 in vdev_open (vd=0xffffff0002d86800) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/ common/fs/zfs/vdev.c:1248 #8 0xffffffff80e5f411 in vdev_compact_children (pvd=0xffffff0002d86800) at /usr/src/sys/modules/zfs/../../cddl/contrib/ope nsolaris/uts/common/fs/zfs/vdev.c:269 #9 0xffffffff80e681cc in zap_idx_to_blk (zap=0xffffff0002747000, idx=18446743525245626136, valp=0xffffff804d00a720) at /us r/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zap.c:297 #10 0xffffffff80e5e877 in vdev_open (vd=0xffffffff80e681cc) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/ common/fs/zfs/vdev.c:1248 #11 0xffffffff80e4fef1 in spa_load (spa=0xffffff0002747000, state=SPA_LOAD_OPEN, type=SPA_IMPORT_EXISTING, mosconfig=411852 80) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/spa.c:1922 #12 0xffffffff80e513c2 in spa_open_common (pool=0x1
, spapp=0x0, tag=0x1, nvpolicy=0xffffff00027 47000, config=0x0) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/spa.c:2376 #13 0xffffffff80e516ba in spa_open_common (pool=0xffffff8000b0b000 "sp3", spapp=0xffffff804d00a920, tag=0xffffffff80ef7800, nvpolicy=Variable "nvpolicy" is not available. ) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/spa.c:2439 #14 0xffffffff80e8e459 in zfs_unmount_snap (name=0xffffff8000b0b000 "sp3", arg=Variable "arg" is not available. ) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_ioctl.c:3112 #15 0xffffffff80e8e6b3 in zfs_ioc_snapshot (zc=0x0) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/f s/zfs/zfs_ioctl.c:2397 #16 0xffffffff8054cedb in devfs_ioctl_f (fp=0xffffff0023b20c30, com=3583531538, data=Variable "data" is not available. ) at ../../../fs/devfs/devfs_vnops.c:678 #17 0xffffffff806043b2 in kern_ioctl (td=Variable "td" is not available. ) at file.h:262 #18 0xffffffff806045ed in ioctl (td=0xffffff0002d118c0, uap=0xffffff804d00abb0) at ../../../kern/sys_generic.c:679 #19 0xffffffff80600dc5 in syscallenter (td=0xffffff0002d118c0, sa=0xffffff804d00aba0) at ../../../kern/subr_trap.c:315 #20 0xffffffff808acb7b in syscall (frame=0xffffff804d00ac40) at ../../../amd64/amd64/trap.c:888 #21 0xffffffff808953c2 in Xfast_syscall () at ../../../amd64/amd64/exception.S:377 #22 0x0000000801104f8c in ?? () Previous frame inner to this frame (corrupt stack?) ------------------------------------------------------------------------------------------------------------------- ----------------------------------- thread 2 ------------------------ panic thread ---------------------------- Thread 548 (Thread 100432): #0 doadump () at pcpu.h:224 #1 0xffffffff805c28ae in boot (howto=260) at ../../../kern/kern_shutdown.c:419 #2 0xffffffff805c2ce1 in panic (fmt=Variable "fmt" is not available. ) at ../../../kern/kern_shutdown.c:592 #3 0xffffffff808ac720 in trap_fatal (frame=0xc, eva=Variable "eva" is not available. ) at ../../../amd64/amd64/trap.c:783 #4 0xffffffff808acaff in trap_pfault (frame=0xffffff804d0b4600, usermode=0) at ../../../amd64/amd64/trap.c:699 #5 0xffffffff808acfdf in trap (frame=0xffffff804d0b4600) at ../../../amd64/amd64/trap.c:449 #6 0xffffffff808950e4 in calltrap () at ../../../amd64/amd64/exception.S:224 #7 0xffffffff805cae61 in _sx_slock (sx=0x368, opts=0, file=0xffffffff80ef9ee0 "/home/chs/freebsd/8.2/src/sys/modules/zfs/. ./../cddl/contrib/opensolaris/uts/common/fs/zfs/dsl_dir.c", line=334) at ../../../kern/kern_sx.c:239 #8 0xffffffff80e30a7b in dsl_dir_open_spa (spa=0xffffff0002747000, name=Variable "name" is not available. ) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/dsl_dir.c:396 #9 0xffffffff80e3498b in dsl_dataset_hold (name=Variable "name" is not available. ) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/dsl_dataset.c:642 #10 0xffffffff80e26973 in dmu_objset_snapshot (fsname=0xffffff0002d1d460 "\200}\xb6\200\xff\xff\xff\xff", snapname=0x0, tag =0xffffffff80ef9ee0 "/home/chs/freebsd/8.2/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/dsl_dir.c", props=0x14e, recursive=47305824, temporary=1, cleanup_fd=0) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts /common/fs/zfs/dmu_objset.c:946 #11 0xffffffff80e8d3ab in zfs_ioc_vdev_set_state (zc=0xffffff804d0b4918) at /usr/src/sys/modules/zfs/../../cddl/contrib/ope nsolaris/uts/common/fs/zfs/zfs_ioctl.c:1629 #12 0xffffffff80e8e576 in zfs_secpolicy_write_perms (name=Variable "name" is not available. ) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_ioctl.c:360 #13 0xffffffff8054cedb in devfs_ioctl_f (fp=0xffffffff80f0e820, com=18446742974794065712, data=Variable "data" is not avail able. ) at ../../../fs/devfs/devfs_vnops.c:678 #14 0xffffffff806043b2 in kern_ioctl (td=Variable "td" is not available. ) at file.h:262 #15 0xffffffff806045ed in ioctl (td=0xffffff0002d1d460, uap=0xffffff804d0b4bb0) at ../../../kern/sys_generic.c:679 #16 0xffffffff80600dc5 in syscallenter (td=0xffffff0002d1d460, sa=0xffffff804d0b4ba0) at ../../../kern/subr_trap.c:315 #17 0xffffffff808acb7b in syscall (frame=0xffffff804d0b4c40) at ../../../amd64/amd64/trap.c:888 #18 0xffffffff808953c2 in Xfast_syscall () at ../../../amd64/amd64/exception.S:377 #19 0x0000000801104f8c in ?? () ---------------------------------------------------------------------------------------------------- here is the spa structure --------------------------------- (kgdb) print {spa_t}0xffffff0002747000 $1 = {spa_name = "sp3", '\0' , spa_avl = {avl_child = {0x0, 0x0}, avl_pcb = 18446742974239113477}, spa_config = 0xffffff0002b73820, spa_config_syncing = 0x0, spa_config_splitting = 0x0, spa_load_info = 0xffffff0002715c60, spa_config_txg = 10, spa_sync_pass = 0, spa_state = POOL_STATE_ACTIVE, spa_inject_ref = 0, spa_sync_on = 0 '\0', spa_load_state = SPA_LOAD_OPEN, spa_import_flags = 0, spa_zio_taskq = {{ 0xffffff0023c09790, 0x0, 0xffffff0023c09330, 0x0}, {0xffffff0023c09800, 0x0, 0xffffff00239b1050, 0x0}, { 0xffffff0023c093b0, 0xffffff0023c091e0, 0xffffff0023a1f900, 0xffffff0023c09260}, {0xffffff0023a1f570, 0x0, 0xffffff0023c09620, 0x0}, {0xffffff0002b9a830, 0x0, 0xffffff0023a1f530, 0x0}, {0xffffff0023a1f7d0, 0x0, 0xffffff0023a1f960, 0x0}}, spa_dsl_pool = 0x0, spa_normal_class = 0xffffff0002feca80, spa_log_class = 0xffffff0002bf8b40, spa_first_txg = 0, spa_final_txg = 18446744073709551615, spa_freeze_txg = 18446744073709551615, spa_load_max_txg = 18446744073709551615, spa_claim_max_txg = 0, spa_loaded_ts = { tv_sec = 1317106605, tv_nsec = 601195533}, spa_meta_objset = 0x0, spa_vdev_txg_list = {tl_lock = {lock_object = { lo_name = 0xffffffff80f072bb "tl->tl_lock", lo_flags = 40960000, lo_data = 0, lo_witness = 0x0}, sx_lock = 1}, tl_offset = 1040, tl_head = {0x0, 0x0, 0x0, 0x0}}, spa_root_vdev = 0xffffff0002d89800, spa_load_guid = 4958651374771581097, spa_config_dirty_list = {list_size = 1800, list_offset = 1096, list_head = { list_next = 0xffffff00027472e0, list_prev = 0xffffff00027472e0}}, spa_state_dirty_list = {list_size = 1800, list_offset = 1112, list_head = {list_next = 0xffffff0002747300, list_prev = 0xffffff0002747300}}, spa_spares = { sav_object = 0, sav_config = 0x0, sav_vdevs = 0x0, sav_count = 0, sav_sync = 0, sav_pending = 0x0, sav_npending = 0}, spa_l2cache = {sav_object = 0, sav_config = 0x0, sav_vdevs = 0x0, sav_count = 0, sav_sync = 0, sav_pending = 0x0, sav_npending = 0}, spa_config_object = 0, spa_config_generation = 0, spa_syncing_txg = 0, spa_deferred_bpobj = { bpo_lock = {lock_object = {lo_name = 0x0, lo_flags = 0, lo_data = 0, lo_witness = 0x0}, sx_lock = 0}, bpo_os = 0x0, bpo_object = 0, bpo_epb = 0, bpo_havecomp = 0 '\0', bpo_havesubobj = 0 '\0', bpo_phys = 0x0, bpo_dbuf = 0x0, bpo_cached_dbuf = 0x0}, spa_free_bplist = {{bpl_lock = {lock_object = {lo_name = 0xffffffff80f05bd9 "bpl->bpl_lock", lo_flags = 40960000, lo_data = 0, lo_witness = 0x0}, sx_lock = 1}, bpl_list = {list_size = 144, list_offset = 128, list_head = {list_next = 0xffffff0002747408, list_prev = 0xffffff0002747408}}}, {bpl_lock = { lock_object = {lo_name = 0xffffffff80f05bd9 "bpl->bpl_lock", lo_flags = 40960000, lo_data = 0, lo_witness = 0x0}, sx_lock = 1}, bpl_list = {list_size = 144, list_offset = 128, list_head = {list_next = 0xffffff0002747448, list_prev = 0xffffff0002747448}}}, {bpl_lock = {lock_object = {lo_name = 0xffffffff80f05bd9 "bpl->bpl_lock", lo_flags = 40960000, lo_data = 0, lo_witness = 0x0}, sx_lock = 1}, bpl_list = {list_size = 144, list_offset = 128, list_head = {list_next = 0xffffff0002747488, list_prev = 0xffffff0002747488}}}, {bpl_lock = { lock_object = {lo_name = 0xffffffff80f05bd9 "bpl->bpl_lock", lo_flags = 40960000, lo_data = 0, lo_witness = 0x0}, sx_lock = 1}, bpl_list = {list_size = 144, list_offset = 128, list_head = {list_next = 0xffffff00027474c8, list_prev = 0xffffff00027474c8}}}}, spa_ubsync = {ub_magic = 0, ub_version = 28, ub_txg = 0, ub_guid_sum = 0, ub_timestamp = 0, ub_rootbp = {blk_dva = {{dva_word = {0, 0}}, {dva_word = {0, 0}}, {dva_word = {0, 0}}}, blk_prop = 0, blk_pad = {0, 0}, blk_phys_birth = 0, blk_birth = 0, blk_fill = 0, blk_cksum = {zc_word = {0, 0, 0, 0}}}, ---Type to continue, or q to quit--- ub_software_version = 0}, spa_uberblock = {ub_magic = 0, ub_version = 0, ub_txg = 0, ub_guid_sum = 0, ub_timestamp = 0, ub_rootbp = {blk_dva = {{dva_word = {0, 0}}, {dva_word = {0, 0}}, {dva_word = {0, 0}}}, blk_prop = 0, blk_pad = {0, 0}, blk_phys_birth = 0, blk_birth = 0, blk_fill = 0, blk_cksum = {zc_word = {0, 0, 0, 0}}}, ub_software_version = 0}, spa_extreme_rewind = 0, spa_last_io = 0, spa_scrub_lock = {lock_object = { lo_name = 0xffffffff80f07134 "spa->spa_scrub_lock", lo_flags = 40960000, lo_data = 0, lo_witness = 0x0}, sx_lock = 1}, spa_scrub_inflight = 0, spa_scrub_io_cv = {cv_description = 0xffffffff80f071a2 "spa->spa_scrub_io_cv)", cv_waiters = 0}, spa_scrub_active = 0 '\0', spa_scrub_type = 0 '\0', spa_scrub_finished = 0 '\0', spa_scrub_started = 0 '\0', spa_scrub_reopen = 0 '\0', spa_scan_pass_start = 0, spa_scan_pass_exam = 0, spa_async_lock = { lock_object = {lo_name = 0xffffffff80f070b2 "spa->spa_async_lock", lo_flags = 40960000, lo_data = 0, lo_witness = 0x0}, sx_lock = 1}, spa_async_thread = 0x0, spa_async_suspended = 0, spa_async_cv = { cv_description = 0xffffffff80f07179 "spa->spa_async_cv)", cv_waiters = 0}, spa_async_tasks = 0, spa_root = 0x0, spa_ena = 0, spa_last_open_failed = 0, spa_last_ubsync_txg = 0, spa_last_ubsync_txg_ts = 0, spa_load_txg = 0, spa_load_txg_ts = 0, spa_load_meta_errors = 0, spa_load_data_errors = 0, spa_verify_min_txg = 0, spa_errlog_lock = { lock_object = {lo_name = 0xffffffff80f070de "spa->spa_errlog_lock", lo_flags = 40960000, lo_data = 0, lo_witness = 0x0}, sx_lock = 1}, spa_errlog_last = 0, spa_errlog_scrub = 0, spa_errlist_lock = {lock_object = { lo_name = 0xffffffff80f070c7 "spa->spa_errlist_lock", lo_flags = 40960000, lo_data = 0, lo_witness = 0x0}, sx_lock = 1}, spa_errlist_last = {avl_root = 0x0, avl_compar = 0xffffffff80e4d180 , avl_offset = 40, avl_numnodes = 0, avl_size = 64}, spa_errlist_scrub = {avl_root = 0x0, avl_compar = 0xffffffff80e4d180 , avl_offset = 40, avl_numnodes = 0, avl_size = 64}, spa_deflate = 0, spa_history = 0, spa_history_lock = {lock_object = { lo_name = 0xffffffff80f070f4 "spa->spa_history_lock", lo_flags = 40960000, lo_data = 0, lo_witness = 0x0}, sx_lock = 1}, spa_pending_vdev = 0x0, spa_props_lock = {lock_object = { lo_name = 0xffffffff80f0711f "spa->spa_props_lock", lo_flags = 40960000, lo_data = 0, lo_witness = 0x0}, sx_lock = 1}, spa_pool_props_object = 0, spa_bootfs = 0, spa_failmode = 0, spa_delegation = 0, spa_config_list = { list_size = 24, list_offset = 0, list_head = {list_next = 0xffffff0002735000, list_prev = 0xffffff0002735000}}, spa_async_zio_root = 0xffffff00028b7760, spa_suspend_zio_root = 0x0, spa_suspend_lock = {lock_object = { lo_name = 0xffffffff80f07149 "spa->spa_suspend_lock", lo_flags = 40960000, lo_data = 0, lo_witness = 0x0}, sx_lock = 1}, spa_suspend_cv = {cv_description = 0xffffffff80f071ba "spa->spa_suspend_cv)", cv_waiters = 0}, spa_suspended = 0 '\0', spa_claiming = 0 '\0', spa_is_root = 0, spa_minref = 0, spa_mode = 1, spa_log_state = SPA_LOG_UNKNOWN, spa_autoexpand = 0, spa_ddt = {0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, spa_ddt_stat_object = 0, spa_dedup_ditto = 0, spa_dedup_checksum = 0, spa_dspace = 0, spa_vdev_top_lock = {lock_object = { lo_name = 0xffffffff80f07160 "spa->spa_vdev_top_lock", lo_flags = 40960000, lo_data = 0, lo_witness = 0x0}, sx_lock = 1}, spa_proc_lock = {lock_object = {lo_name = 0xffffffff80f0710b "spa->spa_proc_lock", lo_flags = 40960000, lo_data = 0, lo_witness = 0x0}, sx_lock = 1}, spa_proc_cv = {cv_description = 0xffffffff80f0718e "spa->spa_proc_cv)", ---Type to continue, or q to quit--- cv_waiters = 0}, spa_proc_state = SPA_PROC_NONE, spa_proc = 0xffffffff80b5db60, spa_did = 0, spa_autoreplace = 0, spa_vdev_locks = 0, spa_creation_version = 0, spa_prev_software_version = 0, spa_config_lock = {{scl_lock = { lock_object = {lo_name = 0xffffffff80f071d0 "scl->scl_lock", lo_flags = 40960000, lo_data = 0, lo_witness = 0x0}, sx_lock = 1}, scl_writer = 0xffffff0002d118c0, scl_write_wanted = 0, scl_cv = { cv_description = 0xffffffff80f071e0 "scl->scl_cv)", cv_waiters = 0}, scl_count = {rc_count = 1}}, {scl_lock = { lock_object = {lo_name = 0xffffffff80f071d0 "scl->scl_lock", lo_flags = 40960000, lo_data = 0, lo_witness = 0x0}, sx_lock = 1}, scl_writer = 0xffffff0002d118c0, scl_write_wanted = 0, scl_cv = { cv_description = 0xffffffff80f071e0 "scl->scl_cv)", cv_waiters = 0}, scl_count = {rc_count = 1}}, {scl_lock = { lock_object = {lo_name = 0xffffffff80f071d0 "scl->scl_lock", lo_flags = 40960000, lo_data = 0, lo_witness = 0x0}, sx_lock = 1}, scl_writer = 0xffffff0002d118c0, scl_write_wanted = 0, scl_cv = { cv_description = 0xffffffff80f071e0 "scl->scl_cv)", cv_waiters = 0}, scl_count = {rc_count = 1}}, {scl_lock = { lock_object = {lo_name = 0xffffffff80f071d0 "scl->scl_lock", lo_flags = 40960000, lo_data = 0, lo_witness = 0x0}, sx_lock = 1}, scl_writer = 0xffffff0002d118c0, scl_write_wanted = 0, scl_cv = { cv_description = 0xffffffff80f071e0 "scl->scl_cv)", cv_waiters = 0}, scl_count = {rc_count = 1}}, {scl_lock = { lock_object = {lo_name = 0xffffffff80f071d0 "scl->scl_lock", lo_flags = 40960000, lo_data = 0, lo_witness = 0x0}, sx_lock = 1}, scl_writer = 0xffffff0002d118c0, scl_write_wanted = 0, scl_cv = { cv_description = 0xffffffff80f071e0 "scl->scl_cv)", cv_waiters = 0}, scl_count = {rc_count = 1}}, {scl_lock = { lock_object = {lo_name = 0xffffffff80f071d0 "scl->scl_lock", lo_flags = 40960000, lo_data = 0, lo_witness = 0x0}, sx_lock = 1}, scl_writer = 0xffffff0002d118c0, scl_write_wanted = 0, scl_cv = { cv_description = 0xffffffff80f071e0 "scl->scl_cv)", cv_waiters = 0}, scl_count = {rc_count = 1}}, {scl_lock = { lock_object = {lo_name = 0xffffffff80f071d0 "scl->scl_lock", lo_flags = 40960000, lo_data = 0, lo_witness = 0x0}, sx_lock = 1}, scl_writer = 0xffffff0002d118c0, scl_write_wanted = 0, scl_cv = { cv_description = 0xffffffff80f071e0 "scl->scl_cv)", cv_waiters = 0}, scl_count = {rc_count = 1}}}, spa_refcount = { rc_count = 1}, spa_splitting_newspa = 0} ------------------------------------------------------------------ Here is the panic message ------------------------------------ Fatal trap 12: page fault while in kernel mode cpuid = 0; apic id = 00 fault virtual address = 0x380 fault code = supervisor read data, page not present instruction pointer = 0x20:0xffffffff805cae61 stack pointer = 0x28:0xffffff804d0b46b0 frame pointer = 0x28:0xffffff804d0b4860 code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags = interrupt enabled, resume, IOPL = 0 current process = 1698 (initial thread) trap number = 12 panic: page fault cpuid = 0 KDB: stack backtrace: #0 0xffffffff805f4dfe at kdb_backtrace+0x5e #1 0xffffffff805c2cf7 at panic+0x187 #2 0xffffffff808ac720 at trap_fatal+0x290 #3 0xffffffff808acaff at trap_pfault+0x28f #4 0xffffffff808acfdf at trap+0x3df #5 0xffffffff808950e4 at calltrap+0x8 #6 0xffffffff80e3498b at dsl_dataset_hold+0x3b #7 0xffffffff80e26973 at dmu_objset_hold+0x23 #8 0xffffffff80e8d3ab at zfs_ioc_objset_stats+0x2b #9 0xffffffff80e8e576 at zfsdev_ioctl+0xe6 #10 0xffffffff8054cedb at devfs_ioctl_f+0x7b #11 0xffffffff806043b2 at kern_ioctl+0x102 #12 0xffffffff806045ed at ioctl+0xfd #13 0xffffffff80600dc5 at syscallenter+0x1e5 #14 0xffffffff808acb7b at syscall+0x4b #15 0xffffffff808953c2 at Xfast_syscall+0xe2 Uptime: 1m12s Dumping 1023 MB (2 chunks) ------------------------------------------------------------------ regards Samir From owner-zfs-devel@FreeBSD.ORG Thu Sep 29 13:21:11 2011 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 42BA6106566B for ; Thu, 29 Sep 2011 13:21:11 +0000 (UTC) (envelope-from pawel@dawidek.net) Received: from mail.dawidek.net (60.wheelsystems.com [83.12.187.60]) by mx1.freebsd.org (Postfix) with ESMTP id EE1668FC17 for ; Thu, 29 Sep 2011 13:21:10 +0000 (UTC) Received: from localhost (58.wheelsystems.com [83.12.187.58]) by mail.dawidek.net (Postfix) with ESMTPSA id A809F80B; Thu, 29 Sep 2011 15:21:08 +0200 (CEST) Date: Thu, 29 Sep 2011 15:20:34 +0200 From: Pawel Jakub Dawidek To: Samir Desai Message-ID: <20110929132034.GD1660@garage.freebsd.pl> References: MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="KlAEzMkarCnErv5Q" Content-Disposition: inline In-Reply-To: X-OS: FreeBSD 9.0-CURRENT amd64 User-Agent: Mutt/1.5.21 (2010-09-15) Cc: zfs-devel@freebsd.org Subject: Re: [BUG] zfs panic in dsl_dir_open_spa() because of NULL ptr X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 Sep 2011 13:21:11 -0000 --KlAEzMkarCnErv5Q Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Thu, Sep 29, 2011 at 05:53:43PM +0530, Samir Desai wrote: > hi >=20 > We are seeing ZFS v28 panic on FreeBSD, which seems to be caused by a race > condition between a thread loading spa and another trying to access it. T= he > two thread stacks trace these paths >=20 > 1. zfs_ioc_snapshot -> zfs_unmount_snap -> ...-> spa_load ->vdev_open ...= =2E. > -> vdev_geom_open ......-> biowait > 2. zfs_secpolicy_write_perms -> ...-> dmu_objset_snapshot -> ... > ->dsl_dir_open_spa -> _sx_slock >=20 > the two threads should be serialized by spa_namespace_lock, but apparently > the first thread, which is initializing spa, drops the lock in > vdev_geom_open(). This lets the second thread come in and see and > uninitialized spa. The spa->spa_dsl_pool pointer is NULL. The second thre= ad > tries to take lock in dsl_dir_open_spa() dp =3D spa_get_dsl(spa); > rw_enter(&dp->dp_config_rwlock, RW_READER); This panics because "dp" is N= ULL >=20 >=20 > Should we be dropping the spa_namespace_lock in vdev_geom_open() ? What > serializes the spa operations ? Could you try this patch: http://people.freebsd.org/~pjd/patches/zfsdev_state_lock.patch It eliminates dropping spa_namespace_lock in vdev_geom.c. --=20 Pawel Jakub Dawidek http://www.wheelsystems.com FreeBSD committer http://www.FreeBSD.org Am I Evil? Yes, I Am! http://yomoli.com --KlAEzMkarCnErv5Q Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (FreeBSD) iEYEARECAAYFAk6EcKIACgkQForvXbEpPzT1tgCePbbHaKQKab/Qg747yyblpiBL VkgAnA7z2Zi8rKrnM5V940GNO/22KrDm =RFsL -----END PGP SIGNATURE----- --KlAEzMkarCnErv5Q-- From owner-zfs-devel@FreeBSD.ORG Tue Oct 4 09:12:44 2011 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 141F31065670; Tue, 4 Oct 2011 09:12:44 +0000 (UTC) (envelope-from hellosamirdesai@gmail.com) Received: from mail-vw0-f54.google.com (mail-vw0-f54.google.com [209.85.212.54]) by mx1.freebsd.org (Postfix) with ESMTP id 756F78FC12; Tue, 4 Oct 2011 09:12:43 +0000 (UTC) Received: by vws11 with SMTP id 11so242251vws.13 for ; Tue, 04 Oct 2011 02:12:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=Kwgf5WwF4qHUM7KxwPcxy6XEqD+wyDjCi6szpR2WOSc=; b=Ro6rSB0uSfHYGh+DznDIN/7XNx9EqUYuItMPSz4ufFC1JHaefzrZvZW3IELzp42MdJ Wioju2TgVJG0wZSglx4BRA9Nubtpg8FiT8ODgjKssKT9CW1lOabjGOq5gmlJTAO6wXqY +BQlaqRfeLQ78onvpwIWtnpvSbAGXHcUuIY6I= MIME-Version: 1.0 Received: by 10.52.23.176 with SMTP id n16mr924699vdf.268.1317719561090; Tue, 04 Oct 2011 02:12:41 -0700 (PDT) Received: by 10.52.182.1 with HTTP; Tue, 4 Oct 2011 02:12:41 -0700 (PDT) In-Reply-To: <20110929132034.GD1660@garage.freebsd.pl> References: <20110929132034.GD1660@garage.freebsd.pl> Date: Tue, 4 Oct 2011 14:42:41 +0530 Message-ID: From: Samir Desai To: Pawel Jakub Dawidek Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: zfs-devel@freebsd.org Subject: Re: [BUG] zfs panic in dsl_dir_open_spa() because of NULL ptr X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Oct 2011 09:12:44 -0000 hi Pawel thanks a lot for this patch. It seems to be holding up so far for us. In case we still see issues, I will get back to you. regards Samir On Thu, Sep 29, 2011 at 6:50 PM, Pawel Jakub Dawidek wrote: > On Thu, Sep 29, 2011 at 05:53:43PM +0530, Samir Desai wrote: > > hi > > > > We are seeing ZFS v28 panic on FreeBSD, which seems to be caused by a > race > > condition between a thread loading spa and another trying to access it. > The > > two thread stacks trace these paths > > > > 1. zfs_ioc_snapshot -> zfs_unmount_snap -> ...-> spa_load ->vdev_open > ..... > > -> vdev_geom_open ......-> biowait > > 2. zfs_secpolicy_write_perms -> ...-> dmu_objset_snapshot -> ... > > ->dsl_dir_open_spa -> _sx_slock > > > > the two threads should be serialized by spa_namespace_lock, but > apparently > > the first thread, which is initializing spa, drops the lock in > > vdev_geom_open(). This lets the second thread come in and see and > > uninitialized spa. The spa->spa_dsl_pool pointer is NULL. The second > thread > > tries to take lock in dsl_dir_open_spa() dp = spa_get_dsl(spa); > > rw_enter(&dp->dp_config_rwlock, RW_READER); This panics because "dp" is > NULL > > > > > > Should we be dropping the spa_namespace_lock in vdev_geom_open() ? What > > serializes the spa operations ? > > Could you try this patch: > > http://people.freebsd.org/~pjd/patches/zfsdev_state_lock.patch > > It eliminates dropping spa_namespace_lock in vdev_geom.c. > > -- > Pawel Jakub Dawidek http://www.wheelsystems.com > FreeBSD committer http://www.FreeBSD.org > Am I Evil? Yes, I Am! http://yomoli.com > From owner-zfs-devel@FreeBSD.ORG Fri Oct 14 06:51:10 2011 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C9FD1106564A for ; Fri, 14 Oct 2011 06:51:10 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 218D58FC15 for ; Fri, 14 Oct 2011 06:51:09 +0000 (UTC) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id JAA16728; Fri, 14 Oct 2011 09:34:53 +0300 (EEST) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1REbM5-000PXI-Gc; Fri, 14 Oct 2011 09:34:53 +0300 Message-ID: <4E97D80B.90204@FreeBSD.org> Date: Fri, 14 Oct 2011 09:34:51 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:7.0.1) Gecko/20111002 Thunderbird/7.0.1 MIME-Version: 1.0 To: Dave Lin References: In-Reply-To: X-Enigmail-Version: undefined Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Mailman-Approved-At: Fri, 14 Oct 2011 16:07:30 +0000 Cc: "freebsd-fs@freebsd.org" Subject: Re: cannot run ztest (zfs test suite) without seeing core dump on 8.2 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 14 Oct 2011 06:51:10 -0000 on 14/10/2011 01:33 Dave Lin said the following: > Hello all, > > It seems that I cannot run ztest (zfs test suite) without seeing core dump on latest 8.2 release. Has anyone seen this before and is there way to resolve this issue? Thanks. > > Here's the output capture: > > dave-free82# ztest -V > 5 vdevs, 7 datasets, 23 threads, 300 seconds... > child died with signal 11 > > Here's the uname -a info: > > 8.2-RELEASE FreeBSD 8.2-RELEASE #0: Thu Feb 17 02:41:51 UTC 2011 root@mason.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC amd64 Please try the following changes: https://gitorious.org/~avg/freebsd/avgbsd/commit/5f0bbe6ff83f463f583358b76cfe2e179449091b https://gitorious.org/~avg/freebsd/avgbsd/commit/b430b23e6cd579c577f8ff1dae445a8ee2603ffa https://gitorious.org/~avg/freebsd/avgbsd/commit/96e8886589b0c6bb91e1019efb204e6aac87f4ef https://gitorious.org/~avg/freebsd/avgbsd/commit/5479bc325beb8fa85f50e50f3dc18069489a2119 -- Andriy Gapon From owner-zfs-devel@FreeBSD.ORG Fri Oct 14 22:21:44 2011 Return-Path: Delivered-To: zfs-devel@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4AA4F1065680 for ; Fri, 14 Oct 2011 22:21:44 +0000 (UTC) (envelope-from delphij@delphij.net) Received: from anubis.delphij.net (anubis.delphij.net [IPv6:2001:470:1:117::25]) by mx1.freebsd.org (Postfix) with ESMTP id 309A08FC0A for ; Fri, 14 Oct 2011 22:21:41 +0000 (UTC) Received: from delta.delphij.net (drawbridge.ixsystems.com [206.40.55.65]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by anubis.delphij.net (Postfix) with ESMTPSA id 7203B18CD9; Fri, 14 Oct 2011 15:21:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=delphij.net; s=anubis; t=1318630900; bh=FTBfjbiZoXAdZmSNeRSkYoOFcVrevY7EEAhr/B4bxa4=; h=Message-ID:Date:From:Reply-To:MIME-Version:To:Subject: Content-Type; b=ef1RyuKsJb6F/po5be6AGYDFd5sk6utkORxVLQRa0f0OcA9+XInF5v714vQSoiyoW 1/kkf0hrBsufJBorw4itMWMKSn1ByhM3Un5g3uYdGTOOaI6g2a4NWApWgO2slijCw9 qyQfrfFqSIq20Gxtn4MkHUeJWNoxu7sn9HlLo+VU= Message-ID: <4E98B5F3.7020103@delphij.net> Date: Fri, 14 Oct 2011 15:21:39 -0700 From: Xin LI Organization: The FreeBSD Project MIME-Version: 1.0 To: zfs-devel@FreeBSD.ORG OpenPGP: id=3FCA37C1; url=http://www.delphij.net/delphij.asc Content-Type: multipart/mixed; boundary="------------020607010000090202070304" Cc: Subject: [PATCH FOR REVIEW] Extended attribute X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: d@delphij.net List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 14 Oct 2011 22:21:44 -0000 This is a multi-part message in MIME format. --------------020607010000090202070304 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 Hi, I think we hit a bug in sa_find_sizes: - hdrsize grows by 4 bytes each time; - On return, hdrsize is rounded up to nearest 8 byte boundary; - On the other hand, when testing for whether spill is needed, the code uses sum of (*total + hdrsize) and rounds that number up to nearest 8 byte boundary. This causes a problem when *total==306 and hdrsize==12. We have: (306 + 12) = 318 ROUNDUP(318, 8) = 320 On return, we have: total = 306 hdrsize = 16 And spill is unset. This would lead to a panic. Comments? Cheers, - -- Xin LI https://www.delphij.net/ FreeBSD - The Power to Serve! Live free or die -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.18 (FreeBSD) iQEcBAEBCAAGBQJOmLXyAAoJEATO+BI/yjfBlikH+wQCt9BFDyFtH7CAaCui8W4b MnGo1Ocpbgzx0UBVcJkb5pglqscEihDMfQXLQHzbf+v0Bu3nfH2MudNsdr4/Ig2X XUD6cBD1cAN1x5jWzBIKrS7nigxICfd+cOsZ2J1vRj0Qu6TJ9t3kAETuZsU6l4Ee 3KfvoTu48E/2CzeCD4aAbqJH6bbIJ26dKRQ+KTp+FTLLk8goqDCQ7RgbEfO1QYZm vV+Ee+olvHDddC0Oxe08/X8seWaZg/N2ZZNT1Umo8VeiRvbKlUEs1hj8KN2ddDtD 7ufTbEuSgQVMRvZT/9aCpEl4/g7o5yIKUFWyJuXViSxT/n9hk1a1rdRaWun6Ges= =nprr -----END PGP SIGNATURE----- --------------020607010000090202070304 Content-Type: text/plain; name="sa.diff" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="sa.diff" SW5kZXg6IHN5cy9jZGRsL2NvbnRyaWIvb3BlbnNvbGFyaXMvdXRzL2NvbW1vbi9mcy96ZnMv c2EuYwo9PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09Ci0tLSBzeXMvY2RkbC9jb250cmliL29wZW5zb2xhcmlzL3V0 cy9jb21tb24vZnMvemZzL3NhLmMJKHJldmlzaW9uIDIyNjM2NikKKysrIHN5cy9jZGRsL2Nv bnRyaWIvb3BlbnNvbGFyaXMvdXRzL2NvbW1vbi9mcy96ZnMvc2EuYwkod29ya2luZyBjb3B5 KQpAQCAtNjA1LDE0ICs2MDUsMTQgQEAgc2FfZmluZF9zaXplcyhzYV9vc190ICpzYSwgc2Ff YnVsa19hdHRyX3QgKmF0dHJfZGUKIAkJICogYW5kIHNwaWxsIGJ1ZmZlci4KIAkJICovCiAJ CWlmIChidWZ0eXBlID09IFNBX0JPTlVTICYmICppbmRleCA9PSAtMSAmJgotCQkgICAgUDJS T1VORFVQKCp0b3RhbCArIGhkcnNpemUsIDgpID4KKwkJICAgICgqdG90YWwgKyBQMlJPVU5E VVAoaGRyc2l6ZSwgOCkpID4KIAkJICAgIChmdWxsX3NwYWNlIC0gc2l6ZW9mIChibGtwdHJf dCkpKSB7CiAJCQkqaW5kZXggPSBpOwogCQkJZG9uZSA9IEJfVFJVRTsKIAkJfQogCiBuZXh0 OgotCQlpZiAoUDJST1VORFVQKCp0b3RhbCArIGhkcnNpemUsIDgpID4gZnVsbF9zcGFjZSAm JgorCQlpZiAoKCp0b3RhbCArIFAyUk9VTkRVUChoZHJzaXplLCA4KSkgPiBmdWxsX3NwYWNl ICYmCiAJCSAgICBidWZ0eXBlID09IFNBX0JPTlVTKQogCQkJKndpbGxfc3BpbGwgPSBCX1RS VUU7CiAJfQo= --------------020607010000090202070304-- From owner-zfs-devel@FreeBSD.ORG Mon Oct 17 22:14:56 2011 Return-Path: Delivered-To: zfs-devel@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C8F1C106566B for ; Mon, 17 Oct 2011 22:14:56 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from mail.vx.sk (mail.vx.sk [IPv6:2a01:4f8:100:1043::3]) by mx1.freebsd.org (Postfix) with ESMTP id 8AD218FC17 for ; Mon, 17 Oct 2011 22:14:56 +0000 (UTC) Received: from core.vx.sk (localhost [127.0.0.1]) by mail.vx.sk (Postfix) with ESMTP id 45FCA147764; Tue, 18 Oct 2011 00:14:55 +0200 (CEST) X-Virus-Scanned: amavisd-new at mail.vx.sk Received: from mail.vx.sk ([127.0.0.1]) by core.vx.sk (mail.vx.sk [127.0.0.1]) (amavisd-new, port 10024) with LMTP id nt6sA8FNJhst; Tue, 18 Oct 2011 00:14:53 +0200 (CEST) Received: from [10.9.8.1] (188-167-78-15.dynamic.chello.sk [188.167.78.15]) by mail.vx.sk (Postfix) with ESMTPSA id 0C515147759; Tue, 18 Oct 2011 00:14:52 +0200 (CEST) Message-ID: <4E9CA8DB.4070404@FreeBSD.org> Date: Tue, 18 Oct 2011 00:14:51 +0200 From: Martin Matuska User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:7.0.1) Gecko/20110929 Thunderbird/7.0.1 MIME-Version: 1.0 To: d@delphij.net References: <4E98B5F3.7020103@delphij.net> In-Reply-To: <4E98B5F3.7020103@delphij.net> X-Enigmail-Version: 1.3.2 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: zfs-devel@FreeBSD.ORG Subject: Re: [PATCH FOR REVIEW] Extended attribute X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Oct 2011 22:14:56 -0000 On 15. 10. 2011 0:21, Xin LI wrote: > Hi, > > I think we hit a bug in sa_find_sizes: > > - hdrsize grows by 4 bytes each time; > - On return, hdrsize is rounded up to nearest 8 byte boundary; > - On the other hand, when testing for whether spill is needed, the > code uses sum of (*total + hdrsize) and rounds that number up to > nearest 8 byte boundary. > > This causes a problem when *total==306 and hdrsize==12. We have: > > (306 + 12) = 318 > ROUNDUP(318, 8) = 320 > > On return, we have: > > total = 306 > hdrsize = 16 > > And spill is unset. This would lead to a panic. > > Comments? > > Cheers, In my opinion your computation is correct, the bugfix should go to HEAD and be reported to OpenSolaris, too. If you want I can report it upstream. -- Martin Matuska FreeBSD committer http://blog.vx.sk From owner-zfs-devel@FreeBSD.ORG Mon Oct 17 22:44:19 2011 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 76DDD106564A; Mon, 17 Oct 2011 22:44:19 +0000 (UTC) (envelope-from delphij@delphij.net) Received: from anubis.delphij.net (anubis.delphij.net [IPv6:2001:470:1:117::25]) by mx1.freebsd.org (Postfix) with ESMTP id 5D3468FC0A; Mon, 17 Oct 2011 22:44:19 +0000 (UTC) Received: from delta.delphij.net (drawbridge.ixsystems.com [206.40.55.65]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by anubis.delphij.net (Postfix) with ESMTPSA id 148CE182E3; Mon, 17 Oct 2011 15:44:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=delphij.net; s=anubis; t=1318891459; bh=/pV+JfWV2oGh5e+obPFqyhYPI1tpDj8mtJt6iWrA4IA=; h=Message-ID:Date:From:Reply-To:MIME-Version:To:CC:Subject: References:In-Reply-To:Content-Type:Content-Transfer-Encoding; b=Yyt3ElTPcVlpNJM2FLA/KArqqs6abubvokknNTGZ7YjeI3vLz4rTdDaiHc48MEU9B 9XpltUswhmODZuvx+XPGjHzdBQeTGT50xbiH26RUevWswIMR2XV8UkShl85U9p6I9d S3kOnr5Pf34Gk1xftUXqSs0lHvOZsFxX4hpLfuJc= Message-ID: <4E9CAFC2.9020200@delphij.net> Date: Mon, 17 Oct 2011 15:44:18 -0700 From: Xin LI Organization: The FreeBSD Project MIME-Version: 1.0 To: Martin Matuska References: <4E82FA22.5020406@FreeBSD.org> In-Reply-To: <4E82FA22.5020406@FreeBSD.org> OpenPGP: id=3FCA37C1; url=http://www.delphij.net/delphij.asc Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: zfs-devel@FreeBSD.org, Pawel Jakub Dawidek Subject: Re: [PATCH] Illumos changeset #13469 (dmu_tx.c) X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: d@delphij.net List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Oct 2011 22:44:19 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 On 09/28/11 03:42, Martin Matuska wrote: > Illumos has now provided a fix for the Issue #1475 regarding > invalid spill blkptr: https://www.illumos.org/issues/1475 > > changeset: 13469:b8e89e5c4167 tag: tip user: > Albert Lee date: Sun Sep 25 03:07:35 > 2011 -0400 summary: 1475 zfs spill block hold can access > invalid spill blkptr > https://github.com/illumos/illumos-gate/commit/9dccfd2a04cd1645e2616b7307b3a88041aba991 > > A partial fix by pjd was already in our code: > http://p4db.freebsd.org/changeView.cgi?CH=185940 > http://p4db.freebsd.org/changeView.cgi?CH=185942 > > I suggest importing the full fix to match the Illumos version. > Please review and/or comment the attached patch. I think these changes are safe because they did not change the logic internals and should be incorporated since they reduced diff against upstream. Cheers, - -- Xin LI https://www.delphij.net/ FreeBSD - The Power to Serve! Live free or die -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.18 (FreeBSD) iQEcBAEBCAAGBQJOnK/BAAoJEATO+BI/yjfBnFsH+gMc+Hj5867phimJ6eJNACHQ xo5TK8eTVRs5dpOkpiqAtZuSEFQ5y8he1DYjTPrQOe3lPWJDoGJwAsHZtgQsgtAT ls8HmScri1RYkAlpEtCxvGKImFfB84agwWf27coEH54oDBQ4DVhPXhUlZ6xzgU1u w44lydOUjIWyO1RVydOckDnr7i6CjNm6lSHXBVVM40He0ScDdcG6O5MQChVuJrSD ibBJmXqIpWvNF97n2uvT6wSFV7Kk+npCdKIysG1fVqMEFiIQixlvXBk4DmNKcwuj eWPeKuL32wpDk04bJp0ca1heNavQwojkQn2IYdGFrdAYFFXT7kPOrb01ju+YTLg= =f4Z7 -----END PGP SIGNATURE----- From owner-zfs-devel@FreeBSD.ORG Sun Oct 30 09:16:38 2011 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9ED8F106566B for ; Sun, 30 Oct 2011 09:16:38 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from mail.vx.sk (unknown [IPv6:2a01:4f8:150:6101::4]) by mx1.freebsd.org (Postfix) with ESMTP id 24FD18FC0C for ; Sun, 30 Oct 2011 09:16:38 +0000 (UTC) Received: from core2.vx.sk (localhost [127.0.0.2]) by mail.vx.sk (Postfix) with ESMTP id 83FA5BD77 for ; Sun, 30 Oct 2011 10:16:37 +0100 (CET) X-Virus-Scanned: amavisd-new at mail.vx.sk Received: from mail.vx.sk by core2.vx.sk (amavisd-new, unix socket) with LMTP id 0RFvJ5qKAces for ; Sun, 30 Oct 2011 10:16:35 +0100 (CET) Received: from [10.9.8.1] (188-167-78-15.dynamic.chello.sk [188.167.78.15]) by mail.vx.sk (Postfix) with ESMTPSA id D6877BD61 for ; Sun, 30 Oct 2011 10:16:34 +0100 (CET) Message-ID: <4EAD15F2.7070807@FreeBSD.org> Date: Sun, 30 Oct 2011 10:16:34 +0100 From: Martin Matuska User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:7.0.1) Gecko/20110929 Thunderbird/7.0.1 MIME-Version: 1.0 To: zfs-devel@FreeBSD.org X-Enigmail-Version: 1.3.2 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: Subject: Preview: upcoming Illumos ZFS changes in code review phase X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 30 Oct 2011 09:16:38 -0000 Hi everyone, ZFS development in Illumos seems to be going on. I am in touch with the developers and closely following the activity on the mailing lists. Here is a preview of the patches and enhancements that are in the code review phase: 1.) show zfs ioctl args in truss (might be of interest for us) https://www.illumos.org/issues/1688 WEBREV: http://yalms.org/cr/truss-zfs/ 2.) 1644 add ZFS "clones" property https://www.illumos.org/issues/1644 1645 add ZFS "written" and "written@..." properties https://www.illumos.org/issues/1645 1646 "zfs send" should estimate size of stream https://www.illumos.org/issues/1646 1647 "zfs destroy" should determine space reclaimed by destroying multiple snapshots https://www.illumos.org/issues/1647 WEBREV: http://code.delphix.com/webrev-props/ 3.) persistent 'comment' field for a zpool https://www.illumos.org/issues/1693 WEBREV: http://www.kebe.com/~danmcd/webrevs/hackathon/zfs-comment/ 4.) "zpool reguid" command (change zpool's guid number) WEBREV: http://dev1.illumos.org/~gdamore/reguid/ Some of the patches have already received lots of comments so the final code added to Illumos is going to be different. -- Martin Matuska FreeBSD committer http://blog.vx.sk From owner-zfs-devel@FreeBSD.ORG Sat Nov 5 18:40:38 2011 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5B1CC106566B; Sat, 5 Nov 2011 18:40:38 +0000 (UTC) (envelope-from yanegomi@gmail.com) Received: from mail-iy0-f182.google.com (mail-iy0-f182.google.com [209.85.210.182]) by mx1.freebsd.org (Postfix) with ESMTP id 153F78FC19; Sat, 5 Nov 2011 18:40:38 +0000 (UTC) Received: by iabz21 with SMTP id z21so6140959iab.13 for ; Sat, 05 Nov 2011 11:40:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=from:content-type:subject:date:message-id:cc:to:mime-version :x-mailer; bh=VyBC9/xc+RK58PcD+CYs+yatwGiKrv1SrvBg2Exnw94=; b=ostZgN57vieakv/T9ogETgo1FN7oXavAN3t6YMroBasjIYbzHLHqWYpjIXmm4zHvlN 7pG9S0jCSY5zMJMlkLXQVxhQvxKWuPPyXzMFgm+rkgqLgLhu69lTr6FN4N6ve9lGwY1x 5/oUt6cXMvahRl7Hi0d/cSQib/bmlKgcW2Apk= Received: by 10.42.147.72 with SMTP id m8mr28044670icv.56.1320516873555; Sat, 05 Nov 2011 11:14:33 -0700 (PDT) Received: from [192.168.20.5] (c-24-6-49-154.hsd1.ca.comcast.net. [24.6.49.154]) by mx.google.com with ESMTPS id d8sm8125683pbb.6.2011.11.05.11.14.31 (version=TLSv1/SSLv3 cipher=OTHER); Sat, 05 Nov 2011 11:14:32 -0700 (PDT) From: Garrett Cooper Content-Type: multipart/mixed; boundary=Apple-Mail-3-759769076 Date: Sat, 5 Nov 2011 11:14:28 -0700 Message-Id: To: zfs-devel@FreeBSD.org Mime-Version: 1.0 (Apple Message framework v1084) X-Mailer: Apple Mail (2.1084) Cc: rdivacky@freebsd.org, Dimitry Andric Subject: [PATCH] Fix some coding issues in zpool(8) uncovered by clang X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 05 Nov 2011 18:40:38 -0000 --Apple-Mail-3-759769076 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii Long story short, I was compiling rescue with clang this past = week with -g and -O0, which unrooted a few issues with zpool (more than = the standard optimization, BTW..). The following patch fixes some = formatting string and assign/test issues in zpool(8), and shouldn't = introduce any noticeable functional changes. If someone could review and possibly commit it, that would be = great. Thanks, -Garrett --Apple-Mail-3-759769076 Content-Disposition: attachment; filename=zpool-fix-clang-compilation-warnings.patch Content-Type: application/octet-stream; name="zpool-fix-clang-compilation-warnings.patch" Content-Transfer-Encoding: 7bit Index: cddl/contrib/opensolaris/cmd/zpool/zpool_iter.c =================================================================== --- cddl/contrib/opensolaris/cmd/zpool/zpool_iter.c (revision 227072) +++ cddl/contrib/opensolaris/cmd/zpool/zpool_iter.c (working copy) @@ -132,7 +132,7 @@ for (i = 0; i < argc; i++) { zpool_handle_t *zhp; - if (zhp = zpool_open_canfail(g_zfs, argv[i])) { + if ((zhp = zpool_open_canfail(g_zfs, argv[i]))) { if (add_pool(zhp, zlp) != 0) *err = B_TRUE; } else { Index: cddl/contrib/opensolaris/cmd/zpool/zpool_main.c =================================================================== --- cddl/contrib/opensolaris/cmd/zpool/zpool_main.c (revision 227072) +++ cddl/contrib/opensolaris/cmd/zpool/zpool_main.c (working copy) @@ -2566,9 +2566,9 @@ if (pl->pl_next == NULL && !right_justify) (void) printf("%s", header); else if (right_justify) - (void) printf("%*s", pl->pl_width, header); + (void) printf("%*s", (int)pl->pl_width, header); else - (void) printf("%-*s", pl->pl_width, header); + (void) printf("%-*s", (int)pl->pl_width, header); } (void) printf("\n"); @@ -4066,7 +4066,7 @@ if (cur_version > cbp->cb_version) { (void) printf(gettext("Pool '%s' is already formatted " "using more current version '%llu'.\n"), - zpool_get_name(zhp), cur_version); + zpool_get_name(zhp), (u_longlong_t)cur_version); return (0); } if (cur_version == cbp->cb_version) { @@ -4321,8 +4321,9 @@ continue; (void) snprintf(internalstr, sizeof (internalstr), - "[internal %s txg:%lld] %s", - zfs_history_event_names[ievent], txg, + "[internal %s txg:%llu] %s", + zfs_history_event_names[ievent], + (u_longlong_t)txg, pathstr); cmdstr = internalstr; } --Apple-Mail-3-759769076-- From owner-zfs-devel@FreeBSD.ORG Fri Nov 18 13:04:53 2011 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D71101065672; Fri, 18 Nov 2011 13:04:53 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from mail.vx.sk (mail.vx.sk [IPv6:2a01:4f8:150:6101::4]) by mx1.freebsd.org (Postfix) with ESMTP id 99C478FC1A; Fri, 18 Nov 2011 13:04:53 +0000 (UTC) Received: from core.vx.sk (localhost [127.0.0.2]) by mail.vx.sk (Postfix) with ESMTP id 0B4FC13C69; Fri, 18 Nov 2011 14:04:53 +0100 (CET) X-Virus-Scanned: amavisd-new at mail.vx.sk Received: from mail.vx.sk by core.vx.sk (amavisd-new, unix socket) with LMTP id m0lXdgdl6Z6R; Fri, 18 Nov 2011 14:04:48 +0100 (CET) Received: from [10.9.8.1] (188-167-78-15.dynamic.chello.sk [188.167.78.15]) by mail.vx.sk (Postfix) with ESMTPSA id EFD0F13C57; Fri, 18 Nov 2011 14:04:47 +0100 (CET) Message-ID: <4EC657EF.2070602@FreeBSD.org> Date: Fri, 18 Nov 2011 14:04:47 +0100 From: Martin Matuska User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:8.0) Gecko/20111105 Thunderbird/8.0 MIME-Version: 1.0 To: Garrett Cooper References: In-Reply-To: X-Enigmail-Version: 1.3.3 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: zfs-devel@FreeBSD.org, Dimitry Andric , rdivacky@freebsd.org Subject: Re: [PATCH] Fix some coding issues in zpool(8) uncovered by clang X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 18 Nov 2011 13:04:53 -0000 On 5.11.2011 19:14, Garrett Cooper wrote: > Long story short, I was compiling rescue with clang this past week with -g and -O0, which unrooted a few issues with zpool (more than the standard optimization, BTW..). The following patch fixes some formatting string and assign/test issues in zpool(8), and shouldn't introduce any noticeable functional changes. > If someone could review and possibly commit it, that would be great. > Thanks, > -Garrett Looks good. -- Martin Matuska FreeBSD committer http://blog.vx.sk From owner-zfs-devel@FreeBSD.ORG Fri Nov 18 13:17:59 2011 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CEA1B106566B; Fri, 18 Nov 2011 13:17:59 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from mail.vx.sk (mail.vx.sk [IPv6:2a01:4f8:150:6101::4]) by mx1.freebsd.org (Postfix) with ESMTP id 945088FC12; Fri, 18 Nov 2011 13:17:59 +0000 (UTC) Received: from core.vx.sk (localhost [127.0.0.2]) by mail.vx.sk (Postfix) with ESMTP id 0836E103A9; Fri, 18 Nov 2011 14:17:59 +0100 (CET) X-Virus-Scanned: amavisd-new at mail.vx.sk Received: from mail.vx.sk by core.vx.sk (amavisd-new, unix socket) with LMTP id mG2xcVfDXOtC; Fri, 18 Nov 2011 14:17:54 +0100 (CET) Received: from [10.9.8.1] (188-167-78-15.dynamic.chello.sk [188.167.78.15]) by mail.vx.sk (Postfix) with ESMTPSA id E508010394; Fri, 18 Nov 2011 14:17:53 +0100 (CET) Message-ID: <4EC65B01.4000906@FreeBSD.org> Date: Fri, 18 Nov 2011 14:17:53 +0100 From: Martin Matuska User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:8.0) Gecko/20111105 Thunderbird/8.0 MIME-Version: 1.0 To: zfs-devel@FreeBSD.org X-Enigmail-Version: 1.3.3 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Cc: Pawel Jakub Dawidek Subject: [PATCH] New ZFS features from Illumos (13514, 13524, 13525) X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 18 Nov 2011 13:17:59 -0000 I have ported the recent changes from Illumos for our tree. The patch can be downloaded here: http://people.freebsd.org/~mm/patches/zfs/head-illumos-13514-13524-13525.patch What is it all about? It is three revisions covering 7 features. There are no changes to the on-disk format. There three new IOCTL's and one has been changed. I have written some compatibility code for the changed IOCTL (ZFS_IOC_DESTROY_SNAPS) so that it detects if the given argument is a nvlist. This way the recursive destroy of ZFS snapshots works with the older binaries on the new kernel. The other way (new binaries, old kernel) is little more problematic. If using the new flags to zfs send there may be (some) crambled output and it is not possible to destroy snapshots. The problem is I cannot properly detect (in another way than osversion) what utiliy versions are we running. But if this isn't such a big problem (new binary, old kernel) we could go with this version. Here is a list of the features, for detailed descriptions see links underneath: 1644 add ZFS "clones" property https://www.illumos.org/issues/1644 1645 add ZFS "written" and "written@..." properties https://www.illumos.org/issues/1645 1646 "zfs send" should estimate size of stream https://www.illumos.org/issues/1646 1647 "zfs destroy" should determine space reclaimed by destroying multiple snapshots https://www.illumos.org/issues/1647 1693 persistent 'comment' field for a zpool https://www.illumos.org/issues/1693 1708 adjust size of zpool history data https://www.illumos.org/issues/1708 1748 desire support for reguid in zfs https://www.illumos.org/issues/1748 Illumos changesets:: 13514: http://hg.openindiana.org/upstream/illumos/illumos-gate/rev/417c34452f03 (1748) 13524: http://hg.openindiana.org/upstream/illumos/illumos-gate/rev/417c34452f03 (1644, 1645, 1646, 1647, 1708) 13525: http://hg.openindiana.org/upstream/illumos/illumos-gate/rev/7059b67f1bc2 (1693) Please review and/or test my patch, comments and suggestions are welcome. http://people.freebsd.org/~mm/patches/zfs/head-illumos-13514-13524-13525.patch -- Martin Matuska FreeBSD committer http://blog.vx.sk From owner-zfs-devel@FreeBSD.ORG Wed Dec 7 18:34:05 2011 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A88411065672 for ; Wed, 7 Dec 2011 18:34:05 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id BBD458FC15 for ; Wed, 7 Dec 2011 18:34:04 +0000 (UTC) Received: from odyssey.starpoint.kiev.ua (alpha-e.starpoint.kiev.ua [212.40.38.101]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id UAA01893 for ; Wed, 07 Dec 2011 20:14:37 +0200 (EET) (envelope-from avg@FreeBSD.org) Message-ID: <4EDFAD0D.2080100@FreeBSD.org> Date: Wed, 07 Dec 2011 20:14:37 +0200 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:8.0) Gecko/20111109 Thunderbird/8.0 MIME-Version: 1.0 To: zfs-devel@FreeBSD.org References: <4EDFABD7.20301@FreeBSD.org> In-Reply-To: <4EDFABD7.20301@FreeBSD.org> X-Enigmail-Version: undefined X-Forwarded-Message-Id: <4EDFABD7.20301@FreeBSD.org> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: Fwd: Re: zfs-related(?) panic in cache_enter: wrong vnode type X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 07 Dec 2011 18:34:05 -0000 Trying to find ZFS experts here too :-) -------- Original Message -------- Message-ID: <4EDFABD7.20301@FreeBSD.org> Date: Wed, 07 Dec 2011 20:09:27 +0200 From: Andriy Gapon To: freebsd-current@FreeBSD.ORG, freebsd-fs@FreeBSD.ORG Subject: Re: zfs-related(?) panic in cache_enter: wrong vnode type References: <4EDF995B.9000407@FreeBSD.org> <4EDFA354.60909@FreeBSD.org> In-Reply-To: <4EDFA354.60909@FreeBSD.org> on 07/12/2011 19:33 Andriy Gapon said the following: > > A detail that may or may not be useful. > It seems that the panic happened when tried to resume a vim process. The process > was suspended, its current directory and a file being edited/viewed may have been > already removed. And another data point: (kgdb) p ((znode_t *)dvp->v_data)->z_unlinked Perhaps lookup in unlinked directories should just fail? I am not sufficiently familiar with VFS, so just guessing. > on 07/12/2011 18:50 Andriy Gapon said the following: >> >> (kgdb) bt >> #0 doadump (textdump=1) at pcpu.h:224 >> #1 0xffffffff804f6d3b in kern_reboot (howto=260) at >> /usr/src/sys/kern/kern_shutdown.c:447 >> #2 0xffffffff804f63e9 in panic (fmt=0x104
) at >> /usr/src/sys/kern/kern_shutdown.c:635 >> #3 0xffffffff80585f46 in cache_enter (dvp=0xfffffe003d4763c0, >> vp=0xfffffe0142517000, cnp=0xffffff82393b3708) at /usr/src/sys/kern/vfs_cache.c:726 >> #4 0xffffffff81a90900 in zfs_lookup (dvp=0xfffffe003d4763c0, >> nm=0xffffff82393b3140 "..", vpp=0xffffff82393b36e0, cnp=0xffffff82393b3708, >> nameiop=0, cr=0xfffffe0042e88100, td=0xfffffe000fdfa480, >> flags=0) at >> /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c:1470 >> #5 0xffffffff81a91570 in zfs_freebsd_lookup (ap=0xffffff82393b32c0) at >> /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c:5858 >> #6 0xffffffff8073f054 in VOP_CACHEDLOOKUP_APV (vop=0xffffffff81b05a20, >> a=0xffffff82393b32c0) at vnode_if.c:187 >> #7 0xffffffff80586bf4 in vfs_cache_lookup (ap=Variable "ap" is not available. >> ) at vnode_if.h:80 >> #8 0xffffffff80740a5c in VOP_LOOKUP_APV (vop=0xffffffff81b05a20, >> a=0xffffff82393b33a0) at vnode_if.c:123 >> #9 0xffffffff8058e42c in lookup (ndp=0xffffff82393b36a0) at vnode_if.h:54 >> #10 0xffffffff8058f17e in namei (ndp=0xffffff82393b36a0) at >> /usr/src/sys/kern/vfs_lookup.c:312 >> #11 0xffffffff805a890d in vn_open_cred (ndp=0xffffff82393b36a0, >> flagp=0xffffff82393b3918, cmode=0, vn_open_flags=Variable "vn_open_flags" is not >> available. >> ) at /usr/src/sys/kern/vfs_vnops.c:195 >> #12 0xffffffff80589e7e in vop_stdvptocnp (ap=Variable "ap" is not available. >> ) at /usr/src/sys/kern/vfs_default.c:774 >> #13 0xffffffff8073b012 in VOP_VPTOCNP_APV (vop=0xffffffff80a99140, >> a=0xffffff82393b39b0) at vnode_if.c:3479 >> #14 0xffffffff80584665 in vn_vptocnp_locked (vp=0xffffff82393b3a50, >> cred=0xfffffe0042e88100, >> buf=0xfffffe000ca06000 >> "������������������������������������������������������������������������������������������������������������������������������������������������������"..., >> buflen=0xffffff82393b3a4c) at vnode_if.h:1564 >> #15 0xffffffff80584bab in vn_fullpath1 (td=0xfffffe000fdfa480, >> vp=0xfffffe003d4763c0, rdir=0xfffffe000cd4d000, >> buf=0xfffffe000ca06000 >> "������������������������������������������������������������������������������������������������������������������������������������������������������"..., >> retbuf=0xffffff82393b3ab0, buflen=1023) at /usr/src/sys/kern/vfs_cache.c:1218 >> #16 0xffffffff8058526a in kern___getcwd (td=0xfffffe000fdfa480, buf=0x80880a000 >>
, bufseg=UIO_USERSPACE, buflen=1024) at >> /usr/src/sys/kern/vfs_cache.c:960 >> #17 0xffffffff805853f4 in sys___getcwd (td=Variable "td" is not available. >> ) at /usr/src/sys/kern/vfs_cache.c:934 >> #18 0xffffffff806d2069 in amd64_syscall (td=0xfffffe000fdfa480, traced=0) at >> subr_syscall.c:131 >> #19 0xffffffff806bb4e7 in Xfast_syscall () at /usr/src/sys/amd64/amd64/exception.S:387 >> #20 0x00000008031adb2c in ?? () >> Previous frame inner to this frame (corrupt stack?) >> (kgdb) fr 3 >> #3 0xffffffff80585f46 in cache_enter (dvp=0xfffffe003d4763c0, >> vp=0xfffffe0142517000, cnp=0xffffff82393b3708) at /usr/src/sys/kern/vfs_cache.c:726 >> 726 KASSERT(vp == NULL || vp->v_type == VDIR, >> (kgdb) list >> 721 if (dvp->v_cache_dd != NULL) { >> 722 CACHE_WUNLOCK(); >> 723 cache_free(ncp); >> 724 return; >> 725 } >> 726 KASSERT(vp == NULL || vp->v_type == VDIR, >> 727 ("wrong vnode type %p", vp)); >> 728 dvp->v_cache_dd = ncp; >> 729 } >> 730 >> (kgdb) p *vp >> $1 = {v_type = VREG, v_tag = 0xffffffff81afe449 "zfs", v_op = 0xffffffff81b05a20, >> v_data = 0xfffffe020d9a8320, v_mount = 0xfffffe001a283600, v_nmntvnodes = >> {tqe_next = 0xfffffe00347c6d20, >> tqe_prev = 0xfffffe013b0575c8}, v_un = {vu_mount = 0x0, vu_socket = 0x0, >> vu_cdev = 0x0, vu_fifoinfo = 0x0}, v_hashlist = {le_next = 0x0, le_prev = 0x0}, >> v_hash = 0, v_cache_src = { >> lh_first = 0x0}, v_cache_dst = {tqh_first = 0xfffffe003dfcf690, tqh_last = >> 0xfffffe003dfcf6b0}, v_cache_dd = 0x0, v_cstart = 0, v_lasta = 0, v_lastw = 0, >> v_clen = 0, v_lock = {lock_object = { >> lo_name = 0xffffffff81afe449 "zfs", lo_flags = 91947008, lo_data = 0, >> lo_witness = 0xffffff800066e380}, lk_lock = 18446741874952610944, lk_exslpfail = >> 0, lk_timo = 51, lk_pri = 96}, >> v_interlock = {lock_object = {lo_name = 0xffffffff807e610a "vnode interlock", >> lo_flags = 16973824, lo_data = 0, lo_witness = 0xffffff8000665600}, mtx_lock = 4}, >> v_vnlock = 0xfffffe0142517098, >> v_holdcnt = 2, v_usecount = 1, v_iflag = 0, v_vflag = 0, v_writecount = 0, >> v_freelist = {tqe_next = 0xfffffe00347c6d20, tqe_prev = 0xfffffe02110d3e30}, >> v_bufobj = {bo_mtx = {lock_object = { >> lo_name = 0xffffffff807efdfa "bufobj interlock", lo_flags = 16973824, >> lo_data = 0, lo_witness = 0xffffff800066c300}, mtx_lock = 4}, bo_clean = {bv_hd = >> {tqh_first = 0x0, >> tqh_last = 0xfffffe0142517140}, bv_root = 0x0, bv_cnt = 0}, bo_dirty = >> {bv_hd = {tqh_first = 0x0, tqh_last = 0xfffffe0142517160}, bv_root = 0x0, bv_cnt = >> 0}, bo_numoutput = 0, bo_flag = 0, >> bo_ops = 0xffffffff80a95ec0, bo_bsize = 131072, bo_object = >> 0xfffffe01fccb5620, bo_synclist = {le_next = 0x0, le_prev = 0x0}, bo_private = >> 0xfffffe0142517000, __bo_vnode = 0xfffffe0142517000}, >> v_pollinfo = 0x0, v_label = 0x0, v_lockf = 0x0} >> (kgdb) >> > > -- Andriy Gapon From owner-zfs-devel@FreeBSD.ORG Sat Dec 17 13:44:45 2011 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2AD4B106566B; Sat, 17 Dec 2011 13:44:45 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 2A94D8FC12; Sat, 17 Dec 2011 13:44:43 +0000 (UTC) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id PAA04645; Sat, 17 Dec 2011 15:44:42 +0200 (EET) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1RbuZ8-0009ct-II; Sat, 17 Dec 2011 15:44:42 +0200 Message-ID: <4EEC9CC9.9030001@FreeBSD.org> Date: Sat, 17 Dec 2011 15:44:41 +0200 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:8.0) Gecko/20111206 Thunderbird/8.0 MIME-Version: 1.0 To: zfs-devel@FreeBSD.org, Pawel Jakub Dawidek X-Enigmail-Version: undefined Content-Type: text/plain; charset=X-VIET-VPS Content-Transfer-Encoding: 7bit Cc: Subject: vcmn_err: incorrect panic message X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 17 Dec 2011 13:44:45 -0000 Guys, what do you think about the following change or a variation of it? With the current code, in the unfortunate event of panic-ing via vcmn_err() a panic message would actually be a format string (placeholders are not substituted). commit f6cd1dc812b4cfdb3a123052ce0cb49a5afa941d Author: Andriy Gapon Date: Thu Dec 15 00:04:31 2011 +0200 opensolaris compat: fix vcmn_err so that panic produces a proper message ... instead of just a verbatim format string diff --git a/sys/cddl/compat/opensolaris/kern/opensolaris_cmn_err.c b/sys/cddl/compat/opensolaris/kern/opensolaris_cmn_err.c index 12e1854..abde30d 100644 --- a/sys/cddl/compat/opensolaris/kern/opensolaris_cmn_err.c +++ b/sys/cddl/compat/opensolaris/kern/opensolaris_cmn_err.c @@ -28,29 +28,35 @@ void vcmn_err(int ce, const char *fmt, va_list adx) { char buf[256]; + const char *prefix; + prefix = NULL; /* silence unwitty compilers */ switch (ce) { case CE_CONT: - snprintf(buf, sizeof(buf), "Solaris(cont): %s\n", fmt); + prefix = "Solaris(cont): "; break; case CE_NOTE: - snprintf(buf, sizeof(buf), "Solaris: NOTICE: %s\n", fmt); + prefix = "Solaris: NOTICE: "; break; case CE_WARN: - snprintf(buf, sizeof(buf), "Solaris: WARNING: %s\n", fmt); + prefix = "Solaris: WARNING: "; break; case CE_PANIC: - snprintf(buf, sizeof(buf), "Solaris(panic): %s\n", fmt); + prefix = "Solaris(panic): "; break; case CE_IGNORE: break; default: panic("Solaris: unknown severity level"); } - if (ce == CE_PANIC) - panic("%s", buf); - if (ce != CE_IGNORE) - vprintf(buf, adx); + if (ce == CE_PANIC) { + vsnprintf(buf, sizeof(buf), fmt, adx); + panic("%s%s", prefix, buf); + } + if (ce != CE_IGNORE) { + printf("%s", prefix); + vprintf(fmt, adx); + } } void -- Andriy Gapon From owner-zfs-devel@FreeBSD.ORG Sun Dec 18 12:50:05 2011 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D8706106566B for ; Sun, 18 Dec 2011 12:50:05 +0000 (UTC) (envelope-from pawel@dawidek.net) Received: from mail.dawidek.net (60.wheelsystems.com [83.12.187.60]) by mx1.freebsd.org (Postfix) with ESMTP id 8476F8FC12 for ; Sun, 18 Dec 2011 12:50:05 +0000 (UTC) Received: from localhost (89-73-195-149.dynamic.chello.pl [89.73.195.149]) by mail.dawidek.net (Postfix) with ESMTPSA id B349AE9A; Sun, 18 Dec 2011 13:32:44 +0100 (CET) Date: Sun, 18 Dec 2011 13:31:38 +0100 From: Pawel Jakub Dawidek To: Andriy Gapon Message-ID: <20111218123137.GD1685@garage.freebsd.pl> References: <4EEC9CC9.9030001@FreeBSD.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="TybLhxa8M7aNoW+V" Content-Disposition: inline In-Reply-To: <4EEC9CC9.9030001@FreeBSD.org> X-OS: FreeBSD 9.0-CURRENT amd64 User-Agent: Mutt/1.5.21 (2010-09-15) Cc: zfs-devel@FreeBSD.org Subject: Re: vcmn_err: incorrect panic message X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 18 Dec 2011 12:50:05 -0000 --TybLhxa8M7aNoW+V Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sat, Dec 17, 2011 at 03:44:41PM +0200, Andriy Gapon wrote: > Guys, >=20 > what do you think about the following change or a variation of it? > With the current code, in the unfortunate event of panic-ing via vcmn_err= () a > panic message would actually be a format string (placeholders are not sub= stituted). The patch looks good. > commit f6cd1dc812b4cfdb3a123052ce0cb49a5afa941d > Author: Andriy Gapon > Date: Thu Dec 15 00:04:31 2011 +0200 >=20 > opensolaris compat: fix vcmn_err so that panic produces a proper mess= age >=20 > ... instead of just a verbatim format string >=20 > diff --git a/sys/cddl/compat/opensolaris/kern/opensolaris_cmn_err.c > b/sys/cddl/compat/opensolaris/kern/opensolaris_cmn_err.c > index 12e1854..abde30d 100644 > --- a/sys/cddl/compat/opensolaris/kern/opensolaris_cmn_err.c > +++ b/sys/cddl/compat/opensolaris/kern/opensolaris_cmn_err.c > @@ -28,29 +28,35 @@ void > vcmn_err(int ce, const char *fmt, va_list adx) > { > char buf[256]; > + const char *prefix; >=20 > + prefix =3D NULL; /* silence unwitty compilers */ > switch (ce) { > case CE_CONT: > - snprintf(buf, sizeof(buf), "Solaris(cont): %s\n", fmt); > + prefix =3D "Solaris(cont): "; > break; > case CE_NOTE: > - snprintf(buf, sizeof(buf), "Solaris: NOTICE: %s\n", fmt); > + prefix =3D "Solaris: NOTICE: "; > break; > case CE_WARN: > - snprintf(buf, sizeof(buf), "Solaris: WARNING: %s\n", fmt); > + prefix =3D "Solaris: WARNING: "; > break; > case CE_PANIC: > - snprintf(buf, sizeof(buf), "Solaris(panic): %s\n", fmt); > + prefix =3D "Solaris(panic): "; > break; > case CE_IGNORE: > break; > default: > panic("Solaris: unknown severity level"); > } > - if (ce =3D=3D CE_PANIC) > - panic("%s", buf); > - if (ce !=3D CE_IGNORE) > - vprintf(buf, adx); > + if (ce =3D=3D CE_PANIC) { > + vsnprintf(buf, sizeof(buf), fmt, adx); > + panic("%s%s", prefix, buf); > + } > + if (ce !=3D CE_IGNORE) { > + printf("%s", prefix); > + vprintf(fmt, adx); > + } > } >=20 > void --=20 Pawel Jakub Dawidek http://www.wheelsystems.com FreeBSD committer http://www.FreeBSD.org Am I Evil? Yes, I Am! http://yomoli.com --TybLhxa8M7aNoW+V Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (FreeBSD) iEYEARECAAYFAk7t3SgACgkQForvXbEpPzS/UQCgzG+1NjbxNM6eaqiVWoOGHK4I bLkAnisMJYS93VN3R6kithVmrYK4cPWN =u/vl -----END PGP SIGNATURE----- --TybLhxa8M7aNoW+V-- From owner-zfs-devel@FreeBSD.ORG Sat Feb 25 09:56:26 2012 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B60E8106564A; Sat, 25 Feb 2012 09:56:26 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from mail.vx.sk (mail.vx.sk [IPv6:2a01:4f8:150:6101::4]) by mx1.freebsd.org (Postfix) with ESMTP id 2EA948FC12; Sat, 25 Feb 2012 09:56:23 +0000 (UTC) Received: from core2.vx.sk (localhost [127.0.0.2]) by mail.vx.sk (Postfix) with ESMTP id 5B65B2510A; Sat, 25 Feb 2012 10:56:22 +0100 (CET) X-Virus-Scanned: amavisd-new at mail.vx.sk Received: from mail.vx.sk by core2.vx.sk (amavisd-new, unix socket) with LMTP id PFje1-WLoNTh; Sat, 25 Feb 2012 10:56:20 +0100 (CET) Received: from [10.9.8.1] (188-167-78-15.dynamic.chello.sk [188.167.78.15]) by mail.vx.sk (Postfix) with ESMTPSA id 258A7250FD; Sat, 25 Feb 2012 10:56:20 +0100 (CET) Message-ID: <4F48B043.6020006@FreeBSD.org> Date: Sat, 25 Feb 2012 10:56:19 +0100 From: Martin Matuska User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:10.0.2) Gecko/20120216 Thunderbird/10.0.2 MIME-Version: 1.0 To: Pawel Jakub Dawidek References: <201201052216.q05MGfLY058717@svn.freebsd.org> In-Reply-To: <201201052216.q05MGfLY058717@svn.freebsd.org> X-Enigmail-Version: 1.3.5 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Cc: zfs-devel@FreeBSD.org, Andriy Gapon Subject: Re: svn commit: r229663 - head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 25 Feb 2012 09:56:26 -0000 As of our current code arc_meta_limit is not enforced at all, because dnlc_reduce_cache() does nothing. This way the meta cache usually outgrows its limit which should be 1/4 of arc by default. Users in mailinglists report performance penalties due to this problem. We could learn from Brian Behlendorf's zfsonlinux and implement something similiar. In his first attempt: https://github.com/behlendorf/zfs/commit/6a8f9b6bf0de3e3d09fcfa32e129c978e7641a8f He added a arc_kmem_reap_now() if metadata is outside the arc_meta_limit. Then he introduced arc_adjust_meta() which looks promising, we might be interested in doing something like that: https://github.com/behlendorf/zfs/commit/ab26409db753bb087842ab6f1af943f3386c764f#L53R2004 On 5.1.2012 23:16, Pawel Jakub Dawidek wrote: > Author: pjd > Date: Thu Jan 5 22:16:41 2012 > New Revision: 229663 > URL: http://svn.freebsd.org/changeset/base/229663 > > Log: > - Allow to change vfs.zfs.arc_meta_limit at runtime. > - Change vfs.zfs.arc_meta_used from CTLFLAG_RDTUN to CTLFLAG_RD, as it is > not a tunable. > > MFC after: 3 days > > Modified: > head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c > > Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c > ============================================================================== > --- head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c Thu Jan 5 22:14:49 2012 (r229662) > +++ head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c Thu Jan 5 22:16:41 2012 (r229663) > @@ -464,10 +464,10 @@ static uint64_t arc_loaned_bytes; > static uint64_t arc_meta_used; > static uint64_t arc_meta_limit; > static uint64_t arc_meta_max = 0; > -SYSCTL_UQUAD(_vfs_zfs, OID_AUTO, arc_meta_used, CTLFLAG_RDTUN, > - &arc_meta_used, 0, "ARC metadata used"); > -SYSCTL_UQUAD(_vfs_zfs, OID_AUTO, arc_meta_limit, CTLFLAG_RDTUN, > - &arc_meta_limit, 0, "ARC metadata limit"); > +SYSCTL_UQUAD(_vfs_zfs, OID_AUTO, arc_meta_used, CTLFLAG_RD, &arc_meta_used, 0, > + "ARC metadata used"); > +SYSCTL_UQUAD(_vfs_zfs, OID_AUTO, arc_meta_limit, CTLFLAG_RW, &arc_meta_limit, 0, > + "ARC metadata limit"); > > typedef struct l2arc_buf_hdr l2arc_buf_hdr_t; > -- Martin Matuska FreeBSD committer http://blog.vx.sk From owner-zfs-devel@FreeBSD.ORG Sat Feb 25 10:15:04 2012 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4DBE7106566B; Sat, 25 Feb 2012 10:15:04 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from mail.vx.sk (mail.vx.sk [IPv6:2a01:4f8:150:6101::4]) by mx1.freebsd.org (Postfix) with ESMTP id 091EA8FC12; Sat, 25 Feb 2012 10:15:04 +0000 (UTC) Received: from core2.vx.sk (localhost [127.0.0.2]) by mail.vx.sk (Postfix) with ESMTP id F10A325890; Sat, 25 Feb 2012 11:15:01 +0100 (CET) X-Virus-Scanned: amavisd-new at mail.vx.sk Received: from mail.vx.sk by core2.vx.sk (amavisd-new, unix socket) with LMTP id nwA9sXoe1lX9; Sat, 25 Feb 2012 11:15:00 +0100 (CET) Received: from [10.9.8.1] (188-167-78-15.dynamic.chello.sk [188.167.78.15]) by mail.vx.sk (Postfix) with ESMTPSA id CA02325886; Sat, 25 Feb 2012 11:14:59 +0100 (CET) Message-ID: <4F48B4A4.7060800@FreeBSD.org> Date: Sat, 25 Feb 2012 11:15:00 +0100 From: Martin Matuska User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:10.0.2) Gecko/20120216 Thunderbird/10.0.2 MIME-Version: 1.0 To: Pawel Jakub Dawidek References: <201201052216.q05MGfLY058717@svn.freebsd.org> <4F48B043.6020006@FreeBSD.org> In-Reply-To: <4F48B043.6020006@FreeBSD.org> X-Enigmail-Version: 1.3.5 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Cc: zfs-devel@FreeBSD.org, Andriy Gapon Subject: Enforcing arc_meta_limit X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 25 Feb 2012 10:15:04 -0000 I am correcting myself: The wording "not at all" is incorrect, but it is possible to outgrow the limit and I can confirm that on several of my servers with higher ZFS utilization. On 25.2.2012 10:56, Martin Matuska wrote: > As of our current code arc_meta_limit is not enforced at all, because > dnlc_reduce_cache() does nothing. > This way the meta cache usually outgrows its limit which should be 1/4 > of arc by default. > Users in mailinglists report performance penalties due to this problem. > > We could learn from Brian Behlendorf's zfsonlinux and implement > something similiar. > > In his first attempt: > https://github.com/behlendorf/zfs/commit/6a8f9b6bf0de3e3d09fcfa32e129c978e7641a8f > He added a arc_kmem_reap_now() if metadata is outside the arc_meta_limit. > > Then he introduced arc_adjust_meta() which looks promising, we might be > interested in doing something like that: > https://github.com/behlendorf/zfs/commit/ab26409db753bb087842ab6f1af943f3386c764f#L53R2004 > > On 5.1.2012 23:16, Pawel Jakub Dawidek wrote: >> Author: pjd >> Date: Thu Jan 5 22:16:41 2012 >> New Revision: 229663 >> URL: http://svn.freebsd.org/changeset/base/229663 >> >> Log: >> - Allow to change vfs.zfs.arc_meta_limit at runtime. >> - Change vfs.zfs.arc_meta_used from CTLFLAG_RDTUN to CTLFLAG_RD, as it is >> not a tunable. >> >> MFC after: 3 days >> >> Modified: >> head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c >> >> Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c >> ============================================================================== >> --- head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c Thu Jan 5 22:14:49 2012 (r229662) >> +++ head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c Thu Jan 5 22:16:41 2012 (r229663) >> @@ -464,10 +464,10 @@ static uint64_t arc_loaned_bytes; >> static uint64_t arc_meta_used; >> static uint64_t arc_meta_limit; >> static uint64_t arc_meta_max = 0; >> -SYSCTL_UQUAD(_vfs_zfs, OID_AUTO, arc_meta_used, CTLFLAG_RDTUN, >> - &arc_meta_used, 0, "ARC metadata used"); >> -SYSCTL_UQUAD(_vfs_zfs, OID_AUTO, arc_meta_limit, CTLFLAG_RDTUN, >> - &arc_meta_limit, 0, "ARC metadata limit"); >> +SYSCTL_UQUAD(_vfs_zfs, OID_AUTO, arc_meta_used, CTLFLAG_RD, &arc_meta_used, 0, >> + "ARC metadata used"); >> +SYSCTL_UQUAD(_vfs_zfs, OID_AUTO, arc_meta_limit, CTLFLAG_RW, &arc_meta_limit, 0, >> + "ARC metadata limit"); >> >> typedef struct l2arc_buf_hdr l2arc_buf_hdr_t; >> > -- Martin Matuska FreeBSD committer http://blog.vx.sk From owner-zfs-devel@FreeBSD.ORG Mon May 14 22:03:44 2012 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 078A11065786; Mon, 14 May 2012 22:03:44 +0000 (UTC) (envelope-from dewcopper@gmail.com) Received: from mail-pz0-f54.google.com (mail-pz0-f54.google.com [209.85.210.54]) by mx1.freebsd.org (Postfix) with ESMTP id 9CA038FC1B; Mon, 14 May 2012 22:03:40 +0000 (UTC) Received: by dadv36 with SMTP id v36so7371632dad.13 for ; Mon, 14 May 2012 15:03:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:date:x-google-sender-auth:message-id:subject :from:to:content-type; bh=t3otcqBdzxMEfbZH8bw76fc4FPYr5NoxBTPGx+ZBPeM=; b=JPiAyyK+/JsCiENc/kFw+pblFoaN8kN9ajRooNxdUEB2wXOcElmmRmANKW0JRB0XnH UpCnmrEN0DgxULswbO77rDfhTbZHA/lhvAYrekchKiqCReKfs/23RhhIf01JK8dq792u kiPomGzPU9MTuwOpYurg7Gp/qPBrwcmk+jZAwy8xnWoMKcKK69YngKxhABXb7GwK23DB O51mtIx4Am/IE2LG5TSCUlOnI+rVU0Xtc3nHsKdlQFFYgrYYWR1MGLIKbyEyZRzprkhP ghazve40SzcI0tSooKNgGURFLYOUNG+kTlPJwb+wihhZRTJNrHiZ41gvXeRipLsClQ65 ibDw== MIME-Version: 1.0 Received: by 10.68.218.72 with SMTP id pe8mr26849356pbc.45.1337033019956; Mon, 14 May 2012 15:03:39 -0700 (PDT) Sender: dewcopper@gmail.com Received: by 10.143.154.25 with HTTP; Mon, 14 May 2012 15:03:39 -0700 (PDT) Date: Mon, 14 May 2012 15:03:39 -0700 X-Google-Sender-Auth: FQ3HDNLtfWSh7VTZg9xzc4Sdn3U Message-ID: From: Tushar Tambay To: zfs-devel@freebsd.org, Martin Matuska , pawel Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: Subject: introducing chuck silvers X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 14 May 2012 22:03:44 -0000 Hi pawel / xin / martin / all Hope you are all doing well. It has been quite a while since we exchanged mail. At HighCloud ( http://www.highcloudsecurity.com) , we released our first VM Centric Security product (based on freebsd and ZFS) back in October 2011. We are gaining traction with customers and are continuing to enhance the features, performance and stability of our product (just released 1.1.5 a couple of weeks ago). I'd like to introduce Chuck Silvers to this group. He is a former colleague from Veritas / Symantec's file systems team and now works with me at High Cloud Security. Chuck is also a long time netbsd developer (but don't hold that against him :-)) Chuck has been looking at ZFS performance and stability issues for us and wanted to share a couple of interesting results with this group. I'll let him speak about them himself. Regards, -- Tushar | 408.203.9736 | tushar.tambay@gmail.com From owner-zfs-devel@FreeBSD.ORG Mon May 14 22:05:35 2012 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id D68D1106566C; Mon, 14 May 2012 22:05:35 +0000 (UTC) (envelope-from dewcopper@gmail.com) Received: from mail-pz0-f54.google.com (mail-pz0-f54.google.com [209.85.210.54]) by mx1.freebsd.org (Postfix) with ESMTP id 9991F8FC0C; Mon, 14 May 2012 22:05:35 +0000 (UTC) Received: by dadv36 with SMTP id v36so7373501dad.13 for ; Mon, 14 May 2012 15:05:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; bh=px/A3z0tSLjs6cxfpbnR6A8omly+4Y9HLFTQj4jKjJc=; b=THM3w4Igf0M4zMbmd6tejO38oAS/uCpOzxRncOvVOoyX2BJGf4GwFcN0rmx98TlMye yaL5AZ8eHLXd2bOclfqESYV2glIdI5X2GoUCpkVUVx7x7ULj/QyVpDYyJh/wZZRNz7Mm hcrByWMGjMcDOrpY/Ms1KeJy15zUsqpDz2PZgw6d5LzS2KK0OGLMlRG17MTsXubiSbGh Tlet5uDk8QXs5EyhV4lwR0NrDDgKlibUJrj+zm9oMK/Ts/Jqm+BAD/wjexOGO8qYuOoQ EEDFXGBJJ81FR2VTV6i6agdBbStYva5Jk8WdaJ4jw/fqI68Y2+YiEFRuVtc7lx0Oq3Hl aCfQ== MIME-Version: 1.0 Received: by 10.68.202.8 with SMTP id ke8mr26786141pbc.94.1337033135306; Mon, 14 May 2012 15:05:35 -0700 (PDT) Sender: dewcopper@gmail.com Received: by 10.143.154.25 with HTTP; Mon, 14 May 2012 15:05:35 -0700 (PDT) In-Reply-To: References: Date: Mon, 14 May 2012 15:05:35 -0700 X-Google-Sender-Auth: IfNMsMHr2E8zyRQJ3xiDnWP9-xI Message-ID: From: Tushar Tambay To: zfs-devel@freebsd.org, Martin Matuska , pawel Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: Chuck Silvers Subject: Re: introducing chuck silvers X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 14 May 2012 22:05:36 -0000 forgot to CC Chuck himself :-) On Mon, May 14, 2012 at 3:03 PM, Tushar Tambay wrote: > Hi pawel / xin / martin / all > > Hope you are all doing well. > > It has been quite a while since we exchanged mail. At HighCloud ( > http://www.highcloudsecurity.com) , we released our first VM Centric > Security product (based on freebsd and ZFS) back in October 2011. We are > gaining traction with customers and are continuing to enhance the features, > performance and stability of our product (just released 1.1.5 a couple of > weeks ago). > > I'd like to introduce Chuck Silvers to this group. He is a former > colleague from Veritas / Symantec's file systems team and now works with > me at High Cloud Security. Chuck is also a long time netbsd developer (but > don't hold that against him :-)) > > Chuck has been looking at ZFS performance and stability issues for us and > wanted to share a couple of interesting results with this group. I'll let > him speak about them himself. > > Regards, > > -- > Tushar | 408.203.9736 | tushar.tambay@gmail.com > > -- Tushar | 408.203.9736 | tushar.tambay@gmail.com From owner-zfs-devel@FreeBSD.ORG Wed May 23 20:07:04 2012 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 23451106566C; Wed, 23 May 2012 20:07:04 +0000 (UTC) (envelope-from delphij@delphij.net) Received: from anubis.delphij.net (anubis.delphij.net [IPv6:2001:470:1:117::25]) by mx1.freebsd.org (Postfix) with ESMTP id CDBBB8FC12; Wed, 23 May 2012 20:07:03 +0000 (UTC) Received: from delta.delphij.net (drawbridge.ixsystems.com [206.40.55.65]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by anubis.delphij.net (Postfix) with ESMTPSA id 6601012DD6; Wed, 23 May 2012 13:07:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=delphij.net; s=anubis; t=1337803623; bh=Jf+tq2q4zX8iou6SWzq64fc7wJlbWSzHGUOkPZ9AnvA=; h=Date:From:Reply-To:To:CC:Subject:References:In-Reply-To; b=aDbKnUbYq056RT7C9SZIUmdCjyJG+Xm88Au5mNpNdCh+8y1bMcNWrIu7BlX+Q5UPI qpEF5C9kn4ZLXv6tiLv6a1EG8FyBtHNx2l7E6yZwP8CQkm8j8nk8fkvkp8s9kdcbv8 +VXAXbYyVE2fsXcdxaT15PvvHpBw8pSPEavVl+VQ= Message-ID: <4FBD4365.6030201@delphij.net> Date: Wed, 23 May 2012 13:07:01 -0700 From: Xin Li Organization: The FreeBSD Project MIME-Version: 1.0 To: Tushar Tambay References: In-Reply-To: X-Enigmail-Version: 1.5pre Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: d@delphij.net, zfs-devel@freebsd.org, Chuck Silvers Subject: Re: introducing chuck silvers X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: d@delphij.net List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 23 May 2012 20:07:04 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 On 05/14/12 15:05, Tushar Tambay wrote: > forgot to CC Chuck himself :-) > > On Mon, May 14, 2012 at 3:03 PM, Tushar Tambay > wrote: > >> Hi pawel / xin / martin / all >> >> Hope you are all doing well. >> >> It has been quite a while since we exchanged mail. At HighCloud >> ( http://www.highcloudsecurity.com) , we released our first VM >> Centric Security product (based on freebsd and ZFS) back in >> October 2011. We are gaining traction with customers and are >> continuing to enhance the features, performance and stability of >> our product (just released 1.1.5 a couple of weeks ago). >> >> I'd like to introduce Chuck Silvers to this group. He is a >> former colleague from Veritas / Symantec's file systems team and >> now works with me at High Cloud Security. Chuck is also a long >> time netbsd developer (but don't hold that against him :-)) >> >> Chuck has been looking at ZFS performance and stability issues >> for us and wanted to share a couple of interesting results with >> this group. I'll let him speak about them himself. Welcome! Cheers, - -- Xin LI https://www.delphij.net/ FreeBSD - The Power to Serve! Live free or die -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (FreeBSD) iQEcBAEBCAAGBQJPvUNlAAoJEG80Jeu8UPuz6sAH/1PX+BoZuxG5ExCMVBCijee5 KN+x3xOXllf50kOgj3ig3PVqa/loFYC/tXQjRsI+F/zqmBIbt06T7kTIaewgdDG6 0Z1Hl0qGz9xZDjrVuPuuBRsxCoF0gmjYB5bFAuhXsGFbwElA2Dyoqom0/Fjl0HwA SToZ0QNJwnDyZ0z43wuf3M5qMvB0gdzT6cKXsItMDePh1JURi6tPJFCOcBg8BQE6 LXNw9o7CjeTd3YT62X9RFSnW7w5oz6V0gIGlRhWKRZtawu0ObO6+aIqlfvjywQ9W rMxUmYEkb/q/CT/JKyOI7zhsb5mgiS59UHTdb3kKQqBxuAUnPQBBAJ+qyefcr6s= =eQkM -----END PGP SIGNATURE----- From owner-zfs-devel@FreeBSD.ORG Wed May 23 20:15:40 2012 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4A8B7106564A for ; Wed, 23 May 2012 20:15:40 +0000 (UTC) (envelope-from pawel@dawidek.net) Received: from mail.dawidek.net (60.wheelsystems.com [83.12.187.60]) by mx1.freebsd.org (Postfix) with ESMTP id EECD58FC0A for ; Wed, 23 May 2012 20:15:39 +0000 (UTC) Received: from localhost (89-73-195-149.dynamic.chello.pl [89.73.195.149]) by mail.dawidek.net (Postfix) with ESMTPSA id 6C6D535B; Wed, 23 May 2012 22:15:38 +0200 (CEST) Date: Wed, 23 May 2012 22:13:53 +0200 From: Pawel Jakub Dawidek To: zfs-devel@freebsd.org Message-ID: <20120523201353.GC1399@garage.freebsd.pl> References: MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="DIOMP1UsTsWJauNi" Content-Disposition: inline In-Reply-To: X-OS: FreeBSD 10.0-CURRENT amd64 User-Agent: Mutt/1.5.21 (2010-09-15) Cc: Subject: Re: introducing chuck silvers X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 23 May 2012 20:15:40 -0000 --DIOMP1UsTsWJauNi Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Mon, May 14, 2012 at 03:03:39PM -0700, Tushar Tambay wrote: > Hi pawel / xin / martin / all >=20 > Hope you are all doing well. >=20 > It has been quite a while since we exchanged mail. At HighCloud ( > http://www.highcloudsecurity.com) , we released our first VM Centric > Security product (based on freebsd and ZFS) back in October 2011. We are > gaining traction with customers and are continuing to enhance the feature= s, > performance and stability of our product (just released 1.1.5 a couple of > weeks ago). >=20 > I'd like to introduce Chuck Silvers to this group. He is a former colleag= ue > from Veritas / Symantec's file systems team and now works with me at High > Cloud Security. Chuck is also a long time netbsd developer (but don't hold > that against him :-)) >=20 > Chuck has been looking at ZFS performance and stability issues for us and > wanted to share a couple of interesting results with this group. I'll let > him speak about them himself. Welcome Chuck and congratulations on your FreeBSD/ZFS-based product! I added your e-mail to the list. Let me know if you received it. --=20 Pawel Jakub Dawidek http://www.wheelsystems.com FreeBSD committer http://www.FreeBSD.org Am I Evil? Yes, I Am! http://tupytaj.pl --DIOMP1UsTsWJauNi Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (FreeBSD) iEYEARECAAYFAk+9RQEACgkQForvXbEpPzSOoQCg3Q9/ihJiGLPn8ZwWMl2MT6iF xJIAoNl15jaOCIdIDYNTDA1THUMGyyys =rmA+ -----END PGP SIGNATURE----- --DIOMP1UsTsWJauNi-- From owner-zfs-devel@FreeBSD.ORG Thu May 24 23:30:30 2012 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 40B9B106564A for ; Thu, 24 May 2012 23:30:30 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from mail.vx.sk (mail.vx.sk [176.9.45.25]) by mx1.freebsd.org (Postfix) with ESMTP id B36B18FC15 for ; Thu, 24 May 2012 23:30:29 +0000 (UTC) Received: from core2.vx.sk (localhost [127.0.0.2]) by mail.vx.sk (Postfix) with ESMTP id 5773A217EC for ; Fri, 25 May 2012 01:30:23 +0200 (CEST) X-Virus-Scanned: amavisd-new at mail.vx.sk Received: from mail.vx.sk by core2.vx.sk (amavisd-new, unix socket) with LMTP id Ih3AFwTdpEC1 for ; Fri, 25 May 2012 01:30:21 +0200 (CEST) Received: from [10.9.8.1] (188-167-78-15.dynamic.chello.sk [188.167.78.15]) by mail.vx.sk (Postfix) with ESMTPSA id EF6EF217E4 for ; Fri, 25 May 2012 01:30:20 +0200 (CEST) Message-ID: <4FBEC48D.5080600@FreeBSD.org> Date: Fri, 25 May 2012 01:30:21 +0200 From: Martin Matuska User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:12.0) Gecko/20120428 Thunderbird/12.0.1 MIME-Version: 1.0 To: zfs-devel@FreeBSD.org X-Enigmail-Version: 1.4.1 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Cc: Subject: ZFS features (SPA_VERSION successor) X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 May 2012 23:30:30 -0000 Hi folks, last week Illumos has introduced a new concept called "ZFS features", a version-independent successor to SPA_VERSION, together with the first "feature" - async destroy. It is a huge commit and will take some time to implement (there is quite detailed documentation and manpages inside). As to the text in the issue 2619, the feature flags are based on a proposal to the ZFS working group: http://lists.freebsd.org/pipermail/freebsd-fs/2011-May/011568.html Maybe some of FreeBSD-related ZFS development could take advantage of this, when ported. References: https://hg.openindiana.org/upstream/illumos/illumos-gate/rev/2889e2596bd6 https://hg.openindiana.org/upstream/illumos/illumos-gate/rev/1949b688d5fb https://www.illumos.org/issues/2619 https://www.illumos.org/issues/2747 Cheers, mm -- Martin Matuska FreeBSD committer http://blog.vx.sk From owner-zfs-devel@FreeBSD.ORG Tue May 29 18:50:30 2012 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2683C106564A; Tue, 29 May 2012 18:50:30 +0000 (UTC) (envelope-from chs@highcloudsecurity.com) Received: from mail.highcloudsecurity.com (50-0-150-29.dedicated.static.sonic.net [50.0.150.29]) by mx1.freebsd.org (Postfix) with ESMTP id 05C228FC15; Tue, 29 May 2012 18:50:29 +0000 (UTC) Received: from localhost (localhost.localdomain [127.0.0.1]) by mail.highcloudsecurity.com (Postfix) with ESMTP id E5F23321081; Tue, 29 May 2012 19:42:09 +0100 (BST) X-Virus-Scanned: amavisd-new at highcloudsecurity.com Received: from mail.highcloudsecurity.com ([127.0.0.1]) by localhost (mail.highcloudsecurity.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id yfm+NVYjwckl; Tue, 29 May 2012 19:42:08 +0100 (BST) Received: from yoshi.hcs.int (unknown [10.238.16.254]) by mail.highcloudsecurity.com (Postfix) with ESMTPSA id C164A32107E; Tue, 29 May 2012 19:42:08 +0100 (BST) Date: Tue, 29 May 2012 11:42:07 -0700 From: Chuck Silvers To: Pawel Jakub Dawidek Message-ID: <20120529184207.GB11065@yoshi.hcs.int> References: <20120523201353.GC1399@garage.freebsd.pl> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20120523201353.GC1399@garage.freebsd.pl> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: zfs-devel@freebsd.org Subject: Re: introducing chuck silvers X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 29 May 2012 18:50:30 -0000 On Wed, May 23, 2012 at 10:13:53PM +0200, Pawel Jakub Dawidek wrote: > On Mon, May 14, 2012 at 03:03:39PM -0700, Tushar Tambay wrote: > > Hi pawel / xin / martin / all > > > > Hope you are all doing well. > > > > It has been quite a while since we exchanged mail. At HighCloud ( > > http://www.highcloudsecurity.com) , we released our first VM Centric > > Security product (based on freebsd and ZFS) back in October 2011. We are > > gaining traction with customers and are continuing to enhance the features, > > performance and stability of our product (just released 1.1.5 a couple of > > weeks ago). > > > > I'd like to introduce Chuck Silvers to this group. He is a former colleague > > from Veritas / Symantec's file systems team and now works with me at High > > Cloud Security. Chuck is also a long time netbsd developer (but don't hold > > that against him :-)) > > > > Chuck has been looking at ZFS performance and stability issues for us and > > wanted to share a couple of interesting results with this group. I'll let > > him speak about them himself. > > Welcome Chuck and congratulations on your FreeBSD/ZFS-based product! > > I added your e-mail to the list. Let me know if you received it. hi pawel, I got it, thanks. we have a number of patches to freebsd that we're using, most of which modify parts of the system other than ZFS, should I just submit those as PRs? or post them to whichever freebsd-* mailing list seems the most appropriate? the few that are ZFS-specific I'll send here shortly. -Chuck From owner-zfs-devel@FreeBSD.ORG Tue May 29 20:38:31 2012 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 706D9106566C; Tue, 29 May 2012 20:38:31 +0000 (UTC) (envelope-from gibbs@scsiguy.com) Received: from aslan.scsiguy.com (ns1.scsiguy.com [70.89.174.89]) by mx1.freebsd.org (Postfix) with ESMTP id 237BC8FC17; Tue, 29 May 2012 20:38:31 +0000 (UTC) Received: from stevebender.sldomain.com (207-225-98-3.dia.static.qwest.net [207.225.98.3]) (authenticated bits=0) by aslan.scsiguy.com (8.14.5/8.14.5) with ESMTP id q4TKcU5J020230 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NO); Tue, 29 May 2012 14:38:30 -0600 (MDT) (envelope-from gibbs@scsiguy.com) Mime-Version: 1.0 (Apple Message framework v1278) Content-Type: text/plain; charset=us-ascii From: "Justin T. Gibbs" In-Reply-To: <20120529184207.GB11065@yoshi.hcs.int> Date: Tue, 29 May 2012 14:38:24 -0600 Content-Transfer-Encoding: quoted-printable Message-Id: <8DA92D89-BC0E-450F-A4D8-ED64CACD3471@scsiguy.com> References: <20120523201353.GC1399@garage.freebsd.pl> <20120529184207.GB11065@yoshi.hcs.int> To: Chuck Silvers X-Mailer: Apple Mail (2.1278) X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (aslan.scsiguy.com [70.89.174.89]); Tue, 29 May 2012 14:38:30 -0600 (MDT) Cc: zfs-devel@freebsd.org Subject: Re: introducing chuck silvers X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 29 May 2012 20:38:31 -0000 On May 29, 2012, at 12:42 PM, Chuck Silvers wrote: > On Wed, May 23, 2012 at 10:13:53PM +0200, Pawel Jakub Dawidek wrote: >> On Mon, May 14, 2012 at 03:03:39PM -0700, Tushar Tambay wrote: >>> Hi pawel / xin / martin / all >>>=20 >>> Hope you are all doing well. >>>=20 >>> It has been quite a while since we exchanged mail. At HighCloud ( >>> http://www.highcloudsecurity.com) , we released our first VM Centric >>> Security product (based on freebsd and ZFS) back in October 2011. We = are >>> gaining traction with customers and are continuing to enhance the = features, >>> performance and stability of our product (just released 1.1.5 a = couple of >>> weeks ago). >>>=20 >>> I'd like to introduce Chuck Silvers to this group. He is a former = colleague >>> from Veritas / Symantec's file systems team and now works with me = at High >>> Cloud Security. Chuck is also a long time netbsd developer (but = don't hold >>> that against him :-)) >>>=20 >>> Chuck has been looking at ZFS performance and stability issues for = us and >>> wanted to share a couple of interesting results with this group. = I'll let >>> him speak about them himself. >>=20 >> Welcome Chuck and congratulations on your FreeBSD/ZFS-based product! >>=20 >> I added your e-mail to the list. Let me know if you received it. >=20 > hi pawel, >=20 > I got it, thanks. >=20 > we have a number of patches to freebsd that we're using, most of which > modify parts of the system other than ZFS, should I just submit those = as PRs? > or post them to whichever freebsd-* mailing list seems the most = appropriate? > the few that are ZFS-specific I'll send here shortly. >=20 PRs often (unfortunately) get lost. If you can provide an enumeration = of your changes, I can introduce you to a committer that works in that area to = help get the changes into the tree. Longer term, we should figure out how to = make you a committer. :-) -- Justin= From owner-zfs-devel@FreeBSD.ORG Wed May 30 01:00:06 2012 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id E4113106566C; Wed, 30 May 2012 01:00:05 +0000 (UTC) (envelope-from araujobsdport@gmail.com) Received: from mail-gh0-f172.google.com (mail-gh0-f172.google.com [209.85.160.172]) by mx1.freebsd.org (Postfix) with ESMTP id 7B2AC8FC0C; Wed, 30 May 2012 01:00:05 +0000 (UTC) Received: by ghbg16 with SMTP id g16so3027477ghb.17 for ; Tue, 29 May 2012 18:00:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:reply-to:in-reply-to:references:date:message-id :subject:from:to:cc:content-type; bh=sD69/mHguj9WkfEcmCD9rzpVWALJeBuZaw4Ah7WXwVU=; b=xWn5NBm1RJqgDRqXp4Vk9o5bXMaeKz6sM7V0qObW5pXW1UV7pmaBU/YPsIX/s36aHO CwHZTX1yMXbpJM8nTDsHrXR9FJ6osDa/KZwuwLjdrDC6CGNPaQARbfliLn6Gs+O0S3pO kP3mHcAZRu5mc06Pz1fbK0GYgff/s8eXxViiKQuYcNIqhjh4ALw4MN7xuef979Bcb0+8 TFfIGNs8HNVyK6nxYdXVW9YPKBZ/TYVxXp5NEFiTqamKW9wje26rWydPwKqt55Y3prCd /DxlqjKTsPmr684Xxbo7iApkyUt6lx5vFQtlNAP+zMkRNGNJk+8+xh6jzsJhWXtU9319 CZzQ== MIME-Version: 1.0 Received: by 10.43.49.3 with SMTP id uy3mr853165icb.2.1338339604274; Tue, 29 May 2012 18:00:04 -0700 (PDT) Received: by 10.231.244.8 with HTTP; Tue, 29 May 2012 18:00:04 -0700 (PDT) Received: by 10.231.244.8 with HTTP; Tue, 29 May 2012 18:00:04 -0700 (PDT) In-Reply-To: <20120529184207.GB11065@yoshi.hcs.int> References: <20120523201353.GC1399@garage.freebsd.pl> <20120529184207.GB11065@yoshi.hcs.int> Date: Wed, 30 May 2012 09:00:04 +0800 Message-ID: From: Marcelo Araujo To: Chuck Silvers Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: zfs-devel@freebsd.org Subject: Re: introducing chuck silvers X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: araujo@FreeBSD.org List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 30 May 2012 01:00:06 -0000 Hey dear Chuck, I'm wondering if you could share all changes made over ZFS. I'd like to give a try in your improvements. BR, - Araujo On May 30, 2012 2:50 AM, "Chuck Silvers" wrote: > On Wed, May 23, 2012 at 10:13:53PM +0200, Pawel Jakub Dawidek wrote: > > On Mon, May 14, 2012 at 03:03:39PM -0700, Tushar Tambay wrote: > > > Hi pawel / xin / martin / all > > > > > > Hope you are all doing well. > > > > > > It has been quite a while since we exchanged mail. At HighCloud ( > > > http://www.highcloudsecurity.com) , we released our first VM Centric > > > Security product (based on freebsd and ZFS) back in October 2011. We > are > > > gaining traction with customers and are continuing to enhance the > features, > > > performance and stability of our product (just released 1.1.5 a couple > of > > > weeks ago). > > > > > > I'd like to introduce Chuck Silvers to this group. He is a former > colleague > > > from Veritas / Symantec's file systems team and now works with me at > High > > > Cloud Security. Chuck is also a long time netbsd developer (but don't > hold > > > that against him :-)) > > > > > > Chuck has been looking at ZFS performance and stability issues for us > and > > > wanted to share a couple of interesting results with this group. I'll > let > > > him speak about them himself. > > > > Welcome Chuck and congratulations on your FreeBSD/ZFS-based product! > > > > I added your e-mail to the list. Let me know if you received it. > > hi pawel, > > I got it, thanks. > > we have a number of patches to freebsd that we're using, most of which > modify parts of the system other than ZFS, should I just submit those as > PRs? > or post them to whichever freebsd-* mailing list seems the most > appropriate? > the few that are ZFS-specific I'll send here shortly. > > -Chuck > _______________________________________________ > zfs-devel@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/zfs-devel > To unsubscribe, send any mail to "zfs-devel-unsubscribe@freebsd.org" > From owner-zfs-devel@FreeBSD.ORG Wed May 30 07:06:35 2012 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 201091065670 for ; Wed, 30 May 2012 07:06:35 +0000 (UTC) (envelope-from pawel@dawidek.net) Received: from mail.dawidek.net (60.wheelsystems.com [83.12.187.60]) by mx1.freebsd.org (Postfix) with ESMTP id C297D8FC0A for ; Wed, 30 May 2012 07:06:34 +0000 (UTC) Received: from localhost (58.wheelsystems.com [83.12.187.58]) by mail.dawidek.net (Postfix) with ESMTPSA id 8B4E7D35; Wed, 30 May 2012 09:06:33 +0200 (CEST) Date: Wed, 30 May 2012 09:04:44 +0200 From: Pawel Jakub Dawidek To: "Justin T. Gibbs" Message-ID: <20120530070444.GA1372@garage.freebsd.pl> References: <20120523201353.GC1399@garage.freebsd.pl> <20120529184207.GB11065@yoshi.hcs.int> <8DA92D89-BC0E-450F-A4D8-ED64CACD3471@scsiguy.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="0F1p//8PRICkK4MW" Content-Disposition: inline In-Reply-To: <8DA92D89-BC0E-450F-A4D8-ED64CACD3471@scsiguy.com> X-OS: FreeBSD 10.0-CURRENT amd64 User-Agent: Mutt/1.5.21 (2010-09-15) Cc: zfs-devel@freebsd.org Subject: Re: introducing chuck silvers X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 30 May 2012 07:06:35 -0000 --0F1p//8PRICkK4MW Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Tue, May 29, 2012 at 02:38:24PM -0600, Justin T. Gibbs wrote: > > we have a number of patches to freebsd that we're using, most of which > > modify parts of the system other than ZFS, should I just submit those a= s PRs? > > or post them to whichever freebsd-* mailing list seems the most appropr= iate? > > the few that are ZFS-specific I'll send here shortly. > >=20 >=20 > PRs often (unfortunately) get lost. If you can provide an enumeration of= your > changes, I can introduce you to a committer that works in that area to he= lp get > the changes into the tree. Longer term, we should figure out how to make= you > a committer. :-) I agree, the best chances to get your changes in is to find active committer in your area and approach him directly or get a commit bit. This mailing list is a good place to start when it comes to ZFS changes, but changes to other parts of the system that are needed to improve ZFS are welcome here too, of course. --=20 Pawel Jakub Dawidek http://www.wheelsystems.com FreeBSD committer http://www.FreeBSD.org Am I Evil? Yes, I Am! http://tupytaj.pl --0F1p//8PRICkK4MW Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (FreeBSD) iEYEARECAAYFAk/FxowACgkQForvXbEpPzSJaACff+iqloDFmyLdn3qmrFRCnmdZ tiQAnj8kpbNQtWN07Oq9JyLyKE1A/vuf =gYbD -----END PGP SIGNATURE----- --0F1p//8PRICkK4MW-- From owner-zfs-devel@FreeBSD.ORG Thu May 31 00:11:39 2012 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 9AD05106564A; Thu, 31 May 2012 00:11:39 +0000 (UTC) (envelope-from chs@highcloudsecurity.com) Received: from mail.highcloudsecurity.com (50-0-150-29.dedicated.static.sonic.net [50.0.150.29]) by mx1.freebsd.org (Postfix) with ESMTP id 728FE8FC08; Thu, 31 May 2012 00:11:39 +0000 (UTC) Received: from localhost (localhost.localdomain [127.0.0.1]) by mail.highcloudsecurity.com (Postfix) with ESMTP id 1F345321080; Thu, 31 May 2012 01:11:33 +0100 (BST) X-Virus-Scanned: amavisd-new at highcloudsecurity.com Received: from mail.highcloudsecurity.com ([127.0.0.1]) by localhost (mail.highcloudsecurity.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id c7CCLs1WnvZ3; Thu, 31 May 2012 01:11:32 +0100 (BST) Received: from yoshi.hcs.int (unknown [10.238.16.254]) by mail.highcloudsecurity.com (Postfix) with ESMTPSA id 3B1CB32107F; Thu, 31 May 2012 01:11:32 +0100 (BST) Date: Wed, 30 May 2012 17:11:30 -0700 From: Chuck Silvers To: "Justin T. Gibbs" Message-ID: <20120531001130.GC11065@yoshi.hcs.int> References: <20120523201353.GC1399@garage.freebsd.pl> <20120529184207.GB11065@yoshi.hcs.int> <8DA92D89-BC0E-450F-A4D8-ED64CACD3471@scsiguy.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <8DA92D89-BC0E-450F-A4D8-ED64CACD3471@scsiguy.com> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: zfs-devel@freebsd.org Subject: Re: introducing chuck silvers X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 31 May 2012 00:11:39 -0000 On Tue, May 29, 2012 at 02:38:24PM -0600, Justin T. Gibbs wrote: > PRs often (unfortunately) get lost. If you can provide an enumeration of your > changes, I can introduce you to a committer that works in that area to help get > the changes into the tree. thanks, that will be great. here's our list of patches: - fixes and enhancements to the aesni driver - replace the asm version of aesni_decrypt_cbc() with a C wrapper around aesni_dec() (like aesni_encrypt_cbc() was already a wrapper around aesni_enc()) since using the asm one often crashed mysteriously for us. - allow using the same aesni session from multiple CPUs simultaneously by moving the saved FPU context from the session to the stack. - mark aesni as CRYPTOCAP_F_SYNC since it is synchronous. - enhance the NFS server to support multiple writes to the same file in parallel if the underlying fs supports it. this builds on top of the addition of a "flags" parameter to VFS_FHTOVP() that I backported from -current to our 8.2-based tree. - allow adjusting NKPT via the kernel config file on amd64 too. - have uuidgen ignore ipfw0 when looking for a MAC address. (this only shows up when ipfw is built into the kernel and all the interfaces with real MAC addresses are from modules.) there are a couple minor ZFS ones, I'll send those here separately. some of these are really more workarounds than fixes, but you get the idea. we also have a fix for the open-vm-tools vmxnet driver to support more than 4GB of RAM. I suppose I should push that back to the sourceforge project as well as the freebsd ports tree. we also built several other ports (php5-* and py26-django) without gettext or libiconv (to avoid GPL3). I don't know if these are interesting to anyone else. I also ported the openbsd "vmt" driver to freebsd since that was easier than getting the open-vm-tools vmtoolsd to build without libiconv. I think that's all so far. > Longer term, we should figure out how to make you a committer. :-) I'm happy to feed our changes back through other folks. :-) -Chuck From owner-zfs-devel@FreeBSD.ORG Thu May 31 00:32:36 2012 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7B9E1106564A for ; Thu, 31 May 2012 00:32:36 +0000 (UTC) (envelope-from chs@highcloudsecurity.com) Received: from mail.highcloudsecurity.com (50-0-150-29.dedicated.static.sonic.net [50.0.150.29]) by mx1.freebsd.org (Postfix) with ESMTP id 52B818FC12 for ; Thu, 31 May 2012 00:32:36 +0000 (UTC) Received: from localhost (localhost.localdomain [127.0.0.1]) by mail.highcloudsecurity.com (Postfix) with ESMTP id 00F31110002 for ; Thu, 31 May 2012 01:32:35 +0100 (BST) X-Virus-Scanned: amavisd-new at highcloudsecurity.com Received: from mail.highcloudsecurity.com ([127.0.0.1]) by localhost (mail.highcloudsecurity.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 9+GZv9gHI5I6 for ; Thu, 31 May 2012 01:32:35 +0100 (BST) Received: from yoshi.hcs.int (unknown [10.238.16.254]) by mail.highcloudsecurity.com (Postfix) with ESMTPSA id 4A68132107F for ; Thu, 31 May 2012 01:32:35 +0100 (BST) Date: Wed, 30 May 2012 17:32:34 -0700 From: Chuck Silvers To: zfs-devel@freebsd.org Message-ID: <20120531003233.GD11065@yoshi.hcs.int> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="9jxsPFA5p3P2qPhR" Content-Disposition: inline User-Agent: Mutt/1.5.21 (2010-09-15) Subject: ZFS patches X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 31 May 2012 00:32:36 -0000 --9jxsPFA5p3P2qPhR Content-Type: text/plain; charset=us-ascii Content-Disposition: inline we only have a few changes to ZFS itself, and now that I look I see that you've found one of them independently (r230256). the other ones are: - improve performance of booting from a ZFS root under ESXi. previously this would sit there for about 5 minutes before even starting to load the kernel. the problem is that the ZFS pool-discovery code opens every possible GPT partition looking for pools, and it rereads the GPT each time, one sector at a time. we changed the GPT code to read the whole GPT in one shot, which reduced the delay to almost nothing. I remember seeing some discussion about a PR on this topic some time back but I don't know if any fix was ever applied and I don't see the PR now. as I recall, the proposal in that discussion was to improve the boot code caching so that it wouldn't reread the GPT at all, which I imagine would work just as well as what we did. (hmm, this isn't actually a change to ZFS either.) - make zfs_resilver_delay and zfs_resilver_min_time_ms tunable via sysctl. patches for both of these are attached. -Chuck --9jxsPFA5p3P2qPhR Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="patch.biosdisk-gpt" Index: sys/boot/i386/libi386/biosdisk.c =================================================================== RCS file: /home/chs/freebsd/cvs/src/sys/boot/i386/libi386/biosdisk.c,v retrieving revision 1.62.2.2.4.1 diff -u -p -r1.62.2.2.4.1 biosdisk.c --- sys/boot/i386/libi386/biosdisk.c 21 Dec 2010 17:09:25 -0000 1.62.2.2.4.1 +++ sys/boot/i386/libi386/biosdisk.c 6 Jul 2011 20:44:31 -0000 @@ -853,9 +853,10 @@ bd_open_gpt(struct open_disk *od, struct struct gpt_hdr *hdr; struct gpt_ent *ent; struct gpt_part *gp; - int entries_per_sec, error, i, part; + int entries_per_sec, error, i, part, nsec; daddr_t lba, elba; char gpt[BIOSDISK_SECSIZE], tbl[BIOSDISK_SECSIZE]; + char *buf, *tblp; /* * Following calculations attempt to determine the correct value @@ -900,22 +901,31 @@ bd_open_gpt(struct open_disk *od, struct return (EINVAL); } + entries_per_sec = BIOSDISK_SECSIZE / hdr->hdr_entsz; + nsec = hdr->hdr_entries / entries_per_sec; + if (nsec > 128) { + DEBUG("too many GPT table sectors %d", nsec); + return (EINVAL); + } + buf = alloca(nsec * BIOSDISK_SECSIZE); + if (bd_read(od, hdr->hdr_lba_table, nsec, buf)) { + DEBUG("error reading GPT table"); + return (EIO); + } + /* Now walk the partition table to count the number of valid partitions. */ part = 0; - entries_per_sec = BIOSDISK_SECSIZE / hdr->hdr_entsz; + tblp = buf; elba = hdr->hdr_lba_table + hdr->hdr_entries / entries_per_sec; for (lba = hdr->hdr_lba_table; lba < elba; lba++) { - if (bd_read(od, lba, 1, tbl)) { - DEBUG("error reading GPT table"); - return (EIO); - } for (i = 0; i < entries_per_sec; i++) { - ent = (struct gpt_ent *)(tbl + i * hdr->hdr_entsz); + ent = (struct gpt_ent *)(tblp + i * hdr->hdr_entsz); if (uuid_is_nil(&ent->ent_type, NULL) || ent->ent_lba_start == 0 || ent->ent_lba_end < ent->ent_lba_start) continue; part++; } + tblp += BIOSDISK_SECSIZE; } /* Save the important information about all the valid partitions. */ @@ -923,14 +933,10 @@ bd_open_gpt(struct open_disk *od, struct if (part != 0) { od->od_partitions = malloc(part * sizeof(struct gpt_part)); part = 0; + tblp = buf; for (lba = hdr->hdr_lba_table; lba < elba; lba++) { - if (bd_read(od, lba, 1, tbl)) { - DEBUG("error reading GPT table"); - error = EIO; - goto out; - } for (i = 0; i < entries_per_sec; i++) { - ent = (struct gpt_ent *)(tbl + i * hdr->hdr_entsz); + ent = (struct gpt_ent *)(tblp + i * hdr->hdr_entsz); if (uuid_is_nil(&ent->ent_type, NULL) || ent->ent_lba_start == 0 || ent->ent_lba_end < ent->ent_lba_start) @@ -942,6 +948,7 @@ bd_open_gpt(struct open_disk *od, struct od->od_partitions[part].gp_end = ent->ent_lba_end; part++; } + tblp += BIOSDISK_SECSIZE; } } od->od_flags |= BD_GPTOK; --9jxsPFA5p3P2qPhR Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="patch.zfs-resilver-sysctl" --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dsl_scan.c.ORG 2011-10-12 14:27:36.000000000 +0530 +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dsl_scan.c 2011-10-12 14:38:44.000000000 +0530 @@ -69,6 +69,14 @@ enum ddt_class zfs_scrub_ddt_class_max = DDT_CLASS_DUPLICATE; int dsl_scan_delay_completion = B_FALSE; /* set to delay scan completion */ +SYSCTL_DECL(_vfs_zfs); +TUNABLE_INT("vfs.zfs.zfs_resilver_delay", &zfs_resilver_delay); +SYSCTL_INT(_vfs_zfs, OID_AUTO, zfs_resilver_delay, CTLFLAG_RW, + &zfs_resilver_delay, 0, "Resilver delay"); +TUNABLE_INT("vfs.zfs.zfs_resilver_min_time_ms", &zfs_resilver_min_time_ms); +SYSCTL_INT(_vfs_zfs, OID_AUTO, zfs_resilver_min_time_ms, CTLFLAG_RW, + &zfs_resilver_min_time_ms, 0, "Resilver min time"); + #define DSL_SCAN_IS_SCRUB_RESILVER(scn) \ ((scn)->scn_phys.scn_func == POOL_SCAN_SCRUB || \ (scn)->scn_phys.scn_func == POOL_SCAN_RESILVER) --9jxsPFA5p3P2qPhR-- From owner-zfs-devel@FreeBSD.ORG Thu May 31 06:20:16 2012 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6D7BF106564A for ; Thu, 31 May 2012 06:20:16 +0000 (UTC) (envelope-from araujobsdport@gmail.com) Received: from mail-yw0-f54.google.com (mail-yw0-f54.google.com [209.85.213.54]) by mx1.freebsd.org (Postfix) with ESMTP id 2715F8FC17 for ; Thu, 31 May 2012 06:20:15 +0000 (UTC) Received: by yhgm50 with SMTP id m50so485360yhg.13 for ; Wed, 30 May 2012 23:20:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:reply-to:in-reply-to:references:date:message-id :subject:from:to:cc:content-type; bh=gCkKF9+C6sREsyIXgRqnwUEYw+udnfXy1lnNb/JYXcU=; b=Dh5wLEcGvZ+80ZmXBXjAhE03z7qcTVcBXxDs26WecYE6Y/nEnqmjSk3v+SuxCX6vpy 9d7ZkbBpWF2JzZGwbmXP/EaH5ujUgKBdZB7Jwl9GvGkYdVKslXOnyAXBXSJLNUIqqiAa HlLSmyX2oZNmjmzBoda96ImVNoJApBkycWA80ZPG9HITiaperlX4XRJgAzJJIQ8aPWch a9pR1B/uWbKV6XaR2tF9c6lzl7aCM0BIFEXrgGzdY0XtCG/izw4MkZkjTS6rxMeQKUWs P+1wwxzHpQDq6nFPwXKcTytfRN1ZTQ/NkmyRkUX4Y5g8ttAsRSvWVyP0hMcC6lBRjNHL 8IEw== MIME-Version: 1.0 Received: by 10.50.181.232 with SMTP id dz8mr12888881igc.72.1338445215097; Wed, 30 May 2012 23:20:15 -0700 (PDT) Received: by 10.231.244.8 with HTTP; Wed, 30 May 2012 23:20:15 -0700 (PDT) In-Reply-To: <20120531003233.GD11065@yoshi.hcs.int> References: <20120531003233.GD11065@yoshi.hcs.int> Date: Thu, 31 May 2012 14:20:15 +0800 Message-ID: From: Marcelo Araujo To: Chuck Silvers Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: zfs-devel@freebsd.org Subject: Re: ZFS patches X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: araujo@FreeBSD.org List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 31 May 2012 06:20:16 -0000 2012/5/31 Chuck Silvers > > - make zfs_resilver_delay and zfs_resilver_min_time_ms tunable via sysctl. > > Dear Chuck, The patch above looks very good and useful. Thanks to share it with us, I believe you shall open a PR and let's try to forward to someone with write access on zfs. Best Regards, -- Marcelo Araujo araujo@FreeBSD.org From owner-zfs-devel@FreeBSD.ORG Thu May 31 07:48:04 2012 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0AAF41065672 for ; Thu, 31 May 2012 07:48:04 +0000 (UTC) (envelope-from araujobsdport@gmail.com) Received: from mail-yx0-f182.google.com (mail-yx0-f182.google.com [209.85.213.182]) by mx1.freebsd.org (Postfix) with ESMTP id B6D6D8FC08 for ; Thu, 31 May 2012 07:48:03 +0000 (UTC) Received: by yenl8 with SMTP id l8so674437yen.13 for ; Thu, 31 May 2012 00:48:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:reply-to:date:message-id:subject:from:to:content-type; bh=gGCHUOT7LIyYWi5fftWw85TsTG0sLrY55JFL0Z0X7ig=; b=q/Wm1oUIU+oo6ofJ6kUF1hOVo5UzpMAD8x0Wbogw1CSlnExv/RJjW+ll9bm3dqUgA0 4cfeRxzGzIk2yIYgYBWkNfCd23xgqzMf6v4sotsf+HnBtthIU1EioNuQWTJD31lnEfR6 Y86344yiy0SPv3c/Ol/e4q7/ZR0wi0Sz5hb4t0w+bG3kXTZoGlPeKEbr5JDzFwtJoSJs 7Hir7+qLw4qIDuZBG6RUblHPYO5ciul+U2FMZGpTduzVrEyMqsm90b2+ILi4QW7GOVJ5 8ZNpBq3kNZm7UbmivgAxofRHE55yuSFnydmGO+tFGb4RxwNpQ7PCM3mrF030in0adHGO xa0A== MIME-Version: 1.0 Received: by 10.50.47.135 with SMTP id d7mr671666ign.66.1338450482439; Thu, 31 May 2012 00:48:02 -0700 (PDT) Received: by 10.231.244.8 with HTTP; Thu, 31 May 2012 00:48:02 -0700 (PDT) Date: Thu, 31 May 2012 15:48:02 +0800 Message-ID: From: Marcelo Araujo To: zfs-devel@freebsd.org Content-Type: multipart/mixed; boundary=14dae93403ad8e26d204c15049cd X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: SPA version 5000 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: araujo@FreeBSD.org List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 31 May 2012 07:48:04 -0000 --14dae93403ad8e26d204c15049cd Content-Type: text/plain; charset=ISO-8859-1 Hello Martin and all, I'm testing the new patch and it works smoothly, I haven't done a benchmark yet about how long to destroy the pool with async_destroy enabled against not enabled. However, I felt some missing information, as an example on the man page, also the "zpool list -v" wasn't documented. Here attached a very small patch that should be applied after apply the Martin's patch. Best Regards, -- Marcelo Araujo araujo@FreeBSD.org --14dae93403ad8e26d204c15049cd Content-Type: application/octet-stream; name="zfs-features-vs-head-20120531.patch" Content-Disposition: attachment; filename="zfs-features-vs-head-20120531.patch" Content-Transfer-Encoding: base64 X-Attachment-Id: f_h2viuc0s0 LS0tIHpwb29sX21haW4uYwkyMDEyLTA1LTMxIDE1OjMwOjA5LjAwMDAwMDAwMCArMDgwMAorKysg bmV3L3pwb29sX21haW4uYy1uZXcJMjAxMi0wNS0zMSAxNToyODoxNy4wMDAwMDAwMDAgKzA4MDAK QEAgLTIzNSw3ICsyMzUsNyBAQAogCWNhc2UgSEVMUF9MQUJFTENMRUFSOgogCQlyZXR1cm4gKGdl dHRleHQoIlx0bGFiZWxjbGVhciBbLWZdIDx2ZGV2PlxuIikpOwogCWNhc2UgSEVMUF9MSVNUOgot CQlyZXR1cm4gKGdldHRleHQoIlx0bGlzdCBbLUhdIFstbyBwcm9wZXJ0eVssLi4uXV0gIgorCQly ZXR1cm4gKGdldHRleHQoIlx0bGlzdCBbLUh2XSBbLW8gcHJvcGVydHlbLC4uLl1dICIKIAkJICAg ICJbLVQgZHx1XSBbcG9vbF0gLi4uIFtpbnRlcnZhbCBbY291bnRdXVxuIikpOwogCWNhc2UgSEVM UF9PRkZMSU5FOgogCQlyZXR1cm4gKGdldHRleHQoIlx0b2ZmbGluZSBbLXRdIDxwb29sPiA8ZGV2 aWNlPiAuLi5cbiIpKTsKLS0tIHpwb29sLjgJMjAxMi0wNS0yOCAyMDowMjo1MC4wMDAwMDAwMDAg KzA4MDAKKysrIG5ldy96cG9vbC44LW5ldwkyMDEyLTA1LTMxIDE1OjI4OjE3LjAwMDAwMDAwMCAr MDgwMApAQCAtMTM0OCw2ICsxMzQ4LDggQEAKIC5JdCBGbCBICiBTY3JpcHRlZCBtb2RlLiBEbyBu b3QgZGlzcGxheSBoZWFkZXJzLCBhbmQgc2VwYXJhdGUgZmllbGRzIGJ5IGEgc2luZ2xlIHRhYgog aW5zdGVhZCBvZiBhcmJpdHJhcnkgc3BhY2UuCisuSXQgRmwgdgorU2hvdyBtb3JlIGRldGFpbGVk IHBvb2wgaW5mb3JtYXRpb24uIAogLkl0IEZsIG8gQXIgcHJvcGVydHkgTnMgT3AgLCBOcyBBciAu Li4KIENvbW1hLXNlcGFyYXRlZCBsaXN0IG9mIHByb3BlcnRpZXMgdG8gZGlzcGxheS4gU2VlIHRo ZQogLlFxIFN4IFByb3BlcnRpZXMKLS0tIHpwb29sLWZlYXR1cmVzLjUJMjAxMi0wNS0zMSAxNToy OToxMC4wMDAwMDAwMDAgKzA4MDAKKysrIG5ldy96cG9vbC1mZWF0dXJlcy41LW5ldwkyMDEyLTA1 LTMxIDE1OjI4OjE3LjAwMDAwMDAwMCArMDgwMApAQCAtMTU0LDYgKzE1NCwxNiBAQAogYXZhaWxh YmxlIHRocm91Z2ggdGhlCiAuU3kgZnJlZWluZwogcHJvcGVydHkuCisuRWwKKy5FbAorLlNoIEVY QU1QTEVTCisuQmwgLXRhZyAtd2lkdGggMG4KKy5JdCBTeSBFeGFtcGxlIDEgTm8gQWN0aXZhdGlu ZyB0aGUgYXN5bmNfZGVzdHJveSBmZWF0dXJlCisuUHAKK1RoZSBmb2xsb3dpbmcgY29tbWFuZCBh Y3RpdmF0aW5nIHRoZSBhc3luY19kZXN0cm95IGZlYXR1cmUgb24gcG9vbCBjYWxsZWQgdGFuawor LkJkIC1saXRlcmFsIC1vZmZzZXQgMm4KKy5MaSAjIEljIHpwb29sIHNldCBmZWF0dXJlQGFzeW5j X2Rlc3Ryb3k9ZW5hYmxlZCB0YW5rCisuRWQKIC5TaCBTRUUgQUxTTwogLlhyIHpwb29sIDgKIC5T aCBBVVRIT1JTCg== --14dae93403ad8e26d204c15049cd-- From owner-zfs-devel@FreeBSD.ORG Thu May 31 09:13:49 2012 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 14083106564A for ; Thu, 31 May 2012 09:13:49 +0000 (UTC) (envelope-from pawel@dawidek.net) Received: from mail.dawidek.net (60.wheelsystems.com [83.12.187.60]) by mx1.freebsd.org (Postfix) with ESMTP id B6FF78FC08 for ; Thu, 31 May 2012 09:13:48 +0000 (UTC) Received: from localhost (58.wheelsystems.com [83.12.187.58]) by mail.dawidek.net (Postfix) with ESMTPSA id 8EAD9450; Thu, 31 May 2012 11:13:47 +0200 (CEST) Date: Thu, 31 May 2012 11:11:58 +0200 From: Pawel Jakub Dawidek To: Chuck Silvers Message-ID: <20120531091158.GA1401@garage.freebsd.pl> References: <20120531003233.GD11065@yoshi.hcs.int> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="gKMricLos+KVdGMg" Content-Disposition: inline In-Reply-To: <20120531003233.GD11065@yoshi.hcs.int> X-OS: FreeBSD 10.0-CURRENT amd64 User-Agent: Mutt/1.5.21 (2010-09-15) Cc: zfs-devel@freebsd.org Subject: Re: ZFS patches X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 31 May 2012 09:13:49 -0000 --gKMricLos+KVdGMg Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, May 30, 2012 at 05:32:34PM -0700, Chuck Silvers wrote: > we only have a few changes to ZFS itself, and now that I look I see that > you've found one of them independently (r230256). >=20 > the other ones are: >=20 > - improve performance of booting from a ZFS root under ESXi. > previously this would sit there for about 5 minutes before even > starting to load the kernel. the problem is that the ZFS pool-discove= ry > code opens every possible GPT partition looking for pools, and it rere= ads > the GPT each time, one sector at a time. we changed the GPT code to > read the whole GPT in one shot, which reduced the delay to almost noth= ing. > I remember seeing some discussion about a PR on this topic some time b= ack > but I don't know if any fix was ever applied and I don't see the PR no= w. > as I recall, the proposal in that discussion was to improve the boot c= ode > caching so that it wouldn't reread the GPT at all, which I imagine wou= ld > work just as well as what we did. > (hmm, this isn't actually a change to ZFS either.) The problem I remember with this code was that it checked 128 partitions for every single disk in a brute-force fasion in hope that maybe there are no p3-p110 partitions, but maybe there is p111 one. Easy (but not ideal) solution to this was to stop scanning partitions when 3 in a row don't exist. Ideal solution was to teach the code to actually look into partition table and trying only those partitions that really exist. I haven't looked closely at your patch yet, though. > - make zfs_resilver_delay and zfs_resilver_min_time_ms tunable via sysct= l. I have a bigger patch for this too: http://people.freebsd.org/~pjd/patches/dsl_scan.c.patch The reason I didn't commit this patch yet is that some of those values can be safely modified after boot can could be made CTLFLAG_RW like maybe the two you convered. If you would like to analyse them and see which are safe to be made RW that would be great. --=20 Pawel Jakub Dawidek http://www.wheelsystems.com FreeBSD committer http://www.FreeBSD.org Am I Evil? Yes, I Am! http://tupytaj.pl --gKMricLos+KVdGMg Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (FreeBSD) iEYEARECAAYFAk/HNd4ACgkQForvXbEpPzSiLQCfZDRqw/DbcBItRmu3bAun+sjm tg0AoKAncAc6pftmUA5m+ktFBppNXrF0 =d0e3 -----END PGP SIGNATURE----- --gKMricLos+KVdGMg-- From owner-zfs-devel@FreeBSD.ORG Thu Jun 7 15:27:01 2012 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E6C04106566B for ; Thu, 7 Jun 2012 15:27:01 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from mail.vx.sk (mail.vx.sk [176.9.45.25]) by mx1.freebsd.org (Postfix) with ESMTP id A3E7D8FC12 for ; Thu, 7 Jun 2012 15:27:01 +0000 (UTC) Received: from core.vx.sk (localhost [127.0.0.2]) by mail.vx.sk (Postfix) with ESMTP id 57F661D97D; Thu, 7 Jun 2012 17:26:55 +0200 (CEST) X-Virus-Scanned: amavisd-new at mail.vx.sk Received: from mail.vx.sk by core.vx.sk (amavisd-new, unix socket) with LMTP id A8svxPvWXuzc; Thu, 7 Jun 2012 17:26:50 +0200 (CEST) Received: from [10.9.8.1] (188-167-78-15.dynamic.chello.sk [188.167.78.15]) by mail.vx.sk (Postfix) with ESMTPSA id E5FAF1D96D; Thu, 7 Jun 2012 17:26:49 +0200 (CEST) Message-ID: <4FD0C839.9020203@FreeBSD.org> Date: Thu, 07 Jun 2012 17:26:49 +0200 From: Martin Matuska User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:12.0) Gecko/20120428 Thunderbird/12.0.1 MIME-Version: 1.0 To: zfs-devel@FreeBSD.org X-Enigmail-Version: 1.4.2 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Cc: Subject: [CFR] ZFS features support X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 Jun 2012 15:27:02 -0000 I have now completed boot support for ZFS features. Please review and comment my patch: http://people.freebsd.org/~mm/patches/zfs/features/head-zfs-features.patch Boot-only patch: http://people.freebsd.org/~mm/patches/zfs/features/head-zfs-features-boot.patch I have also an alternate boot patch that adds functions like nvlist_name(), nvlist_value(), but the previous one has a smaller footprint: http://people.freebsd.org/~mm/patches/zfs/features/head-zfs-features-boot.fc.patch The boot code can be tested with the "zhack" tool, e.g. with: zhack feature add tank org.freebsd:test zhack feature ref -m tank org.freebsd:test Now you are unable to boot. zhack feature ref -d tank org.freebsd:test Now you are able to boot. If there are no quick objections, I will commit this to -HEAD with a 1-month MFC. Cheers, mm -- Martin Matuska FreeBSD committer http://blog.vx.sk From owner-zfs-devel@FreeBSD.ORG Thu Jun 7 19:32:35 2012 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D5B86106564A for ; Thu, 7 Jun 2012 19:32:35 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: from mail-gh0-f182.google.com (mail-gh0-f182.google.com [209.85.160.182]) by mx1.freebsd.org (Postfix) with ESMTP id 84B768FC17 for ; Thu, 7 Jun 2012 19:32:35 +0000 (UTC) Received: by ghbz22 with SMTP id z22so822723ghb.13 for ; Thu, 07 Jun 2012 12:32:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=delphix.com; s=google; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=G38sg0dD7dGFUnu2BCKuI3Abg7KFYvGHo5cfUnEReYg=; b=N0axwq5QVYmtApv9JsrZnPv1eoIqzQUWFhBDr54h+oyahnV7TWIFwv23FclKLhu18D Ri9lbQg62iMyXIkj+cm5JFmeri6jQjDlMkVsLgn6CstjR8lxOeplPtplkg9yvEis7OGh dwmzHHmG5lhg6qI9NY3jff51UkBqGNyAU5T4o= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:x-gm-message-state; bh=G38sg0dD7dGFUnu2BCKuI3Abg7KFYvGHo5cfUnEReYg=; b=Rs97n8W3vd8iSrWrVzu5omkHG4Q3Yv2uq8upSvLHeiitgf0VYh6p4sD+X8Y/p09brs KQ7gevcB+63ns2ATiJLJTxETrfDsjgym7lRL4kB/8lW+066ARyfHqQUiD4FciDZxee/Z UatyMfXSYU1Mlh6ij02oGnRbAIJoS4ckBOIvhOkqMBkKzesv2tsmLCgFN4P2i6Nw0/SM ejP9FZPqSEArKVjFdu2gGGvR7L+7Rhb98K7T0E/RrCSkNlZB7ZglIlnKKdhtqtpuWO6A iZ5d6183ejwfMVfFn7ue1Xjobt9r5cpReT6xtf0Th/nEB4b3+UJm3grihtFkRzxHsztM DsTA== MIME-Version: 1.0 Received: by 10.50.196.232 with SMTP id ip8mr2004947igc.50.1339097554614; Thu, 07 Jun 2012 12:32:34 -0700 (PDT) Received: by 10.231.237.78 with HTTP; Thu, 7 Jun 2012 12:32:34 -0700 (PDT) In-Reply-To: <4FD0C839.9020203@FreeBSD.org> References: <4FD0C839.9020203@FreeBSD.org> Date: Thu, 7 Jun 2012 12:32:34 -0700 Message-ID: From: Matthew Ahrens To: Martin Matuska X-Gm-Message-State: ALoCoQnI6iGOH6YCdl8AxxCbwflnXhcm7QrUNgSMWQe/TrwrhYBQxgd4IppPynWGXZ/rcla4K3U9 Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: zfs-devel@freebsd.org, Christopher Siden Subject: Re: [CFR] ZFS features support X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 Jun 2012 19:32:35 -0000 Martin, I haven't reviewed the code but I think it's great you're doing this. Thanks! I hope that feature flags will be useful in FreeBSD. Let me or Chris know if you have any issues using it to develop new features. --matt On Thu, Jun 7, 2012 at 8:26 AM, Martin Matuska wrote: > I have now completed boot support for ZFS features. > > Please review and comment my patch: > http://people.freebsd.org/~mm/patches/zfs/features/head-zfs-features.patch > > Boot-only patch: > > http://people.freebsd.org/~mm/patches/zfs/features/head-zfs-features-boot.patch > > I have also an alternate boot patch that adds functions like > nvlist_name(), nvlist_value(), but the previous one has a smaller > footprint: > > http://people.freebsd.org/~mm/patches/zfs/features/head-zfs-features-boot.fc.patch > > The boot code can be tested with the "zhack" tool, e.g. with: > zhack feature add tank org.freebsd:test > zhack feature ref -m tank org.freebsd:test > Now you are unable to boot. > zhack feature ref -d tank org.freebsd:test > Now you are able to boot. > > If there are no quick objections, I will commit this to -HEAD with a > 1-month MFC. > > Cheers, > mm > > -- > Martin Matuska > FreeBSD committer > http://blog.vx.sk > > _______________________________________________ > zfs-devel@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/zfs-devel > To unsubscribe, send any mail to "zfs-devel-unsubscribe@freebsd.org" > From owner-zfs-devel@FreeBSD.ORG Sun Jun 10 13:20:41 2012 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 30D19106564A; Sun, 10 Jun 2012 13:20:41 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 4D4028FC08; Sun, 10 Jun 2012 13:20:40 +0000 (UTC) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id QAA19349; Sun, 10 Jun 2012 16:20:38 +0300 (EEST) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1Sdi4M-000097-Jz; Sun, 10 Jun 2012 16:20:38 +0300 Message-ID: <4FD49F25.7070909@FreeBSD.org> Date: Sun, 10 Jun 2012 16:20:37 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:12.0) Gecko/20120503 Thunderbird/12.0.1 MIME-Version: 1.0 To: Martin Matuska References: <4FD0C839.9020203@FreeBSD.org> In-Reply-To: <4FD0C839.9020203@FreeBSD.org> X-Enigmail-Version: 1.5pre Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Cc: zfs-devel@FreeBSD.org Subject: Re: [CFR] ZFS features support X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 10 Jun 2012 13:20:41 -0000 on 07/06/2012 18:26 Martin Matuska said the following: > I have now completed boot support for ZFS features. > > Please review and comment my patch: > http://people.freebsd.org/~mm/patches/zfs/features/head-zfs-features.patch > > Boot-only patch: > http://people.freebsd.org/~mm/patches/zfs/features/head-zfs-features-boot.patch The boot part looks OK to me. Thank you for working on this! > I have also an alternate boot patch that adds functions like > nvlist_name(), nvlist_value(), but the previous one has a smaller footprint: > http://people.freebsd.org/~mm/patches/zfs/features/head-zfs-features-boot.fc.patch > > The boot code can be tested with the "zhack" tool, e.g. with: > zhack feature add tank org.freebsd:test > zhack feature ref -m tank org.freebsd:test > Now you are unable to boot. > zhack feature ref -d tank org.freebsd:test > Now you are able to boot. > > If there are no quick objections, I will commit this to -HEAD with a > 1-month MFC. -- Andriy Gapon From owner-zfs-devel@FreeBSD.ORG Mon Jun 11 23:30:31 2012 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id AAB941065674; Mon, 11 Jun 2012 23:30:31 +0000 (UTC) (envelope-from chs@highcloudsecurity.com) Received: from mail.highcloudsecurity.com (50-0-150-29.dedicated.static.sonic.net [50.0.150.29]) by mx1.freebsd.org (Postfix) with ESMTP id 8553D8FC1B; Mon, 11 Jun 2012 23:30:31 +0000 (UTC) Received: from localhost (localhost.localdomain [127.0.0.1]) by mail.highcloudsecurity.com (Postfix) with ESMTP id C0ACC321081; Tue, 12 Jun 2012 00:22:16 +0100 (BST) X-Virus-Scanned: amavisd-new at highcloudsecurity.com Received: from mail.highcloudsecurity.com ([127.0.0.1]) by localhost (mail.highcloudsecurity.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id vuB-HrZPu1QK; Tue, 12 Jun 2012 00:22:16 +0100 (BST) Received: from yoshi.hcs.int (unknown [10.238.16.254]) by mail.highcloudsecurity.com (Postfix) with ESMTPSA id 2E2F332107E; Tue, 12 Jun 2012 00:22:16 +0100 (BST) Date: Mon, 11 Jun 2012 16:22:14 -0700 From: Chuck Silvers To: Pawel Jakub Dawidek Message-ID: <20120611232214.GM11065@yoshi.hcs.int> References: <20120531003233.GD11065@yoshi.hcs.int> <20120531091158.GA1401@garage.freebsd.pl> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20120531091158.GA1401@garage.freebsd.pl> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: zfs-devel@freebsd.org Subject: Re: ZFS patches X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 Jun 2012 23:30:31 -0000 On Thu, May 31, 2012 at 11:11:58AM +0200, Pawel Jakub Dawidek wrote: > On Wed, May 30, 2012 at 05:32:34PM -0700, Chuck Silvers wrote: > > we only have a few changes to ZFS itself, and now that I look I see that > > you've found one of them independently (r230256). > > > > the other ones are: > > > > - improve performance of booting from a ZFS root under ESXi. > > previously this would sit there for about 5 minutes before even > > starting to load the kernel. the problem is that the ZFS pool-discovery > > code opens every possible GPT partition looking for pools, and it rereads > > the GPT each time, one sector at a time. we changed the GPT code to > > read the whole GPT in one shot, which reduced the delay to almost nothing. > > I remember seeing some discussion about a PR on this topic some time back > > but I don't know if any fix was ever applied and I don't see the PR now. > > as I recall, the proposal in that discussion was to improve the boot code > > caching so that it wouldn't reread the GPT at all, which I imagine would > > work just as well as what we did. > > (hmm, this isn't actually a change to ZFS either.) > > The problem I remember with this code was that it checked 128 partitions > for every single disk in a brute-force fasion in hope that maybe there > are no p3-p110 partitions, but maybe there is p111 one. > > Easy (but not ideal) solution to this was to stop scanning partitions > when 3 in a row don't exist. Ideal solution was to teach the code to > actually look into partition table and trying only those partitions that > really exist. yea, reading the GPT once and iterating over the valid partitions would be good too. but if there are multiple valid partitions, there's no need to go back and re-read the GPT again for each one. and even when you read the GPT the first time, it would best to read it all at once instead of 1 sector at a time. so really all of these are addressing different aspects of this issue. in practice, fixing any one of these aspects is probably enough to make this not be a problem. > I haven't looked closely at your patch yet, though. did you get a chance to look at it yet? or do you think people are going to want to hold out for a different subset of the complete set of potential changes above? > > - make zfs_resilver_delay and zfs_resilver_min_time_ms tunable via sysctl. > > I have a bigger patch for this too: > > http://people.freebsd.org/~pjd/patches/dsl_scan.c.patch > > The reason I didn't commit this patch yet is that some of those values > can be safely modified after boot can could be made CTLFLAG_RW like > maybe the two you convered. > > If you would like to analyse them and see which are safe to be made RW > that would be great. I looked through these and all of them looked safe to me to be changed on the fly. the values just don't seem to used in any kind of persistent fashion. -Chuck From owner-zfs-devel@FreeBSD.ORG Tue Jun 12 03:32:16 2012 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 93270106566B; Tue, 12 Jun 2012 03:32:16 +0000 (UTC) (envelope-from chs@highcloudsecurity.com) Received: from mail.highcloudsecurity.com (50-0-150-29.dedicated.static.sonic.net [50.0.150.29]) by mx1.freebsd.org (Postfix) with ESMTP id 707C88FC0C; Tue, 12 Jun 2012 03:32:16 +0000 (UTC) Received: from localhost (localhost.localdomain [127.0.0.1]) by mail.highcloudsecurity.com (Postfix) with ESMTP id E0D89110002; Tue, 12 Jun 2012 04:32:15 +0100 (BST) X-Virus-Scanned: amavisd-new at highcloudsecurity.com Received: from mail.highcloudsecurity.com ([127.0.0.1]) by localhost (mail.highcloudsecurity.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 8dNKXvA-H-oG; Tue, 12 Jun 2012 04:32:14 +0100 (BST) Received: from yoshi.hcs.int (unknown [10.238.16.254]) by mail.highcloudsecurity.com (Postfix) with ESMTPSA id DB40032107E; Tue, 12 Jun 2012 04:32:14 +0100 (BST) Date: Mon, 11 Jun 2012 20:32:12 -0700 From: Chuck Silvers To: "Justin T. Gibbs" Message-ID: <20120612033212.GO11065@yoshi.hcs.int> References: <20120523201353.GC1399@garage.freebsd.pl> <20120529184207.GB11065@yoshi.hcs.int> <8DA92D89-BC0E-450F-A4D8-ED64CACD3471@scsiguy.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <8DA92D89-BC0E-450F-A4D8-ED64CACD3471@scsiguy.com> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: zfs-devel@freebsd.org Subject: Re: introducing chuck silvers X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Jun 2012 03:32:16 -0000 On Tue, May 29, 2012 at 02:38:24PM -0600, Justin T. Gibbs wrote: > PRs often (unfortunately) get lost. If you can provide an enumeration of your > changes, I can introduce you to a committer that works in that area to help get > the changes into the tree. ... and some more changes from the last couple of weeks: - support for larger kernel RPC sizes (ie. for the NFS server). this probably needs more work but it would be useful to discuss it with someone. - improve kernel RPC worker thread efficiency by only waking up a thread when there's at least 1 complete RPC waiting in the socket buffer. - enhance the open-vm-tools vmxnet driver to use more rx buffers if the hypervisor supports it, so that it's much less likely to drop packets. it looks like there's still a lot more that could be done to improve this driver, eg. TSO/LRO, checksum offloading, using the large-packet rx ring, etc. it might be better to skip all this and just port the vmxnet3 driver from linux, but that seems likely to be a larger undertaking. I'll guess that rick macklem would be the person to talk to about the NFS/RPC stuff, is that right? -Chuck From owner-zfs-devel@FreeBSD.ORG Tue Jun 12 04:19:30 2012 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C2798106566C for ; Tue, 12 Jun 2012 04:19:30 +0000 (UTC) (envelope-from chs@highcloudsecurity.com) Received: from mail.highcloudsecurity.com (50-0-150-29.dedicated.static.sonic.net [50.0.150.29]) by mx1.freebsd.org (Postfix) with ESMTP id 9F8078FC0C for ; Tue, 12 Jun 2012 04:19:30 +0000 (UTC) Received: from localhost (localhost.localdomain [127.0.0.1]) by mail.highcloudsecurity.com (Postfix) with ESMTP id 7DD20321081; Tue, 12 Jun 2012 05:19:30 +0100 (BST) X-Virus-Scanned: amavisd-new at highcloudsecurity.com Received: from mail.highcloudsecurity.com ([127.0.0.1]) by localhost (mail.highcloudsecurity.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id tifha0MtQZPw; Tue, 12 Jun 2012 05:19:29 +0100 (BST) Received: from yoshi.hcs.int (unknown [10.238.16.254]) by mail.highcloudsecurity.com (Postfix) with ESMTPSA id C972532107E; Tue, 12 Jun 2012 05:19:29 +0100 (BST) Date: Mon, 11 Jun 2012 21:19:28 -0700 From: Chuck Silvers To: "Justin T. Gibbs" Message-ID: <20120612041928.GP11065@yoshi.hcs.int> References: <20120523201353.GC1399@garage.freebsd.pl> <20120529184207.GB11065@yoshi.hcs.int> <8DA92D89-BC0E-450F-A4D8-ED64CACD3471@scsiguy.com> <20120612033212.GO11065@yoshi.hcs.int> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20120612033212.GO11065@yoshi.hcs.int> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: zfs-devel@freebsd.org Subject: Re: introducing chuck silvers X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Jun 2012 04:19:30 -0000 On Mon, Jun 11, 2012 at 08:32:12PM -0700, Chuck Silvers wrote: > On Tue, May 29, 2012 at 02:38:24PM -0600, Justin T. Gibbs wrote: > > PRs often (unfortunately) get lost. If you can provide an enumeration of your > > changes, I can introduce you to a committer that works in that area to help get > > the changes into the tree. oh, and a couple more: - the dtrace profile and lockstat providers both strip off too many stack frames from the stack() data on amd64, which makes it harder to see what's going on. I'm not sure if this is always the case or if there are other interrupt paths with different stack behaviour. - sys/lockstat.h only defines the lockstat hooks as non-empty if KDTRACE_HOOKS is enabled, but nothing ensures that opt_kdtrace.h was previously included, and most of the .c files that include the various lock headers that include sys/lockstat.h don't previously include opt_kdtrace.h. thus, the inline versions of the various locks that are used in drivers are built into the base kernel mostly miss out on the lockstat hooks. -Chuck From owner-zfs-devel@FreeBSD.ORG Sun Jul 1 15:48:20 2012 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 157F8106564A; Sun, 1 Jul 2012 15:48:20 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from mail.vx.sk (mail.vx.sk [IPv6:2a01:4f8:150:6101::4]) by mx1.freebsd.org (Postfix) with ESMTP id 88A978FC0C; Sun, 1 Jul 2012 15:48:19 +0000 (UTC) Received: from core.vx.sk (localhost [127.0.0.2]) by mail.vx.sk (Postfix) with ESMTP id B7AA827BF7; Sun, 1 Jul 2012 17:48:18 +0200 (CEST) X-Virus-Scanned: amavisd-new at mail.vx.sk Received: from mail.vx.sk by core.vx.sk (amavisd-new, unix socket) with LMTP id yO_5TzjTbpnx; Sun, 1 Jul 2012 17:48:16 +0200 (CEST) Received: from [10.9.8.1] (188-167-78-15.dynamic.chello.sk [188.167.78.15]) by mail.vx.sk (Postfix) with ESMTPSA id 4E87627BE4; Sun, 1 Jul 2012 17:48:16 +0200 (CEST) Message-ID: <4FF07140.9020401@FreeBSD.org> Date: Sun, 01 Jul 2012 17:48:16 +0200 From: Martin Matuska User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:13.0) Gecko/20120614 Thunderbird/13.0.1 MIME-Version: 1.0 To: Pawel Jakub Dawidek X-Enigmail-Version: 1.4.2 Content-Type: multipart/mixed; boundary="------------090509050607070809000505" Cc: zfs-devel@FreeBSD.org Subject: [CFR] Tunables for scrub and resilver X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 01 Jul 2012 15:48:20 -0000 This is a multi-part message in MIME format. --------------090509050607070809000505 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Hi, I would like to hear your opinion on the attached patch to add scrub and resilver tunables. This way users can add more priority to scrub and resilver (make it faster) at cost of other I/O etc. On-line version of the patch: http://people.freebsd.org/~mm/patches/zfs/dsl_scan.patch The patch adds tuning for all of the dsl_scan.c tunables, as available in illumos. zfs_resilver_delay and zfs_scrub_delay (resulting in scan_delay) need to be non-negative, otherwise we trigger a kernel assert in pause(). Other values are used for timer comparsions and should be safe even if negative (resulting behavior equals a value of zero). Thanks, mm -- Martin Matuska FreeBSD committer http://blog.vx.sk --------------090509050607070809000505 Content-Type: text/plain; charset=windows-1250; name="dsl_scan.patch" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="dsl_scan.patch" SW5kZXg6IHN5cy9jZGRsL2NvbnRyaWIvb3BlbnNvbGFyaXMvdXRzL2NvbW1vbi9mcy96ZnMv ZHNsX3NjYW4uYwo9PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09Ci0tLSBzeXMvY2RkbC9jb250cmliL29wZW5zb2xh cmlzL3V0cy9jb21tb24vZnMvemZzL2RzbF9zY2FuLmMJKHJldmlzaW9uIDIzNzc0NSkKKysr IHN5cy9jZGRsL2NvbnRyaWIvb3BlbnNvbGFyaXMvdXRzL2NvbW1vbi9mcy96ZnMvZHNsX3Nj YW4uYwkod29ya2luZyBjb3B5KQpAQCAtNjgsNiArNjgsMzYgQEAKIGludCB6ZnNfcmVzaWx2 ZXJfbWluX3RpbWVfbXMgPSAzMDAwOyAvKiBtaW4gbWlsbGlzZWNzIHRvIHJlc2lsdmVyIHBl ciB0eGcgKi8KIGJvb2xlYW5fdCB6ZnNfbm9fc2NydWJfaW8gPSBCX0ZBTFNFOyAvKiBzZXQg dG8gZGlzYWJsZSBzY3J1YiBpL28gKi8KIGJvb2xlYW5fdCB6ZnNfbm9fc2NydWJfcHJlZmV0 Y2ggPSBCX0ZBTFNFOyAvKiBzZXQgdG8gZGlzYWJsZSBzcnViIHByZWZldGNoaW5nICovCisK K1NZU0NUTF9ERUNMKF92ZnNfemZzKTsKK1RVTkFCTEVfSU5UKCJ2ZnMuemZzLnRvcF9tYXhp bmZsaWdodCIsICZ6ZnNfdG9wX21heGluZmxpZ2h0KTsKK1NZU0NUTF9JTlQoX3Zmc196ZnMs IE9JRF9BVVRPLCB0b3BfbWF4aW5mbGlnaHQsIENUTEZMQUdfUlcsCisJJnpmc190b3BfbWF4 aW5mbGlnaHQsIDAsICJNYXhpbXVtIEkvT3MgcGVyIHRvcC1sZXZlbCB2ZGV2Iik7CitUVU5B QkxFX0lOVCgidmZzLnpmcy5yZXNpbHZlcl9kZWxheSIsICZ6ZnNfcmVzaWx2ZXJfZGVsYXkp OworU1lTQ1RMX0lOVChfdmZzX3pmcywgT0lEX0FVVE8sIHJlc2lsdmVyX2RlbGF5LCBDVExG TEFHX1JXLAorICAgICZ6ZnNfcmVzaWx2ZXJfZGVsYXksIDAsICJOdW1iZXIgb2YgdGlja3Mg dG8gZGVsYXkgcmVzaWx2ZXIiKTsKK1RVTkFCTEVfSU5UKCJ2ZnMuemZzLnNjcnViX2RlbGF5 IiwgJnpmc19zY3J1Yl9kZWxheSk7CitTWVNDVExfSU5UKF92ZnNfemZzLCBPSURfQVVUTywg c2NydWJfZGVsYXksIENUTEZMQUdfUlcsCisgICAgJnpmc19zY3J1Yl9kZWxheSwgMCwgIk51 bWJlciBvZiB0aWNrcyB0byBkZWxheSBzY3J1YiIpOworVFVOQUJMRV9JTlQoInZmcy56ZnMu c2Nhbl9pZGxlIiwgJnpmc19zY2FuX2lkbGUpOworU1lTQ1RMX0lOVChfdmZzX3pmcywgT0lE X0FVVE8sIHNjYW5faWRsZSwgQ1RMRkxBR19SVywKKyAgICAmemZzX3NjYW5faWRsZSwgMCwg IklkbGUgc2NhbiB3aW5kb3cgaW4gY2xvY2sgdGlja3MiKTsKK1RVTkFCTEVfSU5UKCJ2ZnMu emZzLnNjYW5fbWluX3RpbWVfbXMiLCAmemZzX3NjYW5fbWluX3RpbWVfbXMpOworU1lTQ1RM X0lOVChfdmZzX3pmcywgT0lEX0FVVE8sIHNjYW5fbWluX3RpbWVfbXMsIENUTEZMQUdfUlcs CisgICAgJnpmc19zY2FuX21pbl90aW1lX21zLCAwLCAiTWluIG1pbGxpc2VjcyB0byBzY3J1 YiBwZXIgdHhnIik7CitUVU5BQkxFX0lOVCgidmZzLnpmcy5mcmVlX21pbl90aW1lX21zIiwg Jnpmc19mcmVlX21pbl90aW1lX21zKTsKK1NZU0NUTF9JTlQoX3Zmc196ZnMsIE9JRF9BVVRP LCBmcmVlX21pbl90aW1lX21zLCBDVExGTEFHX1JXLAorICAgICZ6ZnNfZnJlZV9taW5fdGlt ZV9tcywgMCwgIk1pbiBtaWxsaXNlY3MgdG8gZnJlZSBwZXIgdHhnIik7CitUVU5BQkxFX0lO VCgidmZzLnpmcy5yZXNpbHZlcl9taW5fdGltZV9tcyIsICZ6ZnNfcmVzaWx2ZXJfbWluX3Rp bWVfbXMpOworU1lTQ1RMX0lOVChfdmZzX3pmcywgT0lEX0FVVE8sIHJlc2lsdmVyX21pbl90 aW1lX21zLCBDVExGTEFHX1JXLAorICAgICZ6ZnNfcmVzaWx2ZXJfbWluX3RpbWVfbXMsIDAs ICJNaW4gbWlsbGlzZWNzIHRvIHJlc2lsdmVyIHBlciB0eGciKTsKK1RVTkFCTEVfSU5UKCJ2 ZnMuemZzLm5vX3NjcnViX2lvIiwgJnpmc19ub19zY3J1Yl9pbyk7CitTWVNDVExfSU5UKF92 ZnNfemZzLCBPSURfQVVUTywgbm9fc2NydWJfaW8sIENUTEZMQUdfUlcsCisgICAgJnpmc19u b19zY3J1Yl9pbywgMCwgIkRpc2FibGUgc2NydWIgSS9PIik7CitUVU5BQkxFX0lOVCgidmZz Lnpmcy5ub19zY3J1Yl9wcmVmZXRjaCIsICZ6ZnNfbm9fc2NydWJfcHJlZmV0Y2gpOworU1lT Q1RMX0lOVChfdmZzX3pmcywgT0lEX0FVVE8sIG5vX3NjcnViX3ByZWZldGNoLCBDVExGTEFH X1JXLAorICAgICZ6ZnNfbm9fc2NydWJfcHJlZmV0Y2gsIDAsICJEaXNhYmxlIHNjcnViIHBy ZWZldGNoaW5nIik7CisKIGVudW0gZGR0X2NsYXNzIHpmc19zY3J1Yl9kZHRfY2xhc3NfbWF4 ID0gRERUX0NMQVNTX0RVUExJQ0FURTsKIAogI2RlZmluZQlEU0xfU0NBTl9JU19TQ1JVQl9S RVNJTFZFUihzY24pIFwKQEAgLTE3MDksNyArMTczOSw3IEBACiAJCSAqIHRoZW4gdGhyb3R0 bGUgb3VyIHdvcmtsb2FkIHRvIGxpbWl0IHRoZSBpbXBhY3Qgb2YgYSBzY2FuLgogCQkgKi8K IAkJaWYgKGRkaV9nZXRfbGJvbHQ2NCgpIC0gc3BhLT5zcGFfbGFzdF9pbyA8PSB6ZnNfc2Nh bl9pZGxlKQotCQkJZGVsYXkoc2Nhbl9kZWxheSk7CisJCQlkZWxheShNQVgoc2Nhbl9kZWxh eSwwKSk7CiAKIAkJemlvX25vd2FpdCh6aW9fcmVhZChOVUxMLCBzcGEsIGJwLCBkYXRhLCBz aXplLAogCQkgICAgZHNsX3NjYW5fc2NydWJfZG9uZSwgTlVMTCwgemlvX3ByaW9yaXR5LAo= --------------090509050607070809000505-- From owner-zfs-devel@FreeBSD.ORG Sun Jul 1 16:07:08 2012 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 7D055106564A; Sun, 1 Jul 2012 16:07:08 +0000 (UTC) (envelope-from pawel@dawidek.net) Received: from mail.dawidek.net (60.wheelsystems.com [83.12.187.60]) by mx1.freebsd.org (Postfix) with ESMTP id 2D7BD8FC0A; Sun, 1 Jul 2012 16:07:08 +0000 (UTC) Received: from localhost (89-73-195-149.dynamic.chello.pl [89.73.195.149]) by mail.dawidek.net (Postfix) with ESMTPSA id BC6A0F42; Sun, 1 Jul 2012 18:07:06 +0200 (CEST) Date: Sun, 1 Jul 2012 18:04:59 +0200 From: Pawel Jakub Dawidek To: Martin Matuska Message-ID: <20120701160458.GQ1400@garage.freebsd.pl> References: <4FF07140.9020401@FreeBSD.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="KD3NH8oGZ7XN2Llp" Content-Disposition: inline In-Reply-To: <4FF07140.9020401@FreeBSD.org> X-OS: FreeBSD 10.0-CURRENT amd64 User-Agent: Mutt/1.5.21 (2010-09-15) Cc: zfs-devel@FreeBSD.org Subject: Re: [CFR] Tunables for scrub and resilver X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 01 Jul 2012 16:07:08 -0000 --KD3NH8oGZ7XN2Llp Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sun, Jul 01, 2012 at 05:48:16PM +0200, Martin Matuska wrote: > Hi, >=20 > I would like to hear your opinion on the attached patch to add scrub and > resilver tunables. > This way users can add more priority to scrub and resilver (make it > faster) at cost of other I/O etc. >=20 > On-line version of the patch: > http://people.freebsd.org/~mm/patches/zfs/dsl_scan.patch >=20 > The patch adds tuning for all of the dsl_scan.c tunables, as available > in illumos. > zfs_resilver_delay and zfs_scrub_delay (resulting in scan_delay) need to > be non-negative, otherwise we trigger a kernel assert in pause(). > Other values are used for timer comparsions and should be safe even if > negative (resulting behavior equals a value of zero). I had similar patch for some time now: http://people.freebsd.org/~pjd/patches/dsl_scan.c.patch The only reason I haven't committed it was that I wasn't sure with variables can be safely modified at run-time. If you did the audit and you are sure we can make them RW, then I'm fine with the patch (except for style issues mentioned below). > +SYSCTL_INT(_vfs_zfs, OID_AUTO, top_maxinflight, CTLFLAG_RW, > + &zfs_top_maxinflight, 0, "Maximum I/Os per top-level vdev"); Should be four spaces instead of tab. > - delay(scan_delay); > + delay(MAX(scan_delay,0)); Missing space before comma. Although maybe we should make it unsigned and use SYSCTL_UINT()? --=20 Pawel Jakub Dawidek http://www.wheelsystems.com FreeBSD committer http://www.FreeBSD.org Am I Evil? Yes, I Am! http://tupytaj.pl --KD3NH8oGZ7XN2Llp Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (FreeBSD) iEYEARECAAYFAk/wdSoACgkQForvXbEpPzQL3QCfdbWFMJ8RwNan+rPTJZpRSNWs O20AoMbaBuZXn5o3dSeJtm9moG/GMd44 =1wgh -----END PGP SIGNATURE----- --KD3NH8oGZ7XN2Llp-- From owner-zfs-devel@FreeBSD.ORG Mon Jul 2 06:48:37 2012 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CAA2D1065786; Mon, 2 Jul 2012 06:48:37 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from mail.vx.sk (mail.vx.sk [IPv6:2a01:4f8:150:6101::4]) by mx1.freebsd.org (Postfix) with ESMTP id 4C5608FC0A; Mon, 2 Jul 2012 06:48:37 +0000 (UTC) Received: from core.vx.sk (localhost [127.0.0.2]) by mail.vx.sk (Postfix) with ESMTP id A2CC4287AD; Mon, 2 Jul 2012 08:48:36 +0200 (CEST) X-Virus-Scanned: amavisd-new at mail.vx.sk Received: from mail.vx.sk by core.vx.sk (amavisd-new, unix socket) with LMTP id GzCoSWtijn0u; Mon, 2 Jul 2012 08:48:34 +0200 (CEST) Received: from [10.9.8.1] (188-167-78-15.dynamic.chello.sk [188.167.78.15]) by mail.vx.sk (Postfix) with ESMTPSA id 335452879F; Mon, 2 Jul 2012 08:48:34 +0200 (CEST) Message-ID: <4FF14442.5080905@FreeBSD.org> Date: Mon, 02 Jul 2012 08:48:34 +0200 From: Martin Matuska User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:13.0) Gecko/20120614 Thunderbird/13.0.1 MIME-Version: 1.0 To: Pawel Jakub Dawidek References: <4FF07140.9020401@FreeBSD.org> <20120701160458.GQ1400@garage.freebsd.pl> In-Reply-To: <20120701160458.GQ1400@garage.freebsd.pl> X-Enigmail-Version: 1.4.2 Content-Type: multipart/mixed; boundary="------------000308000506080402090900" Cc: zfs-devel@FreeBSD.org Subject: Re: [CFR] Tunables for scrub and resilver X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Jul 2012 06:48:37 -0000 This is a multi-part message in MIME format. --------------000308000506080402090900 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit On 1.7.2012 18:04, Pawel Jakub Dawidek wrote: > On Sun, Jul 01, 2012 at 05:48:16PM +0200, Martin Matuska wrote: >> Hi, >> >> I would like to hear your opinion on the attached patch to add scrub and >> resilver tunables. >> This way users can add more priority to scrub and resilver (make it >> faster) at cost of other I/O etc. >> >> On-line version of the patch: >> http://people.freebsd.org/~mm/patches/zfs/dsl_scan.patch >> >> The patch adds tuning for all of the dsl_scan.c tunables, as available >> in illumos. >> zfs_resilver_delay and zfs_scrub_delay (resulting in scan_delay) need to >> be non-negative, otherwise we trigger a kernel assert in pause(). >> Other values are used for timer comparsions and should be safe even if >> negative (resulting behavior equals a value of zero). > I had similar patch for some time now: > > http://people.freebsd.org/~pjd/patches/dsl_scan.c.patch > > The only reason I haven't committed it was that I wasn't sure with > variables can be safely modified at run-time. If you did the audit and > you are sure we can make them RW, then I'm fine with the patch (except > for style issues mentioned below). > >> +SYSCTL_INT(_vfs_zfs, OID_AUTO, top_maxinflight, CTLFLAG_RW, >> + &zfs_top_maxinflight, 0, "Maximum I/Os per top-level vdev"); > Should be four spaces instead of tab. > >> - delay(scan_delay); >> + delay(MAX(scan_delay,0)); > Missing space before comma. > > Although maybe we should make it unsigned and use SYSCTL_UINT()? > I updated my patch to use unsigned integers, as negative values make here really no sense. >From my practical tests, zfs_top_maxinflight should never be zero (or negative), as this makes the zpool command hang (scrub runs in terms of bytes/sec). Changing all other variables to any values on run-time is fine. Please check the updated patch: http://people.freebsd.org/~mm/patches/zfs/dsl_scan.patch -- Martin Matuska FreeBSD committer http://blog.vx.sk --------------000308000506080402090900 Content-Type: text/plain; charset=windows-1250; name="dsl_scan.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="dsl_scan.patch" Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dsl_scan.c =================================================================== --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dsl_scan.c (revision 237745) +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dsl_scan.c (working copy) @@ -58,16 +58,47 @@ static dsl_syncfunc_t dsl_scan_cancel_sync; static void dsl_scan_sync_state(dsl_scan_t *, dmu_tx_t *tx); -int zfs_top_maxinflight = 32; /* maximum I/Os per top-level */ -int zfs_resilver_delay = 2; /* number of ticks to delay resilver */ -int zfs_scrub_delay = 4; /* number of ticks to delay scrub */ -int zfs_scan_idle = 50; /* idle window in clock ticks */ +unsigned int zfs_top_maxinflight = 32; /* maximum I/Os per top-level */ +unsigned int zfs_resilver_delay = 2; /* number of ticks to delay resilver */ +unsigned int zfs_scrub_delay = 4; /* number of ticks to delay scrub */ +unsigned int zfs_scan_idle = 50; /* idle window in clock ticks */ -int zfs_scan_min_time_ms = 1000; /* min millisecs to scrub per txg */ -int zfs_free_min_time_ms = 1000; /* min millisecs to free per txg */ -int zfs_resilver_min_time_ms = 3000; /* min millisecs to resilver per txg */ +unsigned int zfs_scan_min_time_ms = 1000; /* min millisecs to scrub per txg */ +unsigned int zfs_free_min_time_ms = 1000; /* min millisecs to free per txg */ +unsigned int zfs_resilver_min_time_ms = 3000; /* min millisecs to resilver + per txg */ boolean_t zfs_no_scrub_io = B_FALSE; /* set to disable scrub i/o */ boolean_t zfs_no_scrub_prefetch = B_FALSE; /* set to disable srub prefetching */ + +SYSCTL_DECL(_vfs_zfs); +TUNABLE_INT("vfs.zfs.top_maxinflight", &zfs_top_maxinflight); +SYSCTL_UINT(_vfs_zfs, OID_AUTO, top_maxinflight, CTLFLAG_RW, + &zfs_top_maxinflight, 0, "Maximum I/Os per top-level vdev"); +TUNABLE_INT("vfs.zfs.resilver_delay", &zfs_resilver_delay); +SYSCTL_UINT(_vfs_zfs, OID_AUTO, resilver_delay, CTLFLAG_RW, + &zfs_resilver_delay, 0, "Number of ticks to delay resilver"); +TUNABLE_INT("vfs.zfs.scrub_delay", &zfs_scrub_delay); +SYSCTL_UINT(_vfs_zfs, OID_AUTO, scrub_delay, CTLFLAG_RW, + &zfs_scrub_delay, 0, "Number of ticks to delay scrub"); +TUNABLE_INT("vfs.zfs.scan_idle", &zfs_scan_idle); +SYSCTL_UINT(_vfs_zfs, OID_AUTO, scan_idle, CTLFLAG_RW, + &zfs_scan_idle, 0, "Idle scan window in clock ticks"); +TUNABLE_INT("vfs.zfs.scan_min_time_ms", &zfs_scan_min_time_ms); +SYSCTL_UINT(_vfs_zfs, OID_AUTO, scan_min_time_ms, CTLFLAG_RW, + &zfs_scan_min_time_ms, 0, "Min millisecs to scrub per txg"); +TUNABLE_INT("vfs.zfs.free_min_time_ms", &zfs_free_min_time_ms); +SYSCTL_UINT(_vfs_zfs, OID_AUTO, free_min_time_ms, CTLFLAG_RW, + &zfs_free_min_time_ms, 0, "Min millisecs to free per txg"); +TUNABLE_INT("vfs.zfs.resilver_min_time_ms", &zfs_resilver_min_time_ms); +SYSCTL_UINT(_vfs_zfs, OID_AUTO, resilver_min_time_ms, CTLFLAG_RW, + &zfs_resilver_min_time_ms, 0, "Min millisecs to resilver per txg"); +TUNABLE_INT("vfs.zfs.no_scrub_io", &zfs_no_scrub_io); +SYSCTL_INT(_vfs_zfs, OID_AUTO, no_scrub_io, CTLFLAG_RW, + &zfs_no_scrub_io, 0, "Disable scrub I/O"); +TUNABLE_INT("vfs.zfs.no_scrub_prefetch", &zfs_no_scrub_prefetch); +SYSCTL_INT(_vfs_zfs, OID_AUTO, no_scrub_prefetch, CTLFLAG_RW, + &zfs_no_scrub_prefetch, 0, "Disable scrub prefetching"); + enum ddt_class zfs_scrub_ddt_class_max = DDT_CLASS_DUPLICATE; #define DSL_SCAN_IS_SCRUB_RESILVER(scn) \ @@ -405,7 +436,7 @@ dsl_scan_check_pause(dsl_scan_t *scn, const zbookmark_t *zb) { uint64_t elapsed_nanosecs; - int mintime; + unsigned int mintime; /* we never skip user/group accounting objects */ if (zb && (int64_t)zb->zb_object < 0) @@ -1638,7 +1669,7 @@ boolean_t needs_io; int zio_flags = ZIO_FLAG_SCAN_THREAD | ZIO_FLAG_RAW | ZIO_FLAG_CANFAIL; int zio_priority; - int scan_delay = 0; + unsigned int scan_delay = 0; if (phys_birth <= scn->scn_phys.scn_min_txg || phys_birth >= scn->scn_phys.scn_max_txg) @@ -1695,7 +1726,8 @@ if (needs_io && !zfs_no_scrub_io) { vdev_t *rvd = spa->spa_root_vdev; - uint64_t maxinflight = rvd->vdev_children * zfs_top_maxinflight; + uint64_t maxinflight = rvd->vdev_children * + MAX(zfs_top_maxinflight, 1); void *data = zio_data_buf_alloc(size); mutex_enter(&spa->spa_scrub_lock); @@ -1709,7 +1741,7 @@ * then throttle our workload to limit the impact of a scan. */ if (ddi_get_lbolt64() - spa->spa_last_io <= zfs_scan_idle) - delay(scan_delay); + delay(MAX((int)scan_delay, 0)); zio_nowait(zio_read(NULL, spa, bp, data, size, dsl_scan_scrub_done, NULL, zio_priority, --------------000308000506080402090900-- From owner-zfs-devel@FreeBSD.ORG Mon Jul 2 07:06:25 2012 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 8E5B8106564A; Mon, 2 Jul 2012 07:06:25 +0000 (UTC) (envelope-from pawel@dawidek.net) Received: from mail.dawidek.net (60.wheelsystems.com [83.12.187.60]) by mx1.freebsd.org (Postfix) with ESMTP id 43FCE8FC15; Mon, 2 Jul 2012 07:06:25 +0000 (UTC) Received: from localhost (58.wheelsystems.com [83.12.187.58]) by mail.dawidek.net (Postfix) with ESMTPSA id 2270310C; Mon, 2 Jul 2012 09:06:24 +0200 (CEST) Date: Mon, 2 Jul 2012 09:04:18 +0200 From: Pawel Jakub Dawidek To: Martin Matuska Message-ID: <20120702070418.GA1372@garage.freebsd.pl> References: <4FF07140.9020401@FreeBSD.org> <20120701160458.GQ1400@garage.freebsd.pl> <4FF14442.5080905@FreeBSD.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="yrj/dFKFPuw6o+aM" Content-Disposition: inline In-Reply-To: <4FF14442.5080905@FreeBSD.org> X-OS: FreeBSD 10.0-CURRENT amd64 User-Agent: Mutt/1.5.21 (2010-09-15) Cc: zfs-devel@FreeBSD.org Subject: Re: [CFR] Tunables for scrub and resilver X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Jul 2012 07:06:25 -0000 --yrj/dFKFPuw6o+aM Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Mon, Jul 02, 2012 at 08:48:34AM +0200, Martin Matuska wrote: > +SYSCTL_UINT(_vfs_zfs, OID_AUTO, top_maxinflight, CTLFLAG_RW, > + &zfs_top_maxinflight, 0, "Maximum I/Os per top-level vdev"); This tab is still not fixed. Other than that the patch looks good. --=20 Pawel Jakub Dawidek http://www.wheelsystems.com FreeBSD committer http://www.FreeBSD.org Am I Evil? Yes, I Am! http://tupytaj.pl --yrj/dFKFPuw6o+aM Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (FreeBSD) iEYEARECAAYFAk/xR/IACgkQForvXbEpPzR6jgCfQHgI1ahFJMJ1fi0OzeHyQLkw I8gAn3wxj089Kbn3KDyc/YefrNaStqM4 =Dc6F -----END PGP SIGNATURE----- --yrj/dFKFPuw6o+aM-- From owner-zfs-devel@FreeBSD.ORG Tue Aug 28 14:26:10 2012 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id DC99E106566B; Tue, 28 Aug 2012 14:26:10 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 178FC8FC0C; Tue, 28 Aug 2012 14:26:06 +0000 (UTC) Received: from odyssey.starpoint.kiev.ua (alpha-e.starpoint.kiev.ua [212.40.38.101]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id RAA22612; Tue, 28 Aug 2012 17:25:53 +0300 (EEST) (envelope-from avg@FreeBSD.org) Message-ID: <503CD4F1.6060001@FreeBSD.org> Date: Tue, 28 Aug 2012 17:25:53 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:14.0) Gecko/20120730 Thunderbird/14.0 MIME-Version: 1.0 To: Trent Nelson References: <20120824011517.GJ42732@snakebite.org> In-Reply-To: <20120824011517.GJ42732@snakebite.org> X-Enigmail-Version: 1.4.3 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Mailman-Approved-At: Tue, 28 Aug 2012 15:18:45 +0000 Cc: freebsd-fs@FreeBSD.org Subject: Re: chmod -h 000x against symlink has bizarre results on ZFS X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 28 Aug 2012 14:26:11 -0000 on 24/08/2012 04:15 Trent Nelson said the following: > Hi folks, > > I recently set up a FreeBSD build slave for the Python project, > and noticed some symlink tests were failing in a very strange way > (http://bugs.python.org/issue15748). > > When chmod -h 000x is done against a file/link of length less than > 24, the target seems to get padded out to 24 with 0s. If it's > longer than 24, it'll get truncated. 'x' can be 7, 6, 5 or 4 and > the behaviour is the same. > > Here's the output from the attached test_readlink.sh, also available > at http://bugs.python.org/file26979/test_readlink.sh: > > % ./test_readlink.sh > > ****** TEST 1: link/target length less than 24 ****** > before chmod -h 0007: > -rw-r----- /tmp/lt24 > lrwxr-x--- /tmp/lt24.lnk->/tmp/lt24 > python os.readlink(/tmp/lt24.lnk): > '/tmp/lt24' > after chmod -h 0007: > -rw-r----- /tmp/lt24 > l------rwx /tmp/lt24.lnk->/tmp/lt24 > python os.readlink(/tmp/lt24.lnk): > '/tmp/lt24\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00' > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > target is padded out with NULLs to 24 > > > > > ****** TEST 2: link/target length longer than 24 ****** > before chmod -h 0007: > -rw-r----- /tmp/definitelywaylongerthantwentyfour > lrwxr-x--- > /tmp/definitelywaylongerthantwentyfour.lnk->/tmp/definitelywaylongerthantwentyfour > python os.readlink(/tmp/definitelywaylongerthantwentyfour.lnk): > '/tmp/definitelywaylongerthantwentyfour' > after chmod -h 0007: > -rw-r----- /tmp/definitelywaylongerthantwentyfour > l------rwx > /tmp/definitelywaylongerthantwentyfour.lnk->/tmp/definitelywaylonger > python os.readlink(/tmp/definitelywaylongerthantwentyfour.lnk): > '/tmp/definitelywaylonger' > ^^^^^^^^^^^^^^^^^^^^^^^^ > target gets truncated to 24 > > > > ****** Other modes... ****** > after chmod -h 0006: > l------rw- > /tmp/definitelywaylongerthantwentyfour.lnk->/tmp/definitelywaylonger > after chmod -h 0005: > l------r-x > /tmp/definitelywaylongerthantwentyfour.lnk->/tmp/definitelywaylonger > after chmod -h 0004: > l------r-- > /tmp/definitelywaylongerthantwentyfour.lnk->/tmp/definitelywaylonger > after chmod -h 0000: > l--------- > /tmp/definitelywaylongerthantwentyfour.lnk->/tmp/definitelywaylongerthantwentyfour > > > This only happens on ZFS. I'm on v28, don't have any v15s lying > around. > > I'm perplexed. Can others reproduce it? > I can reproduce this problem I can also provide some additional bits of information using a modified version of zdb: $ ln -fs definitelywaylongerthantwentyfour definitelywaylongerthantwentyfour.lnk $ stat -s definitelywaylongerthantwentyfour.lnk st_dev=3895460379 st_ino=27165 st_mode=0120755 st_nlink=1 st_uid=0 st_gid=0 st_rdev=4294967295 st_size=33 st_atime=1346161009 st_mtime=1346161009 st_ctime=1346161009 st_birthtime=1346161009 st_blksize=131072 st_blocks=1 st_flags=0 $ zdb -ddddddd tank/tmp 27165 Dataset tank/tmp [ZPL], ID 69, cr_txg 31, 4.57G, 24910 objects, rootbp DVA[0]=<0:5c5375e000:200> DVA[1]=<0:4c1a80ce00:200> [L0 DMU objset] fletcher4 lzjb LE contiguous unique double size=800L/200P birth=70882769L/70882769P fill=24910 cksum=1c72e8f065:89bbdf9d575:1732432c541ff:2d672d98b0ff66 Object lvl iblk dblk dsize lsize %full type 27165 1 16K 512 0 512 0.00 ZFS plain file (K=inherit) (Z=inherit) 209 bonus System attributes dnode flags: USERUSED_ACCOUNTED dnode maxblkid: 0 path /definitelywaylongerthantwentyfour.lnk uid 0 gid 0 atime Tue Aug 28 16:36:49 2012 mtime Tue Aug 28 16:36:49 2012 ctime Tue Aug 28 16:36:49 2012 crtime Tue Aug 28 16:36:49 2012 gen 70882769 mode 120755 size 33 parent 3 links 1 pflags 40800000104 symlink definitelywaylongerthantwentyfour symlink size 33 Indirect blocks: $ chmod -h 0007 definitelywaylongerthantwentyfour.lnk $ stat -s definitelywaylongerthantwentyfour.lnk st_dev=3895460379 st_ino=27165 st_mode=0120007 st_nlink=1 st_uid=0 st_gid=0 st_rdev=4294967295 st_size=33 st_atime=1346161009 st_mtime=1346161009 st_ctime=1346161227 st_birthtime=1346161227 st_blksize=131072 st_blocks=1 st_flags=0 $ zdb -ddddddd tank/tmp 27165 Dataset tank/tmp [ZPL], ID 69, cr_txg 31, 4.57G, 24910 objects, rootbp DVA[0]=<0:5c556b4400:200> DVA[1]=<0:4c1a989600:200> [L0 DMU objset] fletcher4 lzjb LE contiguous unique double size=800L/200P birth=70882812L/70882812P fill=24910 cksum=170e778d58:737e87307d3:140a45f4106a6:283187f7da9de7 Object lvl iblk dblk dsize lsize %full type 27165 1 16K 512 0 512 0.00 ZFS plain file (K=inherit) (Z=inherit) 216 bonus System attributes dnode flags: USERUSED_ACCOUNTED dnode maxblkid: 0 path /definitelywaylongerthantwentyfour.lnk uid 0 gid 0 atime Tue Aug 28 16:36:49 2012 mtime Tue Aug 28 16:36:49 2012 ctime Tue Aug 28 16:40:27 2012 crtime Tue Aug 28 16:36:49 2012 gen 70882769 mode 120007 size 33 parent 3 links 1 pflags 40800000004 symlink definitelywaylongerthant symlink size 24 Indirect blocks: Note how the file/object size remains 33, but size of ZPL_SYMLINK attribute is changed to 24. -- Andriy Gapon From owner-zfs-devel@FreeBSD.ORG Tue Aug 28 15:01:41 2012 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 06FCB106566C; Tue, 28 Aug 2012 15:01:41 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 1724A8FC14; Tue, 28 Aug 2012 15:01:39 +0000 (UTC) Received: from odyssey.starpoint.kiev.ua (alpha-e.starpoint.kiev.ua [212.40.38.101]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id SAA23086; Tue, 28 Aug 2012 18:01:34 +0300 (EEST) (envelope-from avg@FreeBSD.org) Message-ID: <503CDD4E.6050902@FreeBSD.org> Date: Tue, 28 Aug 2012 18:01:34 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:14.0) Gecko/20120730 Thunderbird/14.0 MIME-Version: 1.0 To: Trent Nelson References: <20120824011517.GJ42732@snakebite.org> <503CD4F1.6060001@FreeBSD.org> In-Reply-To: <503CD4F1.6060001@FreeBSD.org> X-Enigmail-Version: 1.4.3 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Mailman-Approved-At: Tue, 28 Aug 2012 15:52:37 +0000 Cc: freebsd-fs@FreeBSD.org Subject: Re: chmod -h 000x against symlink has bizarre results on ZFS X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 28 Aug 2012 15:01:41 -0000 on 28/08/2012 17:25 Andriy Gapon said the following: [snip] > I can reproduce this problem > I can also provide some additional bits of information using a modified version of > zdb: > > $ ln -fs definitelywaylongerthantwentyfour definitelywaylongerthantwentyfour.lnk > $ stat -s definitelywaylongerthantwentyfour.lnk > st_dev=3895460379 st_ino=27165 st_mode=0120755 st_nlink=1 st_uid=0 st_gid=0 > st_rdev=4294967295 st_size=33 st_atime=1346161009 st_mtime=1346161009 > st_ctime=1346161009 st_birthtime=1346161009 st_blksize=131072 st_blocks=1 st_flags=0 > $ zdb -ddddddd tank/tmp 27165 > Dataset tank/tmp [ZPL], ID 69, cr_txg 31, 4.57G, 24910 objects, rootbp > DVA[0]=<0:5c5375e000:200> DVA[1]=<0:4c1a80ce00:200> [L0 DMU objset] fletcher4 lzjb > LE contiguous unique double size=800L/200P birth=70882769L/70882769P fill=24910 > cksum=1c72e8f065:89bbdf9d575:1732432c541ff:2d672d98b0ff66 > > Object lvl iblk dblk dsize lsize %full type > 27165 1 16K 512 0 512 0.00 ZFS plain file (K=inherit) > (Z=inherit) > 209 bonus System attributes > dnode flags: USERUSED_ACCOUNTED > dnode maxblkid: 0 > path /definitelywaylongerthantwentyfour.lnk > uid 0 > gid 0 > atime Tue Aug 28 16:36:49 2012 > mtime Tue Aug 28 16:36:49 2012 > ctime Tue Aug 28 16:36:49 2012 > crtime Tue Aug 28 16:36:49 2012 > gen 70882769 > mode 120755 > size 33 > parent 3 > links 1 > pflags 40800000104 > symlink definitelywaylongerthantwentyfour > symlink size 33 > Indirect blocks: > > $ chmod -h 0007 definitelywaylongerthantwentyfour.lnk > $ stat -s definitelywaylongerthantwentyfour.lnk > st_dev=3895460379 st_ino=27165 st_mode=0120007 st_nlink=1 st_uid=0 st_gid=0 > st_rdev=4294967295 st_size=33 st_atime=1346161009 st_mtime=1346161009 > st_ctime=1346161227 st_birthtime=1346161227 st_blksize=131072 st_blocks=1 st_flags=0 > $ zdb -ddddddd tank/tmp 27165 > Dataset tank/tmp [ZPL], ID 69, cr_txg 31, 4.57G, 24910 objects, rootbp > DVA[0]=<0:5c556b4400:200> DVA[1]=<0:4c1a989600:200> [L0 DMU objset] fletcher4 lzjb > LE contiguous unique double size=800L/200P birth=70882812L/70882812P fill=24910 > cksum=170e778d58:737e87307d3:140a45f4106a6:283187f7da9de7 > > Object lvl iblk dblk dsize lsize %full type > 27165 1 16K 512 0 512 0.00 ZFS plain file (K=inherit) > (Z=inherit) > 216 bonus System attributes > dnode flags: USERUSED_ACCOUNTED > dnode maxblkid: 0 > path /definitelywaylongerthantwentyfour.lnk > uid 0 > gid 0 > atime Tue Aug 28 16:36:49 2012 > mtime Tue Aug 28 16:36:49 2012 > ctime Tue Aug 28 16:40:27 2012 > crtime Tue Aug 28 16:36:49 2012 > gen 70882769 > mode 120007 > size 33 > parent 3 > links 1 > pflags 40800000004 > symlink definitelywaylongerthant > symlink size 24 > Indirect blocks: > > Note how the file/object size remains 33, but size of ZPL_SYMLINK attribute is > changed to 24. > Will you be able to test the following patch? Preferably on a temporary test pool - I don't want to risk your data. diff --git a/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sa.c b/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sa.c index 69374fb..7f61517 100644 --- a/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sa.c +++ b/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sa.c @@ -1695,6 +1695,7 @@ sa_modify_attrs(sa_handle_t *hdl, sa_attr_type_t newattr, ASSERT(action == SA_REPLACE); SA_ADD_BULK_ATTR(attr_desc, j, attr, locator, datastart, buflen); + length_idx++; } else { length = SA_REGISTERED_LEN(sa, attr); if (length == 0) { -- Andriy Gapon From owner-zfs-devel@FreeBSD.ORG Tue Sep 18 18:43:20 2012 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 54F9D106566B for ; Tue, 18 Sep 2012 18:43:20 +0000 (UTC) (envelope-from gibbs@FreeBSD.org) Received: from aslan.scsiguy.com (www.scsiguy.com [70.89.174.89]) by mx1.freebsd.org (Postfix) with ESMTP id 266928FC0A for ; Tue, 18 Sep 2012 18:43:19 +0000 (UTC) Received: from [192.168.6.100] (207-225-98-3.dia.static.qwest.net [207.225.98.3]) (authenticated bits=0) by aslan.scsiguy.com (8.14.5/8.14.5) with ESMTP id q8IIhIbp038342 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NO); Tue, 18 Sep 2012 12:43:18 -0600 (MDT) (envelope-from gibbs@FreeBSD.org) From: "Justin T. Gibbs" Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Date: Tue, 18 Sep 2012 12:43:12 -0600 Message-Id: To: zfs-devel@FreeBSD.org Mime-Version: 1.0 (Mac OS X Mail 6.0 \(1486\)) X-Mailer: Apple Mail (2.1486) X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (aslan.scsiguy.com [70.89.174.89]); Tue, 18 Sep 2012 12:43:18 -0600 (MDT) Cc: Will Andrews Subject: ZFS STF Test Suite X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Sep 2012 18:43:20 -0000 Hi, Some time back, the folks at HighCloud Security ported a bunch of the ZFS tests from the Solaris test suite to FreeBSD. At Spectra Logic, we have continued the porting effort to enable more test cases, and now runs these tests in our nightly continuous integration system. Will and I are now starting the long process of extracting and pushing back the many changes Spectra has made to ZFS. This will be a lengthy process. However, the test suite is nicely contained and does not rely on any of our other kernel or userland changes. Is there anyone on the list who would be interested in taking the test suite and working with us to get it upstreamed into FreeBSD? Otherwise I fear it will be several months before Will or I can get to it on our own. Thanks, Justin From owner-zfs-devel@FreeBSD.ORG Wed Sep 19 03:01:44 2012 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 893201065670; Wed, 19 Sep 2012 03:01:44 +0000 (UTC) (envelope-from araujobsdport@gmail.com) Received: from mail-qc0-f182.google.com (mail-qc0-f182.google.com [209.85.216.182]) by mx1.freebsd.org (Postfix) with ESMTP id E91738FC14; Wed, 19 Sep 2012 03:01:40 +0000 (UTC) Received: by qcsg15 with SMTP id g15so544422qcs.13 for ; Tue, 18 Sep 2012 20:01:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:reply-to:in-reply-to:references:date:message-id :subject:from:to:cc:content-type; bh=aE6FFwHp/KjIzMKDjkDkqab2ySVqYpEPZAmiVHit2vY=; b=ulUdh+QL4MJr5upacIGeMIjAEpSwAfCM/xOe8bpRLzW7Ho2za30NShYVr2N+ZaMlk9 UV29pIr12tL9LNyT+3/GBEA2RePvlZFNFq6NupGfPzSm/QQWkBOdAxKblYW3h5lHURt2 9ROVbQUp5EmeezSZCOvKtUMFUQmH+MDoFUr1l/GYG88padY1yZMHGTLh7vfqW4vzPW4K tZLKkOAZ98epBpNZg/GPubu91ebQNRSKhJa4sbgK9nzpR8xzPgonYn6arJNl+6CSycry swWl6yJbN5AjQfXcc2EFclajfoQXoJ01tdpiIpZuHX3fXa0X5ksoHhz5n7Uile7POg+u ZIjg== MIME-Version: 1.0 Received: by 10.224.60.17 with SMTP id n17mr4324381qah.20.1348023699742; Tue, 18 Sep 2012 20:01:39 -0700 (PDT) Received: by 10.49.1.202 with HTTP; Tue, 18 Sep 2012 20:01:39 -0700 (PDT) In-Reply-To: References: Date: Wed, 19 Sep 2012 11:01:39 +0800 Message-ID: From: Marcelo Araujo To: "Justin T. Gibbs" Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: zfs-devel@freebsd.org, Will Andrews Subject: Re: ZFS STF Test Suite X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: araujo@FreeBSD.org List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Sep 2012 03:01:44 -0000 Hello Justin, Yes, I have interesting, because currently I'm working in a project that depend directly of ZFS as well as we need a good test suite. I have machines as well as I can dedicate some hours of my night to help uppstreamed the test suite into FreeBSD. Best Regards, - Araujo 2012/9/19 Justin T. Gibbs > Hi, > > Some time back, the folks at HighCloud Security ported a bunch of > the ZFS tests from the Solaris test suite to FreeBSD. At Spectra > Logic, we have continued the porting effort to enable more test > cases, and now runs these tests in our nightly continuous integration > system. > > Will and I are now starting the long process of extracting and > pushing back the many changes Spectra has made to ZFS. This will > be a lengthy process. However, the test suite is nicely contained > and does not rely on any of our other kernel or userland changes. > Is there anyone on the list who would be interested in taking the > test suite and working with us to get it upstreamed into FreeBSD? > Otherwise I fear it will be several months before Will or I can get > to it on our own. > > Thanks, > Justin > > _______________________________________________ > zfs-devel@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/zfs-devel > To unsubscribe, send any mail to "zfs-devel-unsubscribe@freebsd.org" > -- Marcelo Araujo araujo@FreeBSD.org From owner-zfs-devel@FreeBSD.ORG Thu Sep 20 16:46:32 2012 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 735A4106564A for ; Thu, 20 Sep 2012 16:46:32 +0000 (UTC) (envelope-from gibbs@FreeBSD.org) Received: from aslan.scsiguy.com (www.scsiguy.com [70.89.174.89]) by mx1.freebsd.org (Postfix) with ESMTP id 41E698FC16 for ; Thu, 20 Sep 2012 16:46:32 +0000 (UTC) Received: from [192.168.6.120] (207-225-98-3.dia.static.qwest.net [207.225.98.3]) (authenticated bits=0) by aslan.scsiguy.com (8.14.5/8.14.5) with ESMTP id q8KGkVPr052277 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NO); Thu, 20 Sep 2012 10:46:31 -0600 (MDT) (envelope-from gibbs@FreeBSD.org) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 6.1 \(1498\)) From: "Justin T. Gibbs" In-Reply-To: Date: Thu, 20 Sep 2012 10:46:26 -0600 Content-Transfer-Encoding: quoted-printable Message-Id: References: To: zfs-devel@FreeBSD.org X-Mailer: Apple Mail (2.1498) X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (aslan.scsiguy.com [70.89.174.89]); Thu, 20 Sep 2012 10:46:31 -0600 (MDT) Cc: Will Andrews Subject: Re: ZFS STF Test Suite X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Sep 2012 16:46:32 -0000 On Sep 18, 2012, at 12:43 PM, Justin T. Gibbs wrote: > Hi, >=20 > Some time back, the folks at HighCloud Security ported a bunch of > the ZFS tests from the Solaris test suite to FreeBSD. At Spectra > Logic, we have continued the porting effort to enable more test > cases, and now runs these tests in our nightly continuous integration > system. I've published a snapshot of our work here: = http://people.freebsd.org/~gibbs/freebsd_zfs_stf.tar.gz cddl/tools/regression/stf.README has instructions on how to build and = run the tests. cddl/tools/regression/stf.changes is a revision log (with diffs) of what HighCloud and Spectra havedone to the code. In briefly re-reviewing the changes, I believe that David Xu's work to implement process shared mutexes means change set 464984 should be redone. I haven't verified this though. -- Justin From owner-zfs-devel@FreeBSD.ORG Fri Sep 21 02:30:08 2012 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 96524106564A; Fri, 21 Sep 2012 02:30:08 +0000 (UTC) (envelope-from araujobsdport@gmail.com) Received: from mail-qa0-f54.google.com (mail-qa0-f54.google.com [209.85.216.54]) by mx1.freebsd.org (Postfix) with ESMTP id 347688FC12; Fri, 21 Sep 2012 02:30:08 +0000 (UTC) Received: by qady23 with SMTP id y23so1021759qad.13 for ; Thu, 20 Sep 2012 19:30:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:reply-to:in-reply-to:references:date:message-id :subject:from:to:cc:content-type; bh=g/9rU9LnxFntc+MlNp1agMBytnCe04NGbNbBAPchKbU=; b=FcUbxvbFEFDkhdPPYDo5tux4Pc+vgor5ioQl3Zm7gm26S8/WzfzjEwUuaxRdVBzxSK IR/H80Lw+6WcX3D69ulS/mDsJE9A//XlUTE0zaV+lBbpPfpBNzjZ5H5iS0UU2xqwd79F nEgKxBxLS7U77BX3goF4rjnBiml0iSdcpZZKdekxFfy30Q6Cis1wlZscX9O/sfdRuueP x0t3rRrCvsvaRRvqt7uezgQB2kf68H2K1O+OBllQgQMpBtg8w+Fp5iJvf+pjsItGeWOp 5t6gZ+LQ7BVqJpAwCfKFmrDyhRkRzmIi6BpPuMuBjnPoaCSKU5jASnZIrNnIItPJQTJJ 9poA== MIME-Version: 1.0 Received: by 10.224.71.74 with SMTP id g10mr9322254qaj.34.1348194607387; Thu, 20 Sep 2012 19:30:07 -0700 (PDT) Received: by 10.49.1.202 with HTTP; Thu, 20 Sep 2012 19:30:07 -0700 (PDT) In-Reply-To: References: Date: Fri, 21 Sep 2012 10:30:07 +0800 Message-ID: From: Marcelo Araujo To: "Justin T. Gibbs" Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: zfs-devel@freebsd.org, Will Andrews Subject: Re: ZFS STF Test Suite X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: araujo@FreeBSD.org List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Sep 2012 02:30:08 -0000 Justin, Thank you so much to share it, I'm gonna take a close look and make some plan to work on it. Also, would be interesting know, whom is working and in which part, so we could avoid dual effort on the same thing. I'm gonna build a machine today to test it. Best Regards, - Araujo 2012/9/21 Justin T. Gibbs > On Sep 18, 2012, at 12:43 PM, Justin T. Gibbs wrote: > > > Hi, > > > > Some time back, the folks at HighCloud Security ported a bunch of > > the ZFS tests from the Solaris test suite to FreeBSD. At Spectra > > Logic, we have continued the porting effort to enable more test > > cases, and now runs these tests in our nightly continuous integration > > system. > > I've published a snapshot of our work here: > http://people.freebsd.org/~gibbs/freebsd_zfs_stf.tar.gz > > cddl/tools/regression/stf.README has instructions on how to build and run > the tests. > > cddl/tools/regression/stf.changes is a revision log (with diffs) of what > HighCloud and Spectra havedone to the code. > > In briefly re-reviewing the changes, I believe that David Xu's work > to implement process shared mutexes means change set 464984 should > be redone. I haven't verified this though. > > -- > Justin > > > _______________________________________________ > zfs-devel@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/zfs-devel > To unsubscribe, send any mail to "zfs-devel-unsubscribe@freebsd.org" > -- Marcelo Araujo araujo@FreeBSD.org From owner-zfs-devel@FreeBSD.ORG Fri Sep 21 17:02:03 2012 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 20D02106567E; Fri, 21 Sep 2012 17:02:03 +0000 (UTC) (envelope-from mattjeet@gmail.com) Received: from mail-qa0-f54.google.com (mail-qa0-f54.google.com [209.85.216.54]) by mx1.freebsd.org (Postfix) with ESMTP id A2DAF8FC0A; Fri, 21 Sep 2012 17:02:02 +0000 (UTC) Received: by qady23 with SMTP id y23so1584092qad.13 for ; Fri, 21 Sep 2012 10:01:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:reply-to:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; bh=KpUWCHYh/wRR4n/fuT8rFJcelbLwLaOSkDjq8jerD5g=; b=NCRH6RlTxgL1+UxNs4D36YnaXAiF836cqqNb6KD9BAFu/CBUehw565cHBggq4doaid RHQghqvZerxoX0F/PcjuG5G/RlicEGZt5H4hq3xPbbifY4YqSkd7ymmKDAWPuz6oYGYY +vDroCdjEr12vPdW/ItQ34eOWbUqTsmxCN2ReEfA6uI9jCuT45L+qXL4aN7pUmhczIxq owsOLfc2qQiEH5D0K6ycqOMssICm3aEPPW5IlGl8pIlLxEfsP7N/qUHeuLAPZjGdrNpD TY8+7xk3bY8I2lZAO/Musiq+vrli5br44eor/UoARNvW21tWwpM4dt1SSwLVR6SpzTBG qoxg== MIME-Version: 1.0 Received: by 10.224.39.195 with SMTP id h3mr13702631qae.39.1348246916018; Fri, 21 Sep 2012 10:01:56 -0700 (PDT) Sender: mattjeet@gmail.com Received: by 10.49.81.193 with HTTP; Fri, 21 Sep 2012 10:01:55 -0700 (PDT) In-Reply-To: References: Date: Fri, 21 Sep 2012 10:01:55 -0700 X-Google-Sender-Auth: Kg9XiNjkI46wQqAzfldCX7j3NyU Message-ID: From: Matt Olander To: araujo@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 Cc: zfs-devel@freebsd.org, Will Andrews , larry Subject: Re: ZFS STF Test Suite X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: matt@ixsystems.com List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Sep 2012 17:02:03 -0000 On Thu, Sep 20, 2012 at 7:30 PM, Marcelo Araujo wrote: > Thank you so much to share it, I'm gonna take a close look and make some > plan to work on it. > Also, would be interesting know, whom is working and in which part, so we > could avoid dual effort on the same thing. > > I'm gonna build a machine today to test it. Nice job guys! Araujo, I've cc'd Larry at iXsystems. He's already setting up to take a look so maybe you guys can coordinate ;) Cheers, -matt > 2012/9/21 Justin T. Gibbs > >> On Sep 18, 2012, at 12:43 PM, Justin T. Gibbs wrote: >> >> > Hi, >> > >> > Some time back, the folks at HighCloud Security ported a bunch of >> > the ZFS tests from the Solaris test suite to FreeBSD. At Spectra >> > Logic, we have continued the porting effort to enable more test >> > cases, and now runs these tests in our nightly continuous integration >> > system. >> >> I've published a snapshot of our work here: >> http://people.freebsd.org/~gibbs/freebsd_zfs_stf.tar.gz >> >> cddl/tools/regression/stf.README has instructions on how to build and run >> the tests. >> >> cddl/tools/regression/stf.changes is a revision log (with diffs) of what >> HighCloud and Spectra havedone to the code. >> >> In briefly re-reviewing the changes, I believe that David Xu's work >> to implement process shared mutexes means change set 464984 should >> be redone. I haven't verified this though. >> >> -- >> Justin >> >> >> _______________________________________________ >> zfs-devel@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/zfs-devel >> To unsubscribe, send any mail to "zfs-devel-unsubscribe@freebsd.org" >> > > > > -- > Marcelo Araujo > araujo@FreeBSD.org > _______________________________________________ > zfs-devel@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/zfs-devel > To unsubscribe, send any mail to "zfs-devel-unsubscribe@freebsd.org" From owner-zfs-devel@FreeBSD.ORG Sat Sep 22 17:49:05 2012 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 362611065688; Sat, 22 Sep 2012 17:49:05 +0000 (UTC) (envelope-from araujobsdport@gmail.com) Received: from mail-qc0-f182.google.com (mail-qc0-f182.google.com [209.85.216.182]) by mx1.freebsd.org (Postfix) with ESMTP id 856448FC08; Sat, 22 Sep 2012 17:49:04 +0000 (UTC) Received: by qcsg15 with SMTP id g15so411983qcs.13 for ; Sat, 22 Sep 2012 10:48:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:reply-to:in-reply-to:references:date:message-id :subject:from:to:cc:content-type; bh=eTADDmFEtr/Q/A4g1mBoZIHmFwLhhNkAS+iB4Nl1XkU=; b=ElkWpK1QM9zCgUByLo5b0ShZ0c6LgkIanDrsFYvIBdlIlNF4P3UzcYGf4gt5sSmv/b 9db1Ncz0frWvyw3nKDYx67253KFnpaeo1f9Cu6zeu2AawOuU/PQ0B+SOjfDtKZ+XDa89 SA72gqjU2ukI3p+mw9wTQr9UCE3AsaWAxpYcDOcVGE9vySw31Vv5IyPaOi25XagA4caA zhQSlEjj3vfywC9WN8NUCDYKRBiJ6Vv085sX3C4PYKvf4MffjExMGuIbC7QHBcmie6tt onP7g1lwqntZHghGEG/UlHgWAxGZOtyjmsZLhuVBBbHOsYj3XeDwSN2eQWasKylZXwHq Vpsw== MIME-Version: 1.0 Received: by 10.224.177.77 with SMTP id bh13mr2569298qab.56.1348336138172; Sat, 22 Sep 2012 10:48:58 -0700 (PDT) Received: by 10.49.1.202 with HTTP; Sat, 22 Sep 2012 10:48:58 -0700 (PDT) In-Reply-To: References: Date: Sun, 23 Sep 2012 01:48:58 +0800 Message-ID: From: Marcelo Araujo To: matt@ixsystems.com Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: zfs-devel@freebsd.org, Will Andrews , larry Subject: Re: ZFS STF Test Suite X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: araujo@FreeBSD.org List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 22 Sep 2012 17:49:05 -0000 2012/9/22 Matt Olander > On Thu, Sep 20, 2012 at 7:30 PM, Marcelo Araujo > wrote: > > Thank you so much to share it, I'm gonna take a close look and make some > > plan to work on it. > > Also, would be interesting know, whom is working and in which part, so we > > could avoid dual effort on the same thing. > > > > I'm gonna build a machine today to test it. > > Nice job guys! > > Araujo, I've cc'd Larry at iXsystems. He's already setting up to take > a look so maybe you guys can coordinate ;) > > Cheers, > -matt > > Dear Matt and Larry, Would be great if we could be in touch and have some coordination to work together and bring it up on FreeBSD. So, let me know your plan guys, and let's sync our tasks. Best Regards, -- Marcelo Araujo araujo@FreeBSD.org From owner-zfs-devel@FreeBSD.ORG Sat Sep 22 16:28:09 2012 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 5194A106564A; Sat, 22 Sep 2012 16:28:09 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 1B2998FC08; Sat, 22 Sep 2012 16:28:07 +0000 (UTC) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id TAA07410; Sat, 22 Sep 2012 19:28:06 +0300 (EEST) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1TFSYo-000NTB-7M; Sat, 22 Sep 2012 19:28:06 +0300 Message-ID: <505DE715.8020806@FreeBSD.org> Date: Sat, 22 Sep 2012 19:28:05 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:15.0) Gecko/20120913 Thunderbird/15.0.1 MIME-Version: 1.0 To: freebsd-fs@FreeBSD.org X-Enigmail-Version: 1.4.3 Content-Type: text/plain; charset=X-VIET-VPS Content-Transfer-Encoding: 7bit X-Mailman-Approved-At: Sat, 22 Sep 2012 17:51:28 +0000 Cc: Subject: zfs: allow to mount root from a pool not in zpool.cache X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 22 Sep 2012 16:28:09 -0000 Currently FreeBSD ZFS kernel code doesn't allow to mount root filesystem on a pool that is not listed in zpool.cache as only pools from the cache are known to ZFS at that time. This patch is an attempt to improve the behavior: http://people.freebsd.org/~avg/spa_import_rootpool.diff This could be useful when importing pools that were exported from other systems. There is a tunable vfs.zfs.rootpool.prefer_cached_config which is set to 1 by default. 1 means just use a cached pool config if it's found in the cache, 0 means to re-probe disks and read supposedly latest/actual config in any case. -- Andriy Gapon From owner-zfs-devel@FreeBSD.ORG Sat Sep 22 16:49:38 2012 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 3C056106564A; Sat, 22 Sep 2012 16:49:38 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 587368FC08; Sat, 22 Sep 2012 16:49:36 +0000 (UTC) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id TAA07478; Sat, 22 Sep 2012 19:49:35 +0300 (EEST) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1TFStb-000NTq-6o; Sat, 22 Sep 2012 19:49:35 +0300 Message-ID: <505DEC1C.4000305@FreeBSD.org> Date: Sat, 22 Sep 2012 19:49:32 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:15.0) Gecko/20120913 Thunderbird/15.0.1 MIME-Version: 1.0 To: freebsd-fs@FreeBSD.org X-Enigmail-Version: 1.4.3 Content-Type: text/plain; charset=X-VIET-VPS Content-Transfer-Encoding: 7bit X-Mailman-Approved-At: Sat, 22 Sep 2012 18:08:17 +0000 Cc: Subject: znextboot: nextboot-like tool for zfs at zfsboot level X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 22 Sep 2012 16:49:38 -0000 Please find here a patchset that implement znextboot, a nextboot-like tool for zfs at zfsboot level: http://people.freebsd.org/~avg/znextboot.diff Theory of operation. zfsboot, through loader, exports to kernel environment the GUIDs of the very first pool it found ("primary pool") and the very first leaf vdev of that pool ("primary vdev"). Note that the primary pool is not necessarily a boot pool or a root pool, since a user can switch between pools and filesystems at various stages: zfsboot, zfsloader, rootfs specification. znextboot is a new tool that simply passes zfsboot/boot2 options to kernel ZFS via ioctl. Kernel ZFS writes the options as a NUL terminated ASCII string to the Pad2 area of the primary vdev of the primary pool. The Pad2 area has been known as "Boot Block Header" before. Its use was never formalized. Peviously it used to contain a special header (with zero useful information), now ZFS just zeroes it out. So, upon reboot zfsboot reads options from that area and zeros the area. The tool is intended for remote management of systems that use approaches similar to "Boot Environments". It is implemented at zfsboot level as opposed to loader level, because it was easier. My skills weren't sufficient to integrate the ZFS logic with loader's nextboot logic implemented in Forth. Some problematic areas in the current patchset: - I used just the next number for the nextboot ioctl. This will result in conflict when a new ioctl is added upstream. We need to think about reserving a range for OS-specific ioctls. - znextboot userland utility currently lacks any documentation. - znextboot lacks any sanity checking / validation for arguments that are passed to it. - probably more... -- Andriy Gapon From owner-zfs-devel@FreeBSD.ORG Sat Sep 22 17:13:11 2012 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 27BD4106566B; Sat, 22 Sep 2012 17:13:11 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id E32BB8FC17; Sat, 22 Sep 2012 17:13:09 +0000 (UTC) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id UAA07567; Sat, 22 Sep 2012 20:13:08 +0300 (EEST) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1TFTGN-000NUn-W4; Sat, 22 Sep 2012 20:13:08 +0300 Message-ID: <505DF1A3.1020809@FreeBSD.org> Date: Sat, 22 Sep 2012 20:13:07 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:15.0) Gecko/20120913 Thunderbird/15.0.1 MIME-Version: 1.0 To: freebsd-fs@FreeBSD.org X-Enigmail-Version: 1.4.3 Content-Type: text/plain; charset=X-VIET-VPS Content-Transfer-Encoding: 7bit X-Mailman-Approved-At: Sat, 22 Sep 2012 18:08:27 +0000 Cc: freebsd-geom@FreeBSD.org Subject: zfs zvol: set geom mediasize right at creation time X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 22 Sep 2012 17:13:11 -0000 Please review the following patch. In addition to what the description says I almost by accident sneaked another change into the patch. It's setting of stripesize to volblocksize. I think that the change should make sense, but it is really a different change. A side note: setting sectorsize to volblocksize seemed like an overkill and it would certainly mess the existing zvols in use. Maybe there should be another property like reportedblocksize or something. commit 1585e6cfb602c2a2647b9f802445bb174bc430a4 Author: Andriy Gapon Date: Wed Sep 19 20:49:28 2012 +0300 zvol: set mediasize in geom provider right upon its creation ... instead of deferring the action until first open. Unlike upstream this has no benefit on FreeBSD. We know that as soon as the provider is created it is going to be tasted and thus opened. Initial mediasize of zero causes tasting failure and subsequent retasting because of the size change. diff --git a/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zvol.c b/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zvol.c index d47d270..6e9e7a3 100644 --- a/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zvol.c +++ b/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zvol.c @@ -475,6 +475,7 @@ zvol_create_minor(const char *name) zvol_state_t *zv; objset_t *os; dmu_object_info_t doi; + uint64_t volblocksize, volsize; int error; ZFS_LOG(1, "Creating ZVOL %s...", name); @@ -535,9 +536,20 @@ zvol_create_minor(const char *name) zv = zs->zss_data = kmem_zalloc(sizeof (zvol_state_t), KM_SLEEP); #else /* !sun */ + error = zap_lookup(os, ZVOL_ZAP_OBJ, "size", 8, 1, &volsize); + if (error) { + ASSERT(error == 0); + dmu_objset_disown(os, zvol_tag); + mutex_exit(&spa_namespace_lock); + return (error); + } + DROP_GIANT(); g_topology_lock(); zv = zvol_geom_create(name); + zv->zv_volsize = volsize; + zv->zv_provider->mediasize = zv->zv_volsize; + #endif /* !sun */ (void) strlcpy(zv->zv_name, name, MAXPATHLEN); @@ -554,6 +566,7 @@ zvol_create_minor(const char *name) error = dmu_object_info(os, ZVOL_OBJ, &doi); ASSERT(error == 0); zv->zv_volblocksize = doi.doi_data_block_size; + zv->zv_provider->stripesize = zv->zv_volblocksize; if (spa_writeable(dmu_objset_spa(os))) { if (zil_replay_disable) -- Andriy Gapon From owner-zfs-devel@FreeBSD.ORG Mon Sep 24 16:17:47 2012 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 58443106566C; Mon, 24 Sep 2012 16:17:47 +0000 (UTC) (envelope-from larry@ixsystems.com) Received: from mail.iXsystems.com (newknight.ixsystems.com [206.40.55.70]) by mx1.freebsd.org (Postfix) with ESMTP id 202AC8FC20; Mon, 24 Sep 2012 16:17:46 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by mail.iXsystems.com (Postfix) with ESMTP id 63227524; Mon, 24 Sep 2012 09:17:46 -0700 (PDT) Received: from mail.iXsystems.com ([127.0.0.1]) by localhost (mail.ixsystems.com [127.0.0.1]) (maiad, port 10024) with ESMTP id 37921-02; Mon, 24 Sep 2012 09:17:45 -0700 (PDT) Received: from [10.2.3.72] (kruse-72.3.ixsystems.com [10.2.3.72]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mail.iXsystems.com (Postfix) with ESMTPSA id 10D0951E; Mon, 24 Sep 2012 09:17:45 -0700 (PDT) Message-ID: <506087A8.4000001@ixsystems.com> Date: Mon, 24 Sep 2012 09:17:44 -0700 From: Larry Maloney Organization: iX Systems User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:13.0) Gecko/20120621 Thunderbird/13.0.1 MIME-Version: 1.0 To: araujo@FreeBSD.org References: In-Reply-To: Content-Type: multipart/mixed; boundary="------------060605070709090805030809" X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: Will Andrews , zfs-devel@freebsd.org Subject: Re: ZFS STF Test Suite X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: larry@ixsystems.com List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 Sep 2012 16:17:47 -0000 This is a multi-part message in MIME format. --------------060605070709090805030809 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Sounds great! We should schedule a meeting, what is your schedule this week? Larry On 09/22/2012 10:48, Marcelo Araujo wrote: > > > 2012/9/22 Matt Olander > > > On Thu, Sep 20, 2012 at 7:30 PM, Marcelo Araujo > > wrote: > > Thank you so much to share it, I'm gonna take a close look and > make some > > plan to work on it. > > Also, would be interesting know, whom is working and in which > part, so we > > could avoid dual effort on the same thing. > > > > I'm gonna build a machine today to test it. > > Nice job guys! > > Araujo, I've cc'd Larry at iXsystems. He's already setting up to take > a look so maybe you guys can coordinate ;) > > Cheers, > -matt > > > Dear Matt and Larry, > > Would be great if we could be in touch and have some coordination to > work together and bring it up on FreeBSD. > So, let me know your plan guys, and let's sync our tasks. > > > Best Regards, > -- > Marcelo Araujo > araujo@FreeBSD.org -- ========================== Larry P. Maloney Senior Systems Engineer Cell: 408-893-0294 --------------060605070709090805030809-- From owner-zfs-devel@FreeBSD.ORG Mon Sep 24 16:37:10 2012 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id CBB781065672; Mon, 24 Sep 2012 16:37:10 +0000 (UTC) (envelope-from araujobsdport@gmail.com) Received: from mail-qa0-f47.google.com (mail-qa0-f47.google.com [209.85.216.47]) by mx1.freebsd.org (Postfix) with ESMTP id 61FDC8FC08; Mon, 24 Sep 2012 16:37:10 +0000 (UTC) Received: by qafi29 with SMTP id i29so2618583qaf.13 for ; Mon, 24 Sep 2012 09:37:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:reply-to:in-reply-to:references:date:message-id :subject:from:to:cc:content-type; bh=Vzpn9+UP/ku1etxA4xWfF5k3zGR7hP9+siE401MBpcM=; b=iGZzIko4hOOCVT87s6eGq34LHxLcKAdxGlSSpZ4ong30pF65Vgegpxnq1HBiKYBns4 TYGM63Y9FQiKk77bKatSS0mYYihpLZpjToSW2HUIZLRM7xyonN+FgirZSwKQtH6WC8bP FfzBCsJ60ypPKfWjP+cp4Km7YW0vrtmn8OlOLKSYNA8boctUF+/o3r+IRLFN1QZ6qH8/ +P7KhsSey/5PvAToRhgeEENQvfgnuiOyrqKXh41wYMbiEUmXB6rZwory32n9PcBsoLuR hG5k7XqQ9Yxu3V+kHpbU5DiaZUbQSsIV01ypUVXUWryh6+kEuGKllx1itCHDCUrsdYgY ApIA== MIME-Version: 1.0 Received: by 10.224.18.138 with SMTP id w10mr9529005qaa.20.1348504629631; Mon, 24 Sep 2012 09:37:09 -0700 (PDT) Received: by 10.49.1.202 with HTTP; Mon, 24 Sep 2012 09:37:09 -0700 (PDT) In-Reply-To: <506087A8.4000001@ixsystems.com> References: <506087A8.4000001@ixsystems.com> Date: Tue, 25 Sep 2012 00:37:09 +0800 Message-ID: From: Marcelo Araujo To: larry@ixsystems.com Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: Will Andrews , zfs-devel@freebsd.org Subject: Re: ZFS STF Test Suite X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: araujo@FreeBSD.org List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 Sep 2012 16:37:10 -0000 Hello dear Larry, Right now I'm located in Taipei/Taiwan, I do believe we have a difference of 11hours of time-zone. Let me check this week the best day/time to have a meeting with you. Soon as I have the proper date, I'm gonna contact you back. Great know you, and definitely we must have a chat this week. Best Regards, - Araujo 2012/9/25 Larry Maloney > Sounds great! > > We should schedule a meeting, what is your schedule this week? > > Larry > > > > On 09/22/2012 10:48, Marcelo Araujo wrote: > > > > 2012/9/22 Matt Olander > >> On Thu, Sep 20, 2012 at 7:30 PM, Marcelo Araujo >> wrote: >> > Thank you so much to share it, I'm gonna take a close look and make some >> > plan to work on it. >> > Also, would be interesting know, whom is working and in which part, so >> we >> > could avoid dual effort on the same thing. >> > >> > I'm gonna build a machine today to test it. >> >> Nice job guys! >> >> Araujo, I've cc'd Larry at iXsystems. He's already setting up to take >> a look so maybe you guys can coordinate ;) >> >> Cheers, >> -matt >> >> > Dear Matt and Larry, > > Would be great if we could be in touch and have some coordination to work > together and bring it up on FreeBSD. > So, let me know your plan guys, and let's sync our tasks. > > > Best Regards, > -- > Marcelo Araujo > araujo@FreeBSD.org > > > > -- > ========================== > Larry P. Maloney > Senior Systems Engineer > Cell: 408-893-0294 > > -- Marcelo Araujo araujo@FreeBSD.org From owner-zfs-devel@FreeBSD.ORG Mon Sep 24 17:17:32 2012 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 766F1106566B; Mon, 24 Sep 2012 17:17:32 +0000 (UTC) (envelope-from larry@ixsystems.com) Received: from mail.iXsystems.com (newknight.ixsystems.com [206.40.55.70]) by mx1.freebsd.org (Postfix) with ESMTP id 26DA58FC0C; Mon, 24 Sep 2012 17:17:31 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by mail.iXsystems.com (Postfix) with ESMTP id 6BEABC6B; Mon, 24 Sep 2012 10:17:31 -0700 (PDT) Received: from mail.iXsystems.com ([127.0.0.1]) by localhost (mail.ixsystems.com [127.0.0.1]) (maiad, port 10024) with ESMTP id 40745-01; Mon, 24 Sep 2012 10:17:30 -0700 (PDT) Received: from [10.2.3.72] (kruse-72.3.ixsystems.com [10.2.3.72]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mail.iXsystems.com (Postfix) with ESMTPSA id E867FC62; Mon, 24 Sep 2012 10:17:30 -0700 (PDT) Message-ID: <506095AA.2080503@ixsystems.com> Date: Mon, 24 Sep 2012 10:17:30 -0700 From: Larry Maloney Organization: iX Systems User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:13.0) Gecko/20120621 Thunderbird/13.0.1 MIME-Version: 1.0 To: araujo@FreeBSD.org References: <506087A8.4000001@ixsystems.com> In-Reply-To: Content-Type: multipart/mixed; boundary="------------080006090905010202080001" X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: Will Andrews , zfs-devel@freebsd.org Subject: Re: ZFS STF Test Suite X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: larry@ixsystems.com List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 Sep 2012 17:17:32 -0000 This is a multi-part message in MIME format. --------------080006090905010202080001 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Sounds great! Just let me know when, and I'll work my schedule to fit yours. Larry On 09/24/2012 09:37, Marcelo Araujo wrote: > Hello dear Larry, > > Right now I'm located in Taipei/Taiwan, I do believe we have a > difference of 11hours of time-zone. > Let me check this week the best day/time to have a meeting with you. > Soon as I have the proper date, I'm gonna contact you back. > > Great know you, and definitely we must have a chat this week. > > Best Regards, > - Araujo > > 2012/9/25 Larry Maloney > > > Sounds great! > > We should schedule a meeting, what is your schedule this week? > > Larry > > > > On 09/22/2012 10:48, Marcelo Araujo wrote: >> >> >> 2012/9/22 Matt Olander > > >> >> On Thu, Sep 20, 2012 at 7:30 PM, Marcelo Araujo >> > wrote: >> > Thank you so much to share it, I'm gonna take a close look >> and make some >> > plan to work on it. >> > Also, would be interesting know, whom is working and in >> which part, so we >> > could avoid dual effort on the same thing. >> > >> > I'm gonna build a machine today to test it. >> >> Nice job guys! >> >> Araujo, I've cc'd Larry at iXsystems. He's already setting up >> to take >> a look so maybe you guys can coordinate ;) >> >> Cheers, >> -matt >> >> >> Dear Matt and Larry, >> >> Would be great if we could be in touch and have some coordination >> to work together and bring it up on FreeBSD. >> So, let me know your plan guys, and let's sync our tasks. >> >> >> Best Regards, >> -- >> Marcelo Araujo >> araujo@FreeBSD.org > > > -- > ========================== > Larry P. Maloney > Senior Systems Engineer > Cell:408-893-0294 > > > > > -- > Marcelo Araujo > araujo@FreeBSD.org -- ========================== Larry P. Maloney Senior Systems Engineer Cell: 408-893-0294 --------------080006090905010202080001-- From owner-zfs-devel@FreeBSD.ORG Tue Sep 25 07:32:09 2012 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 31412106564A for ; Tue, 25 Sep 2012 07:32:09 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 69A328FC1B for ; Tue, 25 Sep 2012 07:32:07 +0000 (UTC) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id KAA03422 for ; Tue, 25 Sep 2012 10:32:01 +0300 (EEST) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1TGPce-00086U-Ju for zfs-devel@FreeBSD.org; Tue, 25 Sep 2012 10:32:00 +0300 Message-ID: <50615DEE.6030609@FreeBSD.org> Date: Tue, 25 Sep 2012 10:31:58 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:15.0) Gecko/20120913 Thunderbird/15.0.1 MIME-Version: 1.0 To: zfs-devel@FreeBSD.org X-Enigmail-Version: 1.4.3 Content-Type: text/plain; charset=X-VIET-VPS Content-Transfer-Encoding: 7bit Cc: Subject: ZFS_OBJ_HOLD_ENTER/z_hold_mtx recursion via getnewvnode X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Sep 2012 07:32:09 -0000 FYI: http://lists.freebsd.org/pipermail/freebsd-fs/2012-September/015188.html This panic seems to gain popularity lately. -- Andriy Gapon From owner-zfs-devel@FreeBSD.ORG Fri Sep 28 03:25:32 2012 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 99429106564A; Fri, 28 Sep 2012 03:25:32 +0000 (UTC) (envelope-from araujobsdport@gmail.com) Received: from mail-qc0-f182.google.com (mail-qc0-f182.google.com [209.85.216.182]) by mx1.freebsd.org (Postfix) with ESMTP id 317F78FC0C; Fri, 28 Sep 2012 03:25:32 +0000 (UTC) Received: by qcsl39 with SMTP id l39so2400791qcs.13 for ; Thu, 27 Sep 2012 20:25:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:reply-to:in-reply-to:references:date:message-id :subject:from:to:cc:content-type; bh=z/wRAmQrHY0KGxkxJy2jFRZdIsVZBGCH5kEbSTvZO3U=; b=EiydS/SVNvG5M/lFWrsfow445SIDBh+rtB8grz0AmP5jssWkAcD1n2ZxFkX+Bg65XK wrFIDziRrIYtUyOQbk3PJEd+6ORrVOsiD/yme6SDtPh6ONNL3QRXQQZUnGCPI0ckRss1 yU7SKxeot+hP8UtaV7tNmSMj5XOT6VTuUrkodMVAODfLwMDsMR9DxZ2UVNIjLwvy/2MK 3QHdggbsXf6BZIR//VfMNzUQ+Pdk/PV03rwz6xTAx1Vf7QZcN+q+3oTdHB8A3BVs+nZ1 diZOK0J+zkO2z+Q/PkzOiCtIWLpLaS5qRz/fnEEXtqiJVbjSiTqa6u7YTie/3F2nj5oh avew== MIME-Version: 1.0 Received: by 10.224.18.138 with SMTP id w10mr14270496qaa.20.1348802731308; Thu, 27 Sep 2012 20:25:31 -0700 (PDT) Received: by 10.49.1.202 with HTTP; Thu, 27 Sep 2012 20:25:31 -0700 (PDT) In-Reply-To: <506095AA.2080503@ixsystems.com> References: <506087A8.4000001@ixsystems.com> <506095AA.2080503@ixsystems.com> Date: Fri, 28 Sep 2012 11:25:31 +0800 Message-ID: From: Marcelo Araujo To: larry@ixsystems.com Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: Will Andrews , zfs-devel@freebsd.org Subject: Re: ZFS STF Test Suite X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: araujo@FreeBSD.org List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 Sep 2012 03:25:32 -0000 Hello dear Larry, I'm wondering if you could talk with me this Friday. If yes, let me know! So here in Taiwan, I'm 15 hours ahead of you at Sao Jose, so, my time zone is UTC/GMT +8. Would be great if we can talk around 11PM(Taiwan) at my side, it will be 2PM(California) in your side, just let me know few hours in advance if we can talk or not. In case you can't, we can re-schedule to the next week. Here is my skype: araujobsd ** Best Regards, - Marcelo Araujo ** 2012/9/25 Larry Maloney > Sounds great! Just let me know when, and I'll work my schedule to fit > yours. > > Larry > > > On 09/24/2012 09:37, Marcelo Araujo wrote: > > Hello dear Larry, > > Right now I'm located in Taipei/Taiwan, I do believe we have a > difference of 11hours of time-zone. > Let me check this week the best day/time to have a meeting with you. Soon > as I have the proper date, I'm gonna contact you back. > > Great know you, and definitely we must have a chat this week. > > Best Regards, > - Araujo > > 2012/9/25 Larry Maloney > >> Sounds great! >> >> We should schedule a meeting, what is your schedule this week? >> >> Larry >> >> >> >> On 09/22/2012 10:48, Marcelo Araujo wrote: >> >> >> >> 2012/9/22 Matt Olander >> >>> On Thu, Sep 20, 2012 at 7:30 PM, Marcelo Araujo >>> wrote: >>> > Thank you so much to share it, I'm gonna take a close look and make >>> some >>> > plan to work on it. >>> > Also, would be interesting know, whom is working and in which part, so >>> we >>> > could avoid dual effort on the same thing. >>> > >>> > I'm gonna build a machine today to test it. >>> >>> Nice job guys! >>> >>> Araujo, I've cc'd Larry at iXsystems. He's already setting up to take >>> a look so maybe you guys can coordinate ;) >>> >>> Cheers, >>> -matt >>> >>> >> Dear Matt and Larry, >> >> Would be great if we could be in touch and have some coordination to work >> together and bring it up on FreeBSD. >> So, let me know your plan guys, and let's sync our tasks. >> >> >> Best Regards, >> -- >> Marcelo Araujo >> araujo@FreeBSD.org >> >> >> >> -- >> ========================== >> Larry P. Maloney >> Senior Systems Engineer >> Cell: 408-893-0294 >> >> > > > -- > Marcelo Araujo > araujo@FreeBSD.org > > > > -- > ========================== > Larry P. Maloney > Senior Systems Engineer > Cell: 408-893-0294 > > -- Marcelo Araujo araujo@FreeBSD.org From owner-zfs-devel@FreeBSD.ORG Fri Sep 28 08:19:37 2012 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C0A9B1065670; Fri, 28 Sep 2012 08:19:37 +0000 (UTC) (envelope-from delphij@delphij.net) Received: from anubis.delphij.net (anubis.delphij.net [IPv6:2001:470:1:117::25]) by mx1.freebsd.org (Postfix) with ESMTP id 9A1A38FC15; Fri, 28 Sep 2012 08:19:37 +0000 (UTC) Received: from Xins-MacBook-Pro.local (c-67-188-85-47.hsd1.ca.comcast.net [67.188.85.47]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by anubis.delphij.net (Postfix) with ESMTPSA id 5E8C3FF35; Fri, 28 Sep 2012 01:19:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=delphij.net; s=anubis; t=1348820377; bh=6dRh4y9x7/VmMUtIYK/dUnz4MXdb1Fceg7HaTeURQ9A=; h=Date:From:Reply-To:To:CC:Subject:References:In-Reply-To; b=IJrcnep9LcFtzVAVDg6/y3nMSa2nNDE5u+wl8v9d7AoQA9x7ARMVHZLBJqlLAlCmM 21kSCbG/r2oiz7MyJpI9z7eA++in7Gxi+dagfyDTBzgd6Dbim4c+n7WLtJwC2v9zCO 7zsiw4Kc6p4woU3S1NCZOiW1/Ug/OrFmiwH0XFw0= Message-ID: <50655D99.8090703@delphij.net> Date: Fri, 28 Sep 2012 01:19:37 -0700 From: Xin Li Organization: The FreeBSD Project User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:15.0) Gecko/20120907 Thunderbird/15.0.1 MIME-Version: 1.0 To: araujo@FreeBSD.org References: <506087A8.4000001@ixsystems.com> <506095AA.2080503@ixsystems.com> In-Reply-To: X-Enigmail-Version: 1.4.4 Content-Type: text/plain; charset=GB2312 Content-Transfer-Encoding: 7bit Cc: Will Andrews , larry@ixsystems.com, zfs-devel@freebsd.org Subject: Re: ZFS STF Test Suite X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: d@delphij.net List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 Sep 2012 08:19:38 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 Hi, Marcelo, On 9/27/12 8:25 PM, Marcelo Araujo wrote: > Hello dear Larry, > > I'm wondering if you could talk with me this Friday. If yes, let me > know! So here in Taiwan, I'm 15 hours ahead of you at Sao Jose, so, > my time zone is UTC/GMT +8. > > Would be great if we can talk around 11PM(Taiwan) at my side, it > will be 2PM(California) in your side, just let me know few hours in > advance if we can talk or not. In case you can't, we can > re-schedule to the next week. I believe these time was wrong. 23:00 CST for you is 8:00 PDT for us. 14:00 PDT for us is about 5:00 CST in your morning. 17:00 PDT for us is about 8:00 CST in your morning. > Here is my skype: araujobsd ** Best Regards, - Marcelo Araujo ** > > 2012/9/25 Larry Maloney > >> Sounds great! Just let me know when, and I'll work my schedule >> to fit yours. >> >> Larry >> >> >> On 09/24/2012 09:37, Marcelo Araujo wrote: >> >> Hello dear Larry, >> >> Right now I'm located in Taipei/Taiwan, I do believe we have a >> difference of 11hours of time-zone. Let me check this week the >> best day/time to have a meeting with you. Soon as I have the >> proper date, I'm gonna contact you back. >> >> Great know you, and definitely we must have a chat this week. >> >> Best Regards, - Araujo >> >> 2012/9/25 Larry Maloney >> >>> Sounds great! >>> >>> We should schedule a meeting, what is your schedule this week? >>> >>> Larry >>> >>> >>> >>> On 09/22/2012 10:48, Marcelo Araujo wrote: >>> >>> >>> >>> 2012/9/22 Matt Olander >>> >>>> On Thu, Sep 20, 2012 at 7:30 PM, Marcelo Araujo >>>> wrote: >>>>> Thank you so much to share it, I'm gonna take a close look >>>>> and make >>>> some >>>>> plan to work on it. Also, would be interesting know, whom >>>>> is working and in which part, so >>>> we >>>>> could avoid dual effort on the same thing. >>>>> >>>>> I'm gonna build a machine today to test it. >>>> >>>> Nice job guys! >>>> >>>> Araujo, I've cc'd Larry at iXsystems. He's already setting up >>>> to take a look so maybe you guys can coordinate ;) >>>> >>>> Cheers, -matt >>>> >>>> >>> Dear Matt and Larry, >>> >>> Would be great if we could be in touch and have some >>> coordination to work together and bring it up on FreeBSD. So, >>> let me know your plan guys, and let's sync our tasks. >>> >>> >>> Best Regards, -- Marcelo Araujo araujo@FreeBSD.org >>> >>> >>> >>> -- ========================== Larry P. Maloney Senior Systems >>> Engineer Cell: 408-893-0294 >>> >>> >> >> >> -- Marcelo Araujo araujo@FreeBSD.org >> >> >> >> -- ========================== Larry P. Maloney Senior Systems >> Engineer Cell: 408-893-0294 >> >> > > -----BEGIN PGP SIGNATURE----- Version: GnuPG/MacGPG2 v2.0.17 (Darwin) iQEcBAEBCAAGBQJQZV2ZAAoJEG80Jeu8UPuz4yMH/RJbLrdeu37BeHi7Q/vf+ok3 ve4sPTF+k8R9qRRdKJ+xP52V2LeBJdT7TqxHA0+4YTwmGVjZlCaetXy8yUDysoth VA3EGaWFSzdTGV03xZ00KGcLfgH4uMUhN/7AADSr8g0h/ipBYQouzJ07hNe5l+0/ gSJd38FX3vmTI/PmcMHVey2+sa493jWpREnSzrtQs98kxkARGObL4G6bQCf4jEYA iqfqRb4ssyI65kh80MiCYbCYK0J0lnDkfDQjLNK44J7kY/S6Nk4+nKu/SdU7zLc4 aEWvIfZ8dScM1OGhYCOrZ+NaI6M2+yzeSbb0mNkK+x9VnNb55TzrtHW+zhnCuUs= =6WZA -----END PGP SIGNATURE----- From owner-zfs-devel@FreeBSD.ORG Fri Sep 28 08:59:39 2012 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id D9113106564A for ; Fri, 28 Sep 2012 08:59:39 +0000 (UTC) (envelope-from araujobsdport@gmail.com) Received: from mail-qa0-f47.google.com (mail-qa0-f47.google.com [209.85.216.47]) by mx1.freebsd.org (Postfix) with ESMTP id 881398FC1C for ; Fri, 28 Sep 2012 08:59:39 +0000 (UTC) Received: by qafi29 with SMTP id i29so6653343qaf.13 for ; Fri, 28 Sep 2012 01:59:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:reply-to:in-reply-to:references:date:message-id :subject:from:to:cc:content-type; bh=LZaLQl9vunWdzEbUzJhDRIyoQ6Zl+BeaSN/i5SxlmVc=; b=I3CxXI3hTBhjhS9510Z8bjvT+aR/3jprLYm40s47aW3laeuAP61vSADfc7Pk54A6Rc qkB6G3hSTatFqHgVBzs9SOpFciQ8Nt+zTU+i6SKlvf0Zhk6U+2NQULV/i0Etjt6sc1Mb 7tFNPB0ZoM+Ew0ehsh7wh04IPMLU7jGgCoAa4G6Yo+AytQqgEyHj8aTx6lJhxpfNo2V6 7hLxIC/QY3qoLCIsXbINIYJWzIM0P/tgCrvfeijB08hsN1EOG2oQ/UuqglN35MVWu3Yh b7kt2/MwdsB7yCqU6h4Gq2EhETBw+pFrmfX4c35sur/Rm27yb6yvE/EJaLyHw3H2YY9V lpww== MIME-Version: 1.0 Received: by 10.229.69.67 with SMTP id y3mr4115070qci.136.1348822778716; Fri, 28 Sep 2012 01:59:38 -0700 (PDT) Received: by 10.49.1.202 with HTTP; Fri, 28 Sep 2012 01:59:38 -0700 (PDT) In-Reply-To: <50655D99.8090703@delphij.net> References: <506087A8.4000001@ixsystems.com> <506095AA.2080503@ixsystems.com> <50655D99.8090703@delphij.net> Date: Fri, 28 Sep 2012 16:59:38 +0800 Message-ID: From: Marcelo Araujo To: d@delphij.net Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: Will Andrews , larry@ixsystems.com, zfs-devel@freebsd.org Subject: Re: ZFS STF Test Suite X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: araujo@FreeBSD.org List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 Sep 2012 08:59:40 -0000 2012/9/28 Xin Li > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA256 > > Hi, Marcelo, > > On 9/27/12 8:25 PM, Marcelo Araujo wrote: > > Hello dear Larry, > > > > I'm wondering if you could talk with me this Friday. If yes, let me > > know! So here in Taiwan, I'm 15 hours ahead of you at Sao Jose, so, > > my time zone is UTC/GMT +8. > > > > Would be great if we can talk around 11PM(Taiwan) at my side, it > > will be 2PM(California) in your side, just let me know few hours in > > advance if we can talk or not. In case you can't, we can > > re-schedule to the next week. > > I believe these time was wrong. 23:00 CST for you is 8:00 PDT for us. > > 14:00 PDT for us is about 5:00 CST in your morning. > > 17:00 PDT for us is about 8:00 CST in your morning. > Delphij, You are right! I always get confused with timezone. So, if I'm right now, let's setup this at 8AM(Saturday, Taiwan), 5PM(Friday, California). Is that OK? In case you want reschedule, feel free! Best Regards, -- Marcelo Araujo araujo@FreeBSD.org From owner-zfs-devel@FreeBSD.ORG Thu Oct 4 12:31:40 2012 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 1642E106566C; Thu, 4 Oct 2012 12:31:40 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 024BA8FC0C; Thu, 4 Oct 2012 12:31:38 +0000 (UTC) Received: from odyssey.starpoint.kiev.ua (alpha-e.starpoint.kiev.ua [212.40.38.101]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id PAA17803; Thu, 04 Oct 2012 15:31:36 +0300 (EEST) (envelope-from avg@FreeBSD.org) Message-ID: <506D81A7.8030506@FreeBSD.org> Date: Thu, 04 Oct 2012 15:31:35 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:15.0) Gecko/20120911 Thunderbird/15.0.1 MIME-Version: 1.0 To: Nikolay Denev References: <906543F2-96BD-4519-B693-FD5AFB646F87@gmail.com> <506BF372.1090208@FreeBSD.org> <506C4049.4040100@FreeBSD.org> In-Reply-To: X-Enigmail-Version: 1.4.3 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Mailman-Approved-At: Thu, 04 Oct 2012 12:35:21 +0000 Cc: freebsd-fs , Pawel Jakub Dawidek Subject: Re: nfs + zfs hangs on RELENG_9 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Oct 2012 12:31:40 -0000 [restoring cc to fs@] on 04/10/2012 14:32 Nikolay Denev said the following: > I have procstat only for the nfsd threads from the moment of the IO hang. > And this is the only one with "arc" : > > 1422 138630 nfsd nfsd: service mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x390 arc_lowmem+0x77 kmem_malloc+0xc1 > uma_large_malloc+0x4a malloc+0xd9 arc_get_data_buf+0xb5 arc_read_nolock+0x1ec > arc_read+0x93 dbuf_read+0x452 dmu_buf_hold_array_by_dnode+0x16b > dmu_buf_hold_array+0x67 dmu_read_uio+0x3f zfs_freebsd_read+0x3e8 > nfsvno_read+0x2e5 nfsrvd_read+0x3ff nfsrvd_dorpc+0x3c0 Oh, very important stack trace. Earlier Nikolay Denev said the following: > PID TID COMM TDNAME KSTACK > 7 100192 zfskern arc_reclaim_thre mi_switch+0x186 sleepq_wait+0x42 _sx_xlock_hard+0x428 > _sx_xlock+0x51 arc_buf_remove_ref+0x8a dbuf_rele_and_unlock+0x132 dbuf_evict+0x11 > dbuf_do_evict+0x53 arc_do_user_evicts+0xb4 arc_reclaim_thread+0x263 fork_exit+0x11f > fork_trampoline+0xe To me this looks like a deadlock caused by a FreeBSD add-on to ZFS: arc_lowmem handler. I think that this is what happens: The nfsd thread does read, arc_read_nolock finds a buffer in a ghost cache and calls arc_get_data_buf while holding a hash_lock (one of buffer hash locks). arc_get_data_buf needs to allocate some memory and, as luck would have it, there is a memory shortage. Low memory handlers are invoked (directly) and one of them is arc_lowmem. arc_lowmem simply kicks arc_reclaim_thread to do its job and then loops sleep-waiting until memory shortage is less severe. arc_reclaim_thread tries to evict some buffers and, as luck would have it again, it attempts to evict either the same buffer or, most likely, a different buffer that hashes to the same lock. So arc_reclaim_thread is blocked on the arc buffer lock. While the nfsd thread holds the lock, but waits in arc_lowmem for arc_reclaim_thread to make progress. Eventually the held lock stalls other threads that attempt to grab it, the stall propagates to txg_sync_thread threads and all ZFS I/O stops. -- Andriy Gapon From owner-zfs-devel@FreeBSD.ORG Tue Oct 30 03:57:55 2012 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 741F86B7; Tue, 30 Oct 2012 03:57:55 +0000 (UTC) (envelope-from araujobsdport@gmail.com) Received: from mail-qc0-f182.google.com (mail-qc0-f182.google.com [209.85.216.182]) by mx1.freebsd.org (Postfix) with ESMTP id 067738FC0C; Tue, 30 Oct 2012 03:57:54 +0000 (UTC) Received: by mail-qc0-f182.google.com with SMTP id k19so3694530qcs.13 for ; Mon, 29 Oct 2012 20:57:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:reply-to:in-reply-to:references:date:message-id :subject:from:to:cc:content-type; bh=e7h4M5mdHrId6L3pfehNv8DNnuuhBawUZ5KeHJspIaQ=; b=Low7vXBUu1E+cyTc8DkaDxbzZ2JMPK5j6Ly7eQIKDCHRzUig3iXD2jJwXH7aFQ6Eku clAGeMPAY6upaCeMPVyydDRkSckAdnj0QAljI6UTsPtVW0UCMZ0g3kgxcmmoJ67VmJgm bp3Gf/1PmloBB9FeKqapw2JrRbKDHaRN4aB+mPy1GEqNBk3NyLI6LhG7EhGHz0RfuEn3 B0IV6GAEot8ZRoMTEQKX8B8I6bGA7pRTTukemAAP/Xerq9m+pUkijYk5Pw/JgxL2MLko WKQLY3+l1PCRKKEWWvFFln05nhCz+D44gaO8rcOm0XcwP1xWa/RGb0YKolsxa6Cokjvc Ryog== MIME-Version: 1.0 Received: by 10.224.70.140 with SMTP id d12mr14292828qaj.53.1351569474099; Mon, 29 Oct 2012 20:57:54 -0700 (PDT) Received: by 10.49.62.69 with HTTP; Mon, 29 Oct 2012 20:57:53 -0700 (PDT) In-Reply-To: References: Date: Tue, 30 Oct 2012 11:57:53 +0800 Message-ID: Subject: Re: ZFS STF Test Suite From: Marcelo Araujo To: "Justin T. Gibbs" , larry , Matt Olander Content-Type: multipart/mixed; boundary=bcaec51b15f1649aab04cd3ecad6 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: zfs-devel@freebsd.org, Will Andrews X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list Reply-To: araujo@FreeBSD.org List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 Oct 2012 03:57:55 -0000 --bcaec51b15f1649aab04cd3ecad6 Content-Type: text/plain; charset=ISO-8859-1 Hello guys, I have been working a little bit over the zfs-test suite. Unfortunately I'm totally stuck at work these days. I'm still working on it, here is a couple of scripts that I have fixed in last weekend, I intend to keep doing this work as I spoke before. So here is the first changes, I'm wondering where I should place it! Is there any special place that I could make a ci or send to someone else to make it for me. Best Regards, - Marcelo Araujo 2012/9/21 Justin T. Gibbs > On Sep 18, 2012, at 12:43 PM, Justin T. Gibbs wrote: > > > Hi, > > > > Some time back, the folks at HighCloud Security ported a bunch of > > the ZFS tests from the Solaris test suite to FreeBSD. At Spectra > > Logic, we have continued the porting effort to enable more test > > cases, and now runs these tests in our nightly continuous integration > > system. > > I've published a snapshot of our work here: > http://people.freebsd.org/~gibbs/freebsd_zfs_stf.tar.gz > > cddl/tools/regression/stf.README has instructions on how to build and run > the tests. > > cddl/tools/regression/stf.changes is a revision log (with diffs) of what > HighCloud and Spectra havedone to the code. > > In briefly re-reviewing the changes, I believe that David Xu's work > to implement process shared mutexes means change set 464984 should > be redone. I haven't verified this though. > > -- > Justin > > > _______________________________________________ > zfs-devel@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/zfs-devel > To unsubscribe, send any mail to "zfs-devel-unsubscribe@freebsd.org" > -- Marcelo Araujo araujo@FreeBSD.org --bcaec51b15f1649aab04cd3ecad6 Content-Type: text/plain; charset=US-ASCII; name="Modified notes.TXT" Content-Disposition: attachment; filename="Modified notes.TXT" Content-Transfer-Encoding: base64 X-Attachment-Id: f_h8whitln1 Ym9vdGZzDQo9PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PQ0KMS4gYm9vdGZzXzAw MV9wb3MgKFBBU1MsIGFkZCBuZXcgY2FzZSkNCiAgIEFkZCBjYXNlIDogU2V0IGJvb3RmcyBmcm9t IGZzQHNuYXAuIChNb3ZlZCBmcm9tIGJvb3Rmc18wMDJfbmVnKQ0KDQoJIE1vZGlmaWVkIDogLlx0 ZXN0c1xmdW5jdGlvbmFsXGJvb3Rmc1xib290ZnNfMDAxX3BvcyANCgkgDQoyLiBib290ZnNfMDAy X25lZyAoRkFJTCkNCiAgIFRoZSBzbmFwc2hvdCBjYW4gYmUgc2V0IHRvIGJvb3RmcyBpbiBmcmVl YnNkLiAoU29sYXJpcyBjYW5ub3QpDQoNCiAgIFNvbHV0aW9uIDogTW9kaWZ5IHNuYXBzaG90IGJv b3RmcyBmcm9tIGxvZ19tdXN0bm90IHRvIGxvZ19tdXN0IGFuZCBtb3ZlIHRoaXMgY2FzZSB0byBi b290ZnNfMDAxX3Bvcw0KCSBNb2RpZmllZCA6IC5cdGVzdHNcZnVuY3Rpb25hbFxib290ZnNcYm9v dGZzXzAwMl9uZWcgDQoJICAgIA0KMy4gYm9vdGZzXzAwNl9wb3MJKEZBSUwpDQogICBBbnkga2lu ZCBvZiBwb29scyBjYW4gYmUgc2V0IGFzIGJvb3RmcyBpbiBmcmVlYnNkLiAoU29sYXJpcyBzdXBw b3J0IG9ubHkgbm9ybWFsICYgbWlycm9yIHR5cGUpDQogICANCiAgIFNvbHV0aW9uIDogTW9kaWZ5 IGFsbCB0aGUgbG9nX211c3Rub3QgY2FzZXMgaW50byBsb2dfbXVzdCBjYXNlcy4NCgkgTW9kaWZp ZWQgOiAuXHRlc3RzXGZ1bmN0aW9uYWxcYm9vdGZzXGJvb3Rmc18wMDZfcG9zIA0KCSANCjQuIGJv b3Rmc18wMDdfbmVnCShGQUlMKQ0KICAgU3VnZ2VzdCB0byByZW1vdmUgdGhpcyBjYXNlIHNpbmNl IEVGSSBsYWJlbGVkIGxpbWl0YXRpb24gaXMgZm9yIFNVTiBvbmx5Lg0KICAgKCBJbiBsaWJ6cG9v bC5jOiB6cG9vbF92YWxpZF9wcm9wbGlzdCgpICkNCiAgIA0KICAgTW9kaWZpZWQgOiBOb25lDQoN CjUuIGJvb3Rmc18wMDhfbmVnIChQQVNTLCBhZGQgbmV3IGNhc2UpDQoJIFNldHRpbmcgYm9vdGZz IG9uIGEgZGF0YXNldCB3aGljaCBoYXMgemxlIGNvbXByZXNzaW9uIGVuYWJsZWQgd2lsbCBhbHNv IGZhaWwNCgkgDQoJIE1vZGlmaWVkIDogLlx0ZXN0c1xmdW5jdGlvbmFsXGJvb3Rmc1xib290ZnNf MDA4X25lZw0KCSANCmNhY2hlDQo9PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PQkg DQoxLiBjYWNoZV8wMDlfcG9zIChGQUlMKQ0KICAgVGhlIGNhY2hlIGRldmljZSBuYW1lIHdpbGwg YmUgcmVwbGFjZWQgd2l0aCBudW1iZXJzIGFmdGVyIHNldHRpbmcgdG8gT0ZGTElORSBpbiBmcmVl YnNkLg0KICAgKCBTb2xhcmlzIDogZGE0IE9OTElORSAtPiBkYTQgT0ZGTElORQ0KICAgICBGcmVl YnNkIDogZGE0IE9OTElORSAtPiAxMDc3NjA1MTcxNzE5MzEwOTc5NiBPRkZMSU5FKQ0KCSANCgkg U29sdXRpb24gOiBVc2UgYW5vdGhlciBwYXJzZSBtZXRob2QgdG8gZ2V0IE9GRkxJTkUgY2FjaGUg ZGV2aWNlLg0KICAgTW9kaWZpZWQgOiAuXGNvbW1hbmRzLmNmZw0KICAgCQkJCQkJLlxjb21tYW5k cw0KICAgCQkJCQkJLlx0ZXN0c1xmdW5jdGlvbmFsXGNhY2hlXGNhY2hlLmtzaGxpYg0KICAgDQoy LiBjYWNoZV8wMTBfbmVnIChGQUlMKQ0KICAgQS4gRnJlZWJzZCBkb2Vzbid0IGhhdmUgImxvZmlh ZG0iIGNvbW1hbmQsIGl0IHVzZSAibWRjb25maWciIGluc3RlYWQuDQogICAgICBTb2x1dGlvbiA6 IFVzZSBtZGNvbmZpZyB0byB2ZXJpZnkgaWYgaXQgY2FuIGJlIHVzZWQgYXMgY2FjaGUgZGV2aWNl cy4NCiAgICAgIA0KICAgQi4gImxvZmlhZG0iIGNhbm5vdCBiZSB1c2VkIGFzIGNhY2hlIGRldmlj ZSBpbiBTb2xhcmlzLCBidXQgIm1kY29uZmlnIiBpbiBGcmVlYnNkIGlzIG9rLg0KICAgICAgU29s dXRpb24gOiBNb2RpZnkgbG9nX211c3Rub3QgdG8gbG9nX211c3QuDQoNCiAgIE1vZGlmaWVkIDog Llx0ZXN0c1xmdW5jdGlvbmFsXGNhY2hlXGNhY2hlXzAxMF9uZWc= --bcaec51b15f1649aab04cd3ecad6-- From owner-zfs-devel@FreeBSD.ORG Sun Dec 23 18:53:41 2012 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 7F7A6962; Sun, 23 Dec 2012 18:53:41 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from mail.vx.sk (mail.vx.sk [176.9.45.25]) by mx1.freebsd.org (Postfix) with ESMTP id 15B4A8FC13; Sun, 23 Dec 2012 18:53:37 +0000 (UTC) Received: from core.vx.sk (localhost [127.0.0.2]) by mail.vx.sk (Postfix) with ESMTP id 324E0F1F8; Sun, 23 Dec 2012 19:53:31 +0100 (CET) X-Virus-Scanned: amavisd-new at mail.vx.sk Received: from mail.vx.sk by core.vx.sk (amavisd-new, unix socket) with LMTP id 6C0VieW38ERr; Sun, 23 Dec 2012 19:53:25 +0100 (CET) Received: from [10.9.8.1] (188-167-78-15.dynamic.chello.sk [188.167.78.15]) by mail.vx.sk (Postfix) with ESMTPSA id D4493F1EC; Sun, 23 Dec 2012 19:53:24 +0100 (CET) Message-ID: <50D75325.4020206@FreeBSD.org> Date: Sun, 23 Dec 2012 19:53:25 +0100 From: Martin Matuska User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:17.0) Gecko/17.0 Thunderbird/17.0 MIME-Version: 1.0 To: zfs-devel@FreeBSD.org Subject: Removal of zfs v14 compat code X-Enigmail-Version: 1.4.6 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Cc: Xin Li X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 23 Dec 2012 18:53:41 -0000 Hi guys, do you still see reasons to keep the ZFS v14 compat code in the tree? I guess we definitely don't need it in HEAD and surely not in stable/9. The only place where it currently makes any sense is stable/8. If there are no objections, I will remove it from HEAD and schedule it for stable/9 MFC. Cheers, mm -- Martin Matuska FreeBSD committer http://blog.vx.sk From owner-zfs-devel@FreeBSD.ORG Tue Jan 8 21:08:50 2013 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 04380945 for ; Tue, 8 Jan 2013 21:08:50 +0000 (UTC) (envelope-from delphij@delphij.net) Received: from anubis.delphij.net (anubis.delphij.net [64.62.153.212]) by mx1.freebsd.org (Postfix) with ESMTP id DD5F4B26 for ; Tue, 8 Jan 2013 21:08:49 +0000 (UTC) Received: from epsilon.delphij.net (drawbridge.ixsystems.com [206.40.55.65]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by anubis.delphij.net (Postfix) with ESMTPSA id C6DE320E56; Tue, 8 Jan 2013 13:08:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=delphij.net; s=anubis; t=1357679328; bh=9g1Aq7i1VizumBLOfhi+BakKW+naNPT2fsb9jgl4Gss=; h=Date:From:Reply-To:To:Subject; b=4Di39O+Qap9JpdnBqGo3MgSXLl8BGXm2wansAuxqtmmT6F689Z+WglpwTB2LhyYcT mmsAu8XctqLtqwfXV4JCs2mLlxFYyTKF5evKwUfvMRcduy4SkDlS6E0ca5JTJkBtWA WOej/dxr/ZWubCXXT/4PhwS6Uic1J2VqeCxHWXi0= Message-ID: <50EC8AE0.4000101@delphij.net> Date: Tue, 08 Jan 2013 13:08:48 -0800 From: Xin Li Organization: The FreeBSD Project MIME-Version: 1.0 To: zfs-devel@freebsd.org Subject: [PATCH RFC] Fix panic on zap_count(os, object, &count) != 0 X-Enigmail-Version: 1.4.6 Content-Type: multipart/mixed; boundary="------------030105090007010503070501" X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list Reply-To: d@delphij.net List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Jan 2013 21:08:50 -0000 This is a multi-part message in MIME format. --------------030105090007010503070501 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 Hi, I'd like to port ZFS on Linux changeset e8fd45a0f975c6b8ae8cd644714fc21f14fac2bf. We were able to create a pool with damage at lab that panics the system when doing zpool import - -F -X -f, and this change fixes that. Any objections? Cheers, - -- Xin LI https://www.delphij.net/ FreeBSD - The Power to Serve! Live free or die -----BEGIN PGP SIGNATURE----- iQEcBAEBCAAGBQJQ7IrfAAoJEG80Jeu8UPuzBR8H+wZUozdz35LsPiVfrTvHFwmo HGKgS2T6bpQmmvOkuJhvFbvS0B2+ZElsgn+qamkCdZ5A3Er5VaAkt9VM0FDNGvOG M49lYJzC/J9vn+B0QCZFwLw5cXEzcAXa+zE5j6v6SYv2sgESDKo04u745paRtC+P zLK1mtUPWhQdn3PMYBOYF+2dvcFY5XkNgXaerJWJs2LQpjeV1hzQH0Wy/cJ3J0lY dp/eUybTxqafw8uTfwIre/tNPowxYOmDWo+Ht1bflKFZ3NCSxpGZLYxs69Fl5pO8 OeFrcwpoHRXxY8oMrZ424FO3+l3MPjdDozNT3ciNilBs+KfMypPjKsMmb+w50zw= =rfqC -----END PGP SIGNATURE----- --------------030105090007010503070501 Content-Type: text/plain; charset=UTF-8; name="zapcount.diff" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="zapcount.diff" Index: cddl/contrib/opensolaris/cmd/zdb/zdb.c =================================================================== --- cddl/contrib/opensolaris/cmd/zdb/zdb.c (revision 245136) +++ cddl/contrib/opensolaris/cmd/zdb/zdb.c (working copy) @@ -704,7 +704,9 @@ dump_ddt(ddt_t *ddt, enum ddt_type type, enum ddt_ return; ASSERT(error == 0); - if ((count = ddt_object_count(ddt, type, class)) == 0) + error = ddt_object_count(ddt, type, class, &count); + ASSERT(error == 0); + if (count == 0) return; dspace = doi.doi_physical_blocks_512 << 9; Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/ddt.c =================================================================== --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/ddt.c (revision 245136) +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/ddt.c (working copy) @@ -89,12 +89,13 @@ ddt_object_destroy(ddt_t *ddt, enum ddt_type type, spa_t *spa = ddt->ddt_spa; objset_t *os = ddt->ddt_os; uint64_t *objectp = &ddt->ddt_object[type][class]; + uint64_t count; char name[DDT_NAMELEN]; ddt_object_name(ddt, type, class, name); ASSERT(*objectp != 0); - ASSERT(ddt_object_count(ddt, type, class) == 0); + VERIFY(ddt_object_count(ddt, type, class, &count) == 0 && count == 0); ASSERT(ddt_histogram_empty(&ddt->ddt_histogram[type][class])); VERIFY(zap_remove(os, DMU_POOL_DIRECTORY_OBJECT, name, tx) == 0); VERIFY(zap_remove(os, spa->spa_ddt_stat_object, name, tx) == 0); @@ -109,6 +110,7 @@ ddt_object_load(ddt_t *ddt, enum ddt_type type, en { ddt_object_t *ddo = &ddt->ddt_object_stats[type][class]; dmu_object_info_t doi; + uint64_t count; char name[DDT_NAMELEN]; int error; @@ -129,7 +131,11 @@ ddt_object_load(ddt_t *ddt, enum ddt_type type, en */ VERIFY(ddt_object_info(ddt, type, class, &doi) == 0); - ddo->ddo_count = ddt_object_count(ddt, type, class); + error = ddt_object_count(ddt, type, class, &count); + if (error) + return error; + + ddo->ddo_count = count; ddo->ddo_dspace = doi.doi_physical_blocks_512 << 9; ddo->ddo_mspace = doi.doi_fill_count * doi.doi_data_block_size; @@ -143,6 +149,7 @@ ddt_object_sync(ddt_t *ddt, enum ddt_type type, en { ddt_object_t *ddo = &ddt->ddt_object_stats[type][class]; dmu_object_info_t doi; + uint64_t count; char name[DDT_NAMELEN]; ddt_object_name(ddt, type, class, name); @@ -155,8 +162,9 @@ ddt_object_sync(ddt_t *ddt, enum ddt_type type, en * Cache DDT statistics; this is the only time they'll change. */ VERIFY(ddt_object_info(ddt, type, class, &doi) == 0); + VERIFY(ddt_object_count(ddt, type, class, &count) == 0); - ddo->ddo_count = ddt_object_count(ddt, type, class); + ddo->ddo_count = count; ddo->ddo_dspace = doi.doi_physical_blocks_512 << 9; ddo->ddo_mspace = doi.doi_fill_count * doi.doi_data_block_size; } @@ -213,13 +221,13 @@ ddt_object_walk(ddt_t *ddt, enum ddt_type type, en ddt->ddt_object[type][class], dde, walk)); } -uint64_t -ddt_object_count(ddt_t *ddt, enum ddt_type type, enum ddt_class class) +int +ddt_object_count(ddt_t *ddt, enum ddt_type type, enum ddt_class class, uint64_t *count) { ASSERT(ddt_object_exists(ddt, type, class)); return (ddt_ops[type]->ddt_op_count(ddt->ddt_os, - ddt->ddt_object[type][class])); + ddt->ddt_object[type][class], count)); } int @@ -1079,11 +1087,13 @@ ddt_sync_table(ddt_t *ddt, dmu_tx_t *tx, uint64_t } for (enum ddt_type type = 0; type < DDT_TYPES; type++) { - uint64_t count = 0; + uint64_t add, count = 0; for (enum ddt_class class = 0; class < DDT_CLASSES; class++) { if (ddt_object_exists(ddt, type, class)) { ddt_object_sync(ddt, type, class, tx); - count += ddt_object_count(ddt, type, class); + VERIFY(ddt_object_count(ddt, type, class, + &add) == 0); + count += add; } } for (enum ddt_class class = 0; class < DDT_CLASSES; class++) { Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/ddt_zap.c =================================================================== --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/ddt_zap.c (revision 245136) +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/ddt_zap.c (working copy) @@ -133,14 +133,11 @@ ddt_zap_walk(objset_t *os, uint64_t object, ddt_en return (error); } -static uint64_t -ddt_zap_count(objset_t *os, uint64_t object) +static int +ddt_zap_count(objset_t *os, uint64_t object, uint64_t *count) { - uint64_t count = 0; - VERIFY(zap_count(os, object, &count) == 0); - - return (count); + return (zap_count(os, object, count)); } const ddt_ops_t ddt_zap_ops = { Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/ddt.h =================================================================== --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/ddt.h (revision 245136) +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/ddt.h (working copy) @@ -163,7 +163,7 @@ typedef struct ddt_ops { dmu_tx_t *tx); int (*ddt_op_walk)(objset_t *os, uint64_t object, ddt_entry_t *dde, uint64_t *walk); - uint64_t (*ddt_op_count)(objset_t *os, uint64_t object); + int (*ddt_op_count)(objset_t *os, uint64_t object, uint64_t *count); } ddt_ops_t; #define DDT_NAMELEN 80 @@ -172,8 +172,8 @@ extern void ddt_object_name(ddt_t *ddt, enum ddt_t enum ddt_class cls, char *name); extern int ddt_object_walk(ddt_t *ddt, enum ddt_type type, enum ddt_class cls, uint64_t *walk, ddt_entry_t *dde); -extern uint64_t ddt_object_count(ddt_t *ddt, enum ddt_type type, - enum ddt_class cls); +extern int ddt_object_count(ddt_t *ddt, enum ddt_type type, + enum ddt_class class, uint64_t *count); extern int ddt_object_info(ddt_t *ddt, enum ddt_type type, enum ddt_class cls, dmu_object_info_t *); extern boolean_t ddt_object_exists(ddt_t *ddt, enum ddt_type type, --------------030105090007010503070501-- From owner-zfs-devel@FreeBSD.ORG Sun Jan 13 08:34:57 2013 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 7FC18A3B; Sun, 13 Jan 2013 08:34:57 +0000 (UTC) (envelope-from delphij@delphij.net) Received: from anubis.delphij.net (anubis.delphij.net [IPv6:2001:470:1:117::25]) by mx1.freebsd.org (Postfix) with ESMTP id 56D37B60; Sun, 13 Jan 2013 08:34:57 +0000 (UTC) Received: from Xins-MacBook-Pro-2.local (unknown [IPv6:2001:470:83bf:0:95fd:278:2967:9baa]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by anubis.delphij.net (Postfix) with ESMTPSA id D580FAFE9; Sun, 13 Jan 2013 00:34:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=delphij.net; s=anubis; t=1358066097; bh=im53hPjUUGIndoN2JHLeiSbAMU9Mo8mhyZBmEsLpjRg=; h=Date:From:Reply-To:To:CC:Subject:References:In-Reply-To; b=cZqL+jASYNgcZKH9LUNIfYKPQf+92BR4sA+5zJiDKsB/JcanGSvUX4IVFehWQnFXm 9lHzS+sJ1jog4LDuzzaL6ZrUO9QPPCOdoyIT6jdK4AomNEZGsUBlGDUlypl0P47bQQ MXQ4H5By6UemmxpbvPkbMT+HyLD0jQV6bdbcD/8I= Message-ID: <50F271AF.4000704@delphij.net> Date: Sun, 13 Jan 2013 00:34:55 -0800 From: Xin Li Organization: The FreeBSD Project MIME-Version: 1.0 To: Martin Matuska Subject: Re: Removal of zfs v14 compat code References: <50D75325.4020206@FreeBSD.org> In-Reply-To: <50D75325.4020206@FreeBSD.org> X-Enigmail-Version: 1.5 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Cc: Xin Li , zfs-devel@FreeBSD.org X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list Reply-To: d@delphij.net List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 13 Jan 2013 08:34:57 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 On 12/23/12 10:53 AM, Martin Matuska wrote: > Hi guys, > > do you still see reasons to keep the ZFS v14 compat code in the > tree? I guess we definitely don't need it in HEAD and surely not in > stable/9. The only place where it currently makes any sense is > stable/8. > > If there are no objections, I will remove it from HEAD and schedule > it for stable/9 MFC. I think it's probably safe to do it as stable/8 have the compat code, so it can act as the "milestone" stable branch if people upgrade from earlier FreeBSD versions. If you do it, please make sure to mention it in UPDATING. This also breaks freebsd-update upgrade from 7.x to 9.x, keep in mind. If I would do it I'd wait to a point that getting rid of it is necessary for some changes. Cheers, -----BEGIN PGP SIGNATURE----- iQEcBAEBCAAGBQJQ8nGvAAoJEG80Jeu8UPuzoSEH/22DWDbxmQcd0c0etQweogR0 tiN83J1hTeJTfmqWf+u9++0NpWxuiv6YIy+LIn/CuW329kd538WuKRSeu6J6U0P2 ymQrpCpKJPoAvS9aHWgPH1LTAM6HjvZS0cvzMzuqvSLxihHD74WMbX+sEJmaibBT x2eoWTQdZFFY9huWPuWtuyFqtDfRzXgqwznnfNDKZd4kijGiXisyDt9jlXWZAZvB PubCNglqfuxM0bVGPGGHi4Hr5FLKGSGSgL/7gOAKDe8tdCULjWhBqFexYqFHVdvI 3mgxXgqlUt/8qQioQbOXxKmhBWTmKIZ4yB35dH3mgHDtTGEmBwZcqV0nOouTtT8= =fM64 -----END PGP SIGNATURE----- From owner-zfs-devel@FreeBSD.ORG Sat Jan 19 17:01:07 2013 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id F26E8F3D; Sat, 19 Jan 2013 17:01:06 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 96B057A5; Sat, 19 Jan 2013 17:01:05 +0000 (UTC) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id TAA00114; Sat, 19 Jan 2013 19:00:57 +0200 (EET) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1Twbmq-0004Mx-Pt; Sat, 19 Jan 2013 19:00:56 +0200 Message-ID: <50FAD145.10906@FreeBSD.org> Date: Sat, 19 Jan 2013 19:00:53 +0200 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:17.0) Gecko/17.0 Thunderbird/17.0 MIME-Version: 1.0 To: Gavin Atkinson Subject: Re: ZFS lock up 9-stable r244911 (Jan) References: In-Reply-To: X-Enigmail-Version: 1.4.6 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Mailman-Approved-At: Sat, 19 Jan 2013 17:13:30 +0000 Cc: freebsd-fs@FreeBSD.org X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 19 Jan 2013 17:01:07 -0000 on 18/01/2013 17:01 Gavin Atkinson said the following: > > Hi all, > > I have a machine on which ZFS appears to have locked up, and any processes > that attempt to access the ZFS filesystem. This machine is running > 9-stable amd64 r244911 (though from cvs, not SVN), and therefore I believe > has all of avg's ZFS deadlock patches. > > This machine has both UFS and ZFS filesystems. All of the "system" > filesystems are on UFS, and as a result the machine itself is responsive > and I can investigate state using kgdb against the live kernel. I've > included all thread backtraces, a couple of other bits relating to held > locks, and ps/sysctl output at > http://people.freebsd.org/~gavin/tay-zfs-hang.txt > http://people.freebsd.org/~gavin/tay-sysctl-a.txt > http://people.freebsd.org/~gavin/tay-ps-auxwwwH.txt > > This machine was in use as a pkgng package builder, using poudriere. > Poudriere makes heavy use of zfs filesystems within jails, including "zfs > get", "zfs set", "zfs snapshot", "zfs rollback", "zfs diff" and other > commands, although there do not appear to be any instances of the zfs > process running currently. At the time of the hang 16 parallel builds were > in progress, > > The underlying disk subsystem is a single hardware RAID-10 on a twa > controller, and the zpool is on a single partition of this device. The > RAID-10 itself is intact, the controller reports no errors. There is no > L2ARC or separate ZIL. The UFS filesystems (still seem to be fully > functional) are on separate partitions on the same underlying device, so I > do not believe the underlying storage is having issues. I can "dd" from > the underlying ZFS partition without problem. Nothing has been logged to > /var/log/messages. Based on the above information plus some additional debugging information that Gavin has kindly provided to me I've developed the following "theory" to explain this deadlock. I believe that was very high disk load (overwhelmingly high load) before the deadlock occurred. Further I think that there was a substantial number of high priority writes. Under those conditions the number of in-progress/pending zio-s was constantly at zfs_vdev_max_pending (by default 10). Number of queued zio-s was above hundred: vdev_queue = { vq_deadline_tree = {avl_root = 0xfffffe0338dbb248, avl_compar = 0xffffffff816855b0 , avl_offset = 584, avl_numnodes = 116, avl_size = 896}, vq_read_tree = {avl_root = 0xfffffe019d0b65b0, avl_compar = 0xffffffff81685600 , avl_offset = 560, avl_numnodes = 8, avl_size = 896}, vq_write_tree = { avl_root = 0xfffffe03e3d19230, avl_compar = 0xffffffff81685600 , avl_offset = 560, avl_numnodes = 108, avl_size = 896}, vq_pending_tree = {avl_root = 0xfffffe025e32c230, avl_compar = 0xffffffff81685600 , avl_offset = 560, avl_numnodes = 10, avl_size = 896}, vq_lock = {lock_object = {lo_name = 0xffffffff8172afc9 "vq->vq_lock", lo_flags = 40960000, lo_data = 0, lo_witness = 0x0}, sx_lock = 1}}, vdev_cache = {vc_offset_tree = {avl_root = 0x0, avl_compar = 0xffffffff81681740 , avl_offset = 24, avl_numnodes = 0, avl_size = 88}, vc_lastused_tree = { avl_root = 0x0, avl_compar = 0xffffffff81681760 , avl_offset = 48, avl_numnodes = 0, avl_size = 88} Apparently processing of zio-s was so lagging behind that some executed zio-s triggered "late arrival" condition. My incomplete understanding shows here - I am not sure what exactly triggers the condition and what is so special about it, but from the following stack traces we can see that all five of zio_write_intr_high taskqueue threads were executing dmu_sync_late_arrival_done(): 0 100432 kernel zio_write_intr_h mi_switch+0x186 sleepq_wait+0x42 _sx_xlock_hard+0x426 _sx_xlock+0x51 txg_rele_to_sync+0x36 dmu_tx_commit+0xf1 dmu_sync_late_arrival_done+0x52 zio_done+0x353 zio_execute+0xc3 zio_done+0x3d0 zio_execute+0xc3 taskqueue_run_locked+0x74 taskqueue_thread_loop+0x46 fork_exit+0x11f fork_trampoline+0xe 0 100433 kernel zio_write_intr_h mi_switch+0x186 sleepq_wait+0x42 _sx_xlock_hard+0x426 _sx_xlock+0x51 txg_rele_to_sync+0x36 dmu_tx_commit+0xf1 dmu_sync_late_arrival_done+0x52 zio_done+0x353 zio_execute+0xc3 zio_done+0x3d0 zio_execute+0xc3 taskqueue_run_locked+0x74 taskqueue_thread_loop+0x46 fork_exit+0x11f fork_trampoline+0xe 0 100434 kernel zio_write_intr_h mi_switch+0x186 sleepq_wait+0x42 _sx_xlock_hard+0x426 _sx_xlock+0x51 txg_rele_to_sync+0x36 dmu_tx_commit+0xf1 dmu_sync_late_arrival_done+0x52 zio_done+0x353 zio_execute+0xc3 zio_done+0x3d0 zio_execute+0xc3 taskqueue_run_locked+0x74 taskqueue_thread_loop+0x46 fork_exit+0x11f fork_trampoline+0xe 0 100435 kernel zio_write_intr_h mi_switch+0x186 sleepq_wait+0x42 _sx_xlock_hard+0x426 _sx_xlock+0x51 txg_rele_to_sync+0x36 dmu_tx_commit+0xf1 dmu_sync_late_arrival_done+0x52 zio_done+0x353 zio_execute+0xc3 zio_done+0x3d0 zio_execute+0xc3 taskqueue_run_locked+0x74 taskqueue_thread_loop+0x46 fork_exit+0x11f fork_trampoline+0xe 0 100436 kernel zio_write_intr_h mi_switch+0x186 sleepq_wait+0x42 _sx_xlock_hard+0x426 _sx_xlock+0x51 txg_rele_to_sync+0x36 dmu_tx_commit+0xf1 dmu_sync_late_arrival_done+0x52 zio_done+0x353 zio_execute+0xc3 zio_done+0x3d0 zio_execute+0xc3 taskqueue_run_locked+0x74 taskqueue_thread_loop+0x46 fork_exit+0x11f fork_trampoline+0xe In additional to the above, the taskqueue associated with the above threads has another 9 pending tasks. As you can see that "late arrival" code path involves txg_rele_to_sync() where an instance tc_lock is acquired. Further, it looks that tc_lock instances are held by the following threads: 64998 101921 pkg initial thread mi_switch+0x186 sleepq_wait+0x42 _sx_xlock_hard+0x426 _sx_xlock+0x51 txg_delay+0x9d dsl_pool_tempreserve_space+0xd5 dsl_dir_tempreserve_space+0x154 dmu_tx_assign+0x370 zfs_freebsd_create+0x310 VOP_CREATE_APV+0x31 vn_open_cred+0x4b7 kern_openat+0x20a amd64_syscall+0x540 Xfast_syscall+0xf7 66152 102491 pkg initial thread mi_switch+0x186 sleepq_wait+0x42 _sx_xlock_hard+0x426 _sx_xlock+0x51 txg_delay+0x9d dsl_pool_tempreserve_space+0xd5 dsl_dir_tempreserve_space+0x154 dmu_tx_assign+0x370 zfs_freebsd_write+0x45b VOP_WRITE_APV+0xb2 vn_write+0x37e vn_io_fault+0x90 dofilewrite+0x85 kern_writev+0x6c sys_write+0x64 amd64_syscall+0x540 Xfast_syscall+0xf7 75803 101638 find - mi_switch+0x186 sleepq_wait+0x42 _sx_xlock_hard+0x426 _sx_xlock+0x51 txg_delay+0x9d dsl_pool_tempreserve_space+0xd5 dsl_dir_tempreserve_space+0x154 dmu_tx_assign+0x370 zfs_inactive+0x1b7 zfs_freebsd_inactive+0x1a vinactive+0x86 vputx+0x2d8 sys_fchdir+0x356 amd64_syscall+0x540 Xfast_syscall+0xf7 75809 102932 find - mi_switch+0x186 sleepq_wait+0x42 _sx_xlock_hard+0x426 _sx_xlock+0x51 txg_delay+0x9d dsl_pool_tempreserve_space+0xd5 dsl_dir_tempreserve_space+0x154 dmu_tx_assign+0x370 zfs_inactive+0x1b7 zfs_freebsd_inactive+0x1a vinactive+0x86 vputx+0x2d8 sys_fchdir+0x356 amd64_syscall+0x540 Xfast_syscall+0xf7 75813 101515 find - mi_switch+0x186 sleepq_wait+0x42 _sx_xlock_hard+0x426 _sx_xlock+0x51 txg_delay+0x9d dsl_pool_tempreserve_space+0xd5 dsl_dir_tempreserve_space+0x154 dmu_tx_assign+0x370 zfs_inactive+0x1b7 zfs_freebsd_inactive+0x1a vinactive+0x86 vputx+0x2d8 sys_fchdir+0x356 amd64_syscall+0x540 Xfast_syscall+0xf7 77468 101412 update-mime-databas initial thread mi_switch+0x186 sleepq_wait+0x42 _sx_xlock_hard+0x426 _sx_xlock+0x51 txg_delay+0x9d dsl_pool_tempreserve_space+0xd5 dsl_dir_tempreserve_space+0x154 dmu_tx_assign+0x370 zfs_freebsd_write+0x45b VOP_WRITE_APV+0xb2 vn_write+0x37e vn_io_fault+0x90 dofilewrite+0x85 kern_writev+0x6c sys_write+0x64 amd64_syscall+0x540 Xfast_syscall+0xf7 These threads calls txg_delay also because of the high load. In the code we see that dmu_tx_assign first grabs an instance of tc_lock and then calls dsl_dir_tempreserve_space. Also, we see that txg_delay tries to acquire tx_sync_lock and that's where all these threads are block. Then we see that txg_sync_thread holds tx_sync_lock, but in its turn it is blocked waiting for its zio: 1552 100544 zfskern txg_thread_enter mi_switch+0x186 sleepq_wait+0x42 _cv_wait+0x112 zio_wait+0x61 dbuf_read+0x5e5 dmu_buf_hold+0xe0 zap_lockdir+0x58 zap_lookup_norm+0x45 zap_lookup+0x2e feature_get_refcount+0x4b spa_feature_is_active+0x52 dsl_scan_active+0x63 txg_sync_thread+0x20d fork_exit+0x11f fork_trampoline+0xe So a summary. For some completed zio-s their zio_done routines are blocked because of dmu_sync_late_arrival_done->txg_rele_to_sync->tc_lock. tc_locks are held by threads in dmu_tx_assign->..->txg_delay where txg_delay is blocked on tx_sync_lock. tx_sync_lock is held by txg_sync_thread which waits for its zio to be processed. The zio is held on queue and is not getting processed because the vdev already has too many pending/in-progress zio-s. This theory looks plausible to me, but I'd like to hear what the experts think. Even more important question is how this situation can be avoided. -- Andriy Gapon From owner-zfs-devel@FreeBSD.ORG Sun Feb 3 17:32:00 2013 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 98FDF1DD for ; Sun, 3 Feb 2013 17:32:00 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id A3638232 for ; Sun, 3 Feb 2013 17:31:59 +0000 (UTC) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id TAA09478 for ; Sun, 03 Feb 2013 19:31:58 +0200 (EET) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1U23Q5-0006oE-Sl for zfs-devel@FreeBSD.org; Sun, 03 Feb 2013 19:31:57 +0200 Message-ID: <510E9F0D.7070407@FreeBSD.org> Date: Sun, 03 Feb 2013 19:31:57 +0200 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:17.0) Gecko/20130121 Thunderbird/17.0.2 MIME-Version: 1.0 To: zfs-devel@FreeBSD.org Subject: move vnode creation from zfs_znode_cache_constructor to zfs_znode_alloc X-Enigmail-Version: 1.4.6 Content-Type: text/plain; charset=X-VIET-VPS Content-Transfer-Encoding: 7bit X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 03 Feb 2013 17:32:00 -0000 Please review the following proposed change. Thank you. commit db54bca1cc711e5632b26da3f9591ca62a9d7c87 Author: Andriy Gapon Date: Fri Nov 16 11:54:04 2012 +0200 zfs: move vnode creation from zfs_znode_cache_constructor to zfs_znode_alloc All other places where a znode is allocated do not need z_vnode at all. These are: - zfs_create_share_dir - zfs_create_fs This chnage ensures two things: - VN_LOCK_ASHARE is not erroneously called for VFIFO vnodes - vn_lock is called on a fully constructed vnode with correct v_ops The change also allows to make zfs_znode_cache_constructor a normal kmem_cache constructor again (as it is in upstream). This allows to avoid a potential problem where zfs_znode_cache_destructor may be called on un-constructed znodes. With the new arrangement we can call getnewvnode after all the necessary znode checks thus removing a need for zfs_vnode_forget. This change required moving vnode manipulations from zfs_znode_sa_init to zfs_znode_alloc. diff --git a/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_znode.c b/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_znode.c index 7d7b20a..c2ea9ef 100644 --- a/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_znode.c +++ b/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_znode.c @@ -113,37 +113,12 @@ extern struct vop_vector zfs_vnodeops; extern struct vop_vector zfs_fifoops; extern struct vop_vector zfs_shareops; -/* - * XXX: We cannot use this function as a cache constructor, because - * there is one global cache for all file systems and we need - * to pass vfsp here, which is not possible, because argument - * 'cdrarg' is defined at kmem_cache_create() time. - */ -/*ARGSUSED*/ static int zfs_znode_cache_constructor(void *buf, void *arg, int kmflags) { znode_t *zp = buf; - vnode_t *vp; - vfs_t *vfsp = arg; - int error; POINTER_INVALIDATE(&zp->z_zfsvfs); - ASSERT(!POINTER_IS_VALID(zp->z_zfsvfs)); - - if (vfsp != NULL) { - error = getnewvnode("zfs", vfsp, &zfs_vnodeops, &vp); - if (error != 0 && (kmflags & KM_NOSLEEP)) - return (-1); - ASSERT(error == 0); - vn_lock(vp, LK_EXCLUSIVE | LK_RETRY); - zp->z_vnode = vp; - vp->v_data = (caddr_t)zp; - VN_LOCK_AREC(vp); - VN_LOCK_ASHARE(vp); - } else { - zp->z_vnode = NULL; - } list_link_init(&zp->z_link_node); @@ -158,7 +133,9 @@ zfs_znode_cache_constructor(void *buf, void *arg, int kmflags) zp->z_dirlocks = NULL; zp->z_acl_cached = NULL; + zp->z_vnode = NULL; zp->z_moved = 0; + return (0); } @@ -377,7 +354,7 @@ zfs_znode_init(void) rw_init(&zfsvfs_lock, NULL, RW_DEFAULT, NULL); ASSERT(znode_cache == NULL); znode_cache = kmem_cache_create("zfs_znode_cache", - sizeof (znode_t), 0, /* zfs_znode_cache_constructor */ NULL, + sizeof (znode_t), 0, zfs_znode_cache_constructor, zfs_znode_cache_destructor, NULL, NULL, NULL, 0); kmem_cache_set_move(znode_cache, zfs_znode_move); } @@ -501,7 +478,9 @@ zfs_create_share_dir(zfsvfs_t *zfsvfs, dmu_tx_t *tx) zfs_acl_ids_t acl_ids; vattr_t vattr; znode_t *sharezp; - vnode_t *vp, vnode; +#ifdef sun + vnode_t *vp; +#endif znode_t *zp; int error; @@ -512,7 +491,6 @@ zfs_create_share_dir(zfsvfs_t *zfsvfs, dmu_tx_t *tx) vattr.va_gid = crgetgid(kcred); sharezp = kmem_cache_alloc(znode_cache, KM_SLEEP); - zfs_znode_cache_constructor(sharezp, zfsvfs->z_parent->z_vfs, 0); ASSERT(!POINTER_IS_VALID(sharezp->z_zfsvfs)); sharezp->z_moved = 0; sharezp->z_unlinked = 0; @@ -520,11 +498,10 @@ zfs_create_share_dir(zfsvfs_t *zfsvfs, dmu_tx_t *tx) sharezp->z_zfsvfs = zfsvfs; sharezp->z_is_sa = zfsvfs->z_use_sa; - sharezp->z_vnode = &vnode; - vnode.v_data = sharezp; - +#ifdef sun vp = ZTOV(sharezp); vp->v_type = VDIR; +#endif VERIFY(0 == zfs_acl_ids_create(sharezp, IS_ROOT_NODE, &vattr, kcred, NULL, &acl_ids)); @@ -536,12 +513,7 @@ zfs_create_share_dir(zfsvfs_t *zfsvfs, dmu_tx_t *tx) zfsvfs->z_shares_dir = sharezp->z_id; zfs_acl_ids_free(&acl_ids); - ZTOV(sharezp)->v_data = NULL; - ZTOV(sharezp)->v_count = 0; - ZTOV(sharezp)->v_holdcnt = 0; - zp->z_vnode = NULL; sa_handle_destroy(sharezp->z_sa_hdl); - sharezp->z_vnode = NULL; kmem_cache_free(znode_cache, sharezp); return (error); @@ -608,11 +580,13 @@ zfs_znode_sa_init(zfsvfs_t *zfsvfs, znode_t *zp, zp->z_is_sa = (obj_type == DMU_OT_SA) ? B_TRUE : B_FALSE; +#ifdef sun /* * Slap on VROOT if we are the root znode */ if (zp->z_id == zfsvfs->z_root) ZTOV(zp)->v_flag |= VROOT; +#endif mutex_exit(&zp->z_lock); vn_exists(ZTOV(zp)); @@ -629,17 +603,6 @@ zfs_znode_dmu_fini(znode_t *zp) zp->z_sa_hdl = NULL; } -static void -zfs_vnode_forget(vnode_t *vp) -{ - - /* copied from insmntque_stddtr */ - vp->v_data = NULL; - vp->v_op = &dead_vnodeops; - vgone(vp); - vput(vp); -} - /* * Construct a new znode/vnode and intialize. * @@ -657,9 +620,9 @@ zfs_znode_alloc(zfsvfs_t *zfsvfs, dmu_buf_t *db, int blksz, uint64_t parent; sa_bulk_attr_t bulk[9]; int count = 0; + int error; zp = kmem_cache_alloc(znode_cache, KM_SLEEP); - zfs_znode_cache_constructor(zp, zfsvfs->z_parent->z_vfs, 0); ASSERT(zp->z_dirlocks == NULL); ASSERT(!POINTER_IS_VALID(zp->z_zfsvfs)); @@ -678,8 +641,6 @@ zfs_znode_alloc(zfsvfs_t *zfsvfs, dmu_buf_t *db, int blksz, zp->z_seq = 0x7A4653; zp->z_sync_cnt = 0; - vp = ZTOV(zp); - zfs_znode_sa_init(zfsvfs, zp, db, obj_type, hdl); SA_ADD_BULK_ATTR(bulk, count, SA_ZPL_MODE(zfsvfs), NULL, &mode, 8); @@ -701,12 +662,27 @@ zfs_znode_alloc(zfsvfs_t *zfsvfs, dmu_buf_t *db, int blksz, if (sa_bulk_lookup(zp->z_sa_hdl, bulk, count) != 0 || zp->z_gen == 0) { if (hdl == NULL) sa_handle_destroy(zp->z_sa_hdl); - zfs_vnode_forget(vp); - zp->z_vnode = NULL; kmem_cache_free(znode_cache, zp); return (NULL); } + error = getnewvnode("zfs", zfsvfs->z_parent->z_vfs, &zfs_vnodeops, &vp); + ASSERT(error == 0); + if (error != 0) { /* hm, just in case */ + kmem_cache_free(znode_cache, zp); + return (NULL); + } + zp->z_vnode = vp; + vp->v_data = zp; + +#ifdef sun + /* + * Slap on VROOT if we are the root znode + */ + if (zp->z_id == zfsvfs->z_root) + vp->v_flag |= VROOT; +#endif + zp->z_mode = mode; vp->v_type = IFTOVT((mode_t)mode); @@ -749,8 +725,6 @@ zfs_znode_alloc(zfsvfs_t *zfsvfs, dmu_buf_t *db, int blksz, break; #endif /* sun */ } - if (vp->v_type != VFIFO) - VN_LOCK_ASHARE(vp); mutex_enter(&zfsvfs->z_znodes_lock); list_insert_tail(&zfsvfs->z_all_znodes, zp); @@ -762,6 +736,14 @@ zfs_znode_alloc(zfsvfs_t *zfsvfs, dmu_buf_t *db, int blksz, zp->z_zfsvfs = zfsvfs; mutex_exit(&zfsvfs->z_znodes_lock); + /* + * Acquire vnode lock before making it available to the world. + */ + vn_lock(vp, LK_EXCLUSIVE | LK_RETRY); + VN_LOCK_AREC(vp); + if (vp->v_type != VFIFO) + VN_LOCK_ASHARE(vp); + VFS_HOLD(zfsvfs->z_vfs); return (zp); } @@ -1839,7 +1821,9 @@ zfs_create_fs(objset_t *os, cred_t *cr, nvlist_t *zplprops, dmu_tx_t *tx) int error; int i; znode_t *rootzp = NULL; - vnode_t vnode; +#ifdef sun + vnode_t *vp; +#endif vattr_t vattr; znode_t *zp; zfs_acl_ids_t acl_ids; @@ -1918,16 +1902,17 @@ zfs_create_fs(objset_t *os, cred_t *cr, nvlist_t *zplprops, dmu_tx_t *tx) bzero(&zfsvfs, sizeof (zfsvfs_t)); rootzp = kmem_cache_alloc(znode_cache, KM_SLEEP); - zfs_znode_cache_constructor(rootzp, NULL, 0); ASSERT(!POINTER_IS_VALID(rootzp->z_zfsvfs)); rootzp->z_moved = 0; rootzp->z_unlinked = 0; rootzp->z_atime_dirty = 0; rootzp->z_is_sa = USE_SA(version, os); - vnode.v_type = VDIR; - vnode.v_data = rootzp; - rootzp->z_vnode = &vnode; +#ifdef sun + vp = ZTOV(rootzp); + vn_reinit(vp); + vp->v_type = VDIR; +#endif zfsvfs.z_os = os; zfsvfs.z_parent = &zfsvfs; @@ -1966,7 +1951,6 @@ zfs_create_fs(objset_t *os, cred_t *cr, nvlist_t *zplprops, dmu_tx_t *tx) POINTER_INVALIDATE(&rootzp->z_zfsvfs); sa_handle_destroy(rootzp->z_sa_hdl); - rootzp->z_vnode = NULL; kmem_cache_free(znode_cache, rootzp); /* -- Andriy Gapon From owner-zfs-devel@FreeBSD.ORG Sun Feb 3 17:35:59 2013 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id AA82F36A for ; Sun, 3 Feb 2013 17:35:59 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id E6B84260 for ; Sun, 3 Feb 2013 17:35:58 +0000 (UTC) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id TAA09515 for ; Sun, 03 Feb 2013 19:35:57 +0200 (EET) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1U23Tx-0006ot-9l for zfs-devel@FreeBSD.org; Sun, 03 Feb 2013 19:35:57 +0200 Message-ID: <510E9FFC.6000303@FreeBSD.org> Date: Sun, 03 Feb 2013 19:35:56 +0200 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:17.0) Gecko/20130121 Thunderbird/17.0.2 MIME-Version: 1.0 To: zfs-devel@FreeBSD.org Subject: zvol: call zvol_first_open upon the creating of zvol geom X-Enigmail-Version: 1.4.6 Content-Type: text/plain; charset=X-VIET-VPS Content-Transfer-Encoding: 7bit X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 03 Feb 2013 17:35:59 -0000 Please review the following proposed ZFS/ZVOL<->GEOM changes. Thank you. commit 9315659d772b8ebbb9b4bcf16f2d5f5bb8a766bd Author: Andriy Gapon Date: Fri Sep 14 00:34:38 2012 +0300 zvol: call zvol_first_open upon the creating of zvol geom ... and symmetrically call zvol_last_close when the geom is going away. Also, do not needlessly call zvol_size_changed in zvol_first_open. There is no need to blindly mimic Solaris behavior when FreeBSD GEOM interfacing can be more efficient (and proper). Also, this simplifies locking in zvol_open/zvol_close and fixes the really broken behavior in zvol_geom_access of dropping the g_topology_lock. This change introduces a requirement that a zvol's 'minor' (geom) should be destroyed before the zvol itself (as a dataset) can go. Otherwise the minor holds the zvol's dataset busy and thus it can not be destroyed nor its pool can be exported. diff --git a/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_ioctl.c b/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_ioctl.c index 10443fb..8465ea5 100644 --- a/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_ioctl.c +++ b/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_ioctl.c @@ -1291,9 +1291,10 @@ zfs_ioc_pool_destroy(zfs_cmd_t *zc) { int error; zfs_log_history(zc); + zvol_remove_minors(zc->zc_name); error = spa_destroy(zc->zc_name); - if (error == 0) - zvol_remove_minors(zc->zc_name); + if (error != 0) + zvol_create_minors(zc->zc_name); return (error); } @@ -1344,9 +1345,10 @@ zfs_ioc_pool_export(zfs_cmd_t *zc) boolean_t hardforce = (boolean_t)zc->zc_guid; zfs_log_history(zc); + zvol_remove_minors(zc->zc_name); error = spa_export(zc->zc_name, NULL, force, hardforce); - if (error == 0) - zvol_remove_minors(zc->zc_name); + if (error != 0) + zvol_create_minors(zc->zc_name); return (error); } @@ -3272,9 +3274,11 @@ zfs_ioc_destroy(zfs_cmd_t *zc) return (err); } - err = dmu_objset_destroy(zc->zc_name, zc->zc_defer_destroy); - if (zc->zc_objset_type == DMU_OST_ZVOL && err == 0) + if (zc->zc_objset_type == DMU_OST_ZVOL) (void) zvol_remove_minor(zc->zc_name); + err = dmu_objset_destroy(zc->zc_name, zc->zc_defer_destroy); + if (err != 0) + (void) zvol_create_minor(zc->zc_name); return (err); } diff --git a/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zvol.c b/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zvol.c index e5d159b..845702a 100644 --- a/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zvol.c +++ b/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zvol.c @@ -151,6 +151,9 @@ static int zvol_dumpify(zvol_state_t *zv); static int zvol_dump_fini(zvol_state_t *zv); static int zvol_dump_init(zvol_state_t *zv, boolean_t resize); +static int zvol_first_open(zvol_state_t *zv); +static void zvol_last_close(zvol_state_t *zv); + static zvol_state_t *zvol_geom_create(const char *name); static void zvol_geom_run(zvol_state_t *zv); static void zvol_geom_destroy(zvol_state_t *zv); @@ -579,6 +582,9 @@ zvol_create_minor(const char *name) zvol_minors++; + error = zvol_first_open(zv); + ASSERT(error == 0); + mutex_exit(&spa_namespace_lock); zvol_geom_run(zv); @@ -605,6 +611,8 @@ zvol_remove_zv(zvol_state_t *zv) if (zv->zv_total_opens != 0) return (EBUSY); + zvol_last_close(zv); + ZFS_LOG(1, "ZVOL %s destroyed.", zv->zv_name); #ifdef sun @@ -667,8 +675,9 @@ zvol_first_open(zvol_state_t *zv) } zv->zv_volsize = volsize; zv->zv_zilog = zil_open(os, zvol_get_data); +#ifdef sun zvol_size_changed(zv); - +#endif VERIFY(dsl_prop_get_integer(zv->zv_name, "readonly", &readonly, NULL) == 0); if (readonly || dmu_objset_is_snapshot(os) || @@ -887,42 +896,12 @@ out: /*ARGSUSED*/ static int -zvol_open(struct g_provider *pp, int flag, int count) +zvol_open(zvol_state_t *zv, int flag, int count) { - zvol_state_t *zv; int err = 0; - boolean_t locked = B_FALSE; - - /* - * Protect against recursively entering spa_namespace_lock - * when spa_open() is used for a pool on a (local) ZVOL(s). - * This is needed since we replaced upstream zfsdev_state_lock - * with spa_namespace_lock in the ZVOL code. - * We are using the same trick as spa_open(). - * Note that calls in zvol_first_open which need to resolve - * pool name to a spa object will enter spa_open() - * recursively, but that function already has all the - * necessary protection. - */ - if (!MUTEX_HELD(&spa_namespace_lock)) { - mutex_enter(&spa_namespace_lock); - locked = B_TRUE; - } - zv = pp->private; - if (zv == NULL) { - if (locked) - mutex_exit(&spa_namespace_lock); - return (ENXIO); - } + ASSERT(zv != NULL); - if (zv->zv_total_opens == 0) - err = zvol_first_open(zv); - if (err) { - if (locked) - mutex_exit(&spa_namespace_lock); - return (err); - } if ((flag & FWRITE) && (zv->zv_flags & ZVOL_RDONLY)) { err = EROFS; goto out; @@ -942,38 +921,17 @@ zvol_open(struct g_provider *pp, int flag, int count) #endif zv->zv_total_opens += count; - if (locked) - mutex_exit(&spa_namespace_lock); - return (err); out: - if (zv->zv_total_opens == 0) - zvol_last_close(zv); - if (locked) - mutex_exit(&spa_namespace_lock); return (err); } /*ARGSUSED*/ static int -zvol_close(struct g_provider *pp, int flag, int count) +zvol_close(zvol_state_t *zv, int flag, int count) { - zvol_state_t *zv; - int error = 0; - boolean_t locked = B_FALSE; - /* See comment in zvol_open(). */ - if (!MUTEX_HELD(&spa_namespace_lock)) { - mutex_enter(&spa_namespace_lock); - locked = B_TRUE; - } - - zv = pp->private; - if (zv == NULL) { - if (locked) - mutex_exit(&spa_namespace_lock); - return (ENXIO); - } + ASSERT(zv != NULL); if (zv->zv_flags & ZVOL_EXCL) { ASSERT(zv->zv_total_opens == 1); @@ -991,12 +949,7 @@ zvol_close(struct g_provider *pp, int flag, int count) */ zv->zv_total_opens -= count; - if (zv->zv_total_opens == 0) - zvol_last_close(zv); - - if (locked) - mutex_exit(&spa_namespace_lock); - return (error); + return (0); } static void @@ -2052,6 +2005,7 @@ zvol_geom_destroy(zvol_state_t *zv) static int zvol_geom_access(struct g_provider *pp, int acr, int acw, int ace) { + zvol_state_t *zv; int count, error, flags; g_topology_assert(); @@ -2090,12 +2044,14 @@ zvol_geom_access(struct g_provider *pp, int acr, int acw, int ace) if (acw != 0) flags |= FWRITE; - g_topology_unlock(); + zv = pp->private; + if (zv == NULL) + return (ENXIO); + if (count > 0) - error = zvol_open(pp, flags, count); + error = zvol_open(zv, flags, count); else - error = zvol_close(pp, flags, -count); - g_topology_lock(); + error = zvol_close(zv, flags, -count); return (error); } -- Andriy Gapon From owner-zfs-devel@FreeBSD.ORG Fri Feb 22 07:25:12 2013 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 167CF4A9 for ; Fri, 22 Feb 2013 07:25:12 +0000 (UTC) (envelope-from delphij@delphij.net) Received: from anubis.delphij.net (anubis.delphij.net [64.62.153.212]) by mx1.freebsd.org (Postfix) with ESMTP id DE3ECB4A for ; Fri, 22 Feb 2013 07:25:08 +0000 (UTC) Received: from Xins-MacBook-Pro-2.local (c-67-188-85-47.hsd1.ca.comcast.net [67.188.85.47]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by anubis.delphij.net (Postfix) with ESMTPSA id 53954246DA; Thu, 21 Feb 2013 23:25:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=delphij.net; s=anubis; t=1361517908; bh=N0dIJWH6IgWzGtEiUJ3vWCKkGv0AcoEdqqm0xh2mqrE=; h=Date:From:Reply-To:To:Subject; b=hKpyiywDIhZ+2Q5+4I/Ybn02HaapNsAcEK7u670xWw/IWR3b3WYRAeC0/g04gVDas mfxKMXrLHb24VO2vvYK3cKmwo03TiVmXeY1F+3mAZABwJlfnVmBOQO/lf5q7vrryM/ qmP0ouFj0xcppsEd7ic58XxL+gakj/we4wy3Sfl8= Message-ID: <51271D53.30200@delphij.net> Date: Thu, 21 Feb 2013 23:25:07 -0800 From: Xin Li Organization: The FreeBSD Project MIME-Version: 1.0 To: zfs-devel@freebsd.org Subject: MFC of SPA v5000 to 8.x? X-Enigmail-Version: 1.5 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list Reply-To: d@delphij.net List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 22 Feb 2013 07:25:12 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 Hi, Do anyone have plan to MFC SPA v5000 to 8.x? I'm inclined to say "No" but backport a deadlock fix instead, but would like to ask before any action. Cheers, -----BEGIN PGP SIGNATURE----- iQEcBAEBCAAGBQJRJx1TAAoJEG80Jeu8UPuzpfQH/1ttmTLg8ep5X2LuZX526McO /zeR7eia2CjGszA3fJo+LXMAb3moNKHWJQ8i3nLpirArXOMYmrp1tYhZCPXNVWz+ n0nOHtNMx0doVUvYedoJzdCnn7XbQQbG/lMq96+iRsTx46LQDcti5jVFsjFvOEK4 bqx33joYgLz/hL4r+nuVAzF4g4+kWZi8WSKzeL/4H6lxlZEW0sQGAhOe2GZZHWjV xcA+cCGE6DZOogavuYkOGVKRsC4W62+Y0Sa5EdY07n9AVkwdytu2AnupHWrpVG1a iqQTEV8aurSNfSnvcknEnLV35EqUOnbw+7cpn7Hoy40EyTmJAkKnyiKfMfn62lQ= =yqqv -----END PGP SIGNATURE----- From owner-zfs-devel@FreeBSD.ORG Sat Feb 23 11:59:59 2013 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 5D251533 for ; Sat, 23 Feb 2013 11:59:59 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from mail.vx.sk (mail.vx.sk [176.9.45.25]) by mx1.freebsd.org (Postfix) with ESMTP id 15682A59 for ; Sat, 23 Feb 2013 11:59:58 +0000 (UTC) Received: from core.vx.sk (localhost [127.0.0.2]) by mail.vx.sk (Postfix) with ESMTP id 2C530265C3; Sat, 23 Feb 2013 12:59:52 +0100 (CET) X-Virus-Scanned: amavisd-new at mail.vx.sk Received: from mail.vx.sk by core.vx.sk (amavisd-new, unix socket) with LMTP id JudKsgoYGwEe; Sat, 23 Feb 2013 12:59:47 +0100 (CET) Received: from [10.9.8.1] (chello085216226145.chello.sk [85.216.226.145]) by mail.vx.sk (Postfix) with ESMTPSA id BF54D265B3; Sat, 23 Feb 2013 12:59:47 +0100 (CET) Message-ID: <5128AF33.5000509@FreeBSD.org> Date: Sat, 23 Feb 2013 12:59:47 +0100 From: Martin Matuska User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:17.0) Gecko/20130215 Thunderbird/17.0.3 MIME-Version: 1.0 To: d@delphij.net Subject: Re: MFC of SPA v5000 to 8.x? References: <51271D53.30200@delphij.net> In-Reply-To: <51271D53.30200@delphij.net> X-Enigmail-Version: 1.5 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: zfs-devel@freebsd.org X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 23 Feb 2013 11:59:59 -0000 This was already merged two months ago in r243717. On 22.2.2013 8:25, Xin Li wrote: > Hi, > > Do anyone have plan to MFC SPA v5000 to 8.x? I'm inclined to say "No" > but backport a deadlock fix instead, but would like to ask before any > action. > > Cheers, -- Martin Matuska FreeBSD committer http://blog.vx.sk From owner-zfs-devel@FreeBSD.ORG Sat Feb 23 12:57:22 2013 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 65DAA2D8 for ; Sat, 23 Feb 2013 12:57:22 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from mail.vx.sk (mail.vx.sk [176.9.45.25]) by mx1.freebsd.org (Postfix) with ESMTP id A72C0CE8 for ; Sat, 23 Feb 2013 12:57:21 +0000 (UTC) Received: from core.vx.sk (localhost [127.0.0.2]) by mail.vx.sk (Postfix) with ESMTP id B4D641C993; Sat, 23 Feb 2013 13:57:20 +0100 (CET) X-Virus-Scanned: amavisd-new at mail.vx.sk Received: from mail.vx.sk by core.vx.sk (amavisd-new, unix socket) with LMTP id pZPRYXKEP_BS; Sat, 23 Feb 2013 13:57:15 +0100 (CET) Received: from [10.9.8.1] (chello085216226145.chello.sk [85.216.226.145]) by mail.vx.sk (Postfix) with ESMTPSA id AD6251C982; Sat, 23 Feb 2013 13:57:14 +0100 (CET) Message-ID: <5128BCA9.6050801@FreeBSD.org> Date: Sat, 23 Feb 2013 13:57:13 +0100 From: Martin Matuska User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:17.0) Gecko/20130215 Thunderbird/17.0.3 MIME-Version: 1.0 To: zfs-devel@FreeBSD.org Subject: Vendor merge review: ZFS I/O deadman thread X-Enigmail-Version: 1.5 Content-Type: multipart/mixed; boundary="------------090105020004000108000805" X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 23 Feb 2013 12:57:22 -0000 This is a multi-part message in MIME format. --------------090105020004000108000805 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Please review the following proposed merge from vendor. The patch uses callout_drain(), I am not sure if callout_stop() is safe in this case. As of the patch, the deadman thread is disabled by default. Should we enable it? Thank you. http://people.freebsd.org/~mm/patches/zfs/zfs_deadman.patch Link to vendor ZFS issue: 3246 ZFS I/O deadman thread https://www.illumos.org/issues/3246 -- Martin Matuska FreeBSD committer http://blog.vx.sk --------------090105020004000108000805 Content-Type: text/plain; charset=windows-1250; name="zfs_deadman.patch" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="zfs_deadman.patch" SW5kZXg6IGNkZGwvY29udHJpYi9vcGVuc29sYXJpcwo9PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09Ci0tLSBjZGRs L2NvbnRyaWIvb3BlbnNvbGFyaXMJKHJldmlzaW9uIDI0NzE4NykKKysrIGNkZGwvY29udHJp Yi9vcGVuc29sYXJpcwkod29ya2luZyBjb3B5KQoKUHJvcGVydHkgY2hhbmdlcyBvbjogY2Rk bC9jb250cmliL29wZW5zb2xhcmlzCl9fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18KTW9kaWZpZWQ6IHN2bjptZXJn ZWluZm8KICAgTWVyZ2VkIC92ZW5kb3IvaWxsdW1vcy9kaXN0OnIyNDI3MzIKSW5kZXg6IGNk ZGwvY29udHJpYi9vcGVuc29sYXJpcy9jbWQvemluamVjdC90cmFuc2xhdGUuYwo9PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09Ci0tLSBjZGRsL2NvbnRyaWIvb3BlbnNvbGFyaXMvY21kL3ppbmplY3QvdHJhbnNs YXRlLmMJKHJldmlzaW9uIDI0NzE4NykKKysrIGNkZGwvY29udHJpYi9vcGVuc29sYXJpcy9j bWQvemluamVjdC90cmFuc2xhdGUuYwkod29ya2luZyBjb3B5KQpAQCAtMjAsNiArMjAsNyBA QAogICovCiAvKgogICogQ29weXJpZ2h0IChjKSAyMDA2LCAyMDEwLCBPcmFjbGUgYW5kL29y IGl0cyBhZmZpbGlhdGVzLiBBbGwgcmlnaHRzIHJlc2VydmVkLgorICogQ29weXJpZ2h0IChj KSAyMDEyIGJ5IERlbHBoaXguIEFsbCByaWdodHMgcmVzZXJ2ZWQuCiAgKi8KIAogI2luY2x1 ZGUgPGxpYnpmcy5oPgpAQCAtNDU1LDYgKzQ1NiwyMCBAQCB0cmFuc2xhdGVfZGV2aWNlKGNv bnN0IGNoYXIgKnBvb2wsIGNvbnN0IGNoYXIgKmRldgogCQkgICAgJnJlY29yZC0+emlfZ3Vp ZCkgPT0gMCk7CiAJfQogCisJLyoKKwkgKiBEZXZpY2UgZmF1bHRzIGNhbiB0YWtlIG9uIHRo cmVlIGRpZmZlcmVudCBmb3JtczoKKwkgKiAxKS4gZGVsYXllZCBvciBoYW5naW5nIEkvTwor CSAqIDIpLiB6ZnMgbGFiZWwgZmF1bHRzCisJICogMykuIGdlbmVyaWMgZGlzayBmYXVsdHMK KwkgKi8KKwlpZiAocmVjb3JkLT56aV90aW1lciAhPSAwKSB7CisJCXJlY29yZC0+emlfY21k ID0gWklOSkVDVF9ERUxBWV9JTzsKKwl9IGVsc2UgaWYgKGxhYmVsX3R5cGUgIT0gVFlQRV9J TlZBTCkgeworCQlyZWNvcmQtPnppX2NtZCA9IFpJTkpFQ1RfTEFCRUxfRkFVTFQ7CisJfSBl bHNlIHsKKwkJcmVjb3JkLT56aV9jbWQgPSBaSU5KRUNUX0RFVklDRV9GQVVMVDsKKwl9CisK IAlzd2l0Y2ggKGxhYmVsX3R5cGUpIHsKIAljYXNlIFRZUEVfTEFCRUxfVUJFUkJMT0NLOgog CQlyZWNvcmQtPnppX3N0YXJ0ID0gb2Zmc2V0b2YodmRldl9sYWJlbF90LCB2bF91YmVyYmxv Y2tbMF0pOwpJbmRleDogY2RkbC9jb250cmliL29wZW5zb2xhcmlzL2NtZC96aW5qZWN0L3pp bmplY3QuYwo9PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09Ci0tLSBjZGRsL2NvbnRyaWIvb3BlbnNvbGFyaXMvY21k L3ppbmplY3QvemluamVjdC5jCShyZXZpc2lvbiAyNDcxODcpCisrKyBjZGRsL2NvbnRyaWIv b3BlbnNvbGFyaXMvY21kL3ppbmplY3QvemluamVjdC5jCSh3b3JraW5nIGNvcHkpCkBAIC0y MCw2ICsyMCw3IEBACiAgKi8KIC8qCiAgKiBDb3B5cmlnaHQgKGMpIDIwMDUsIDIwMTAsIE9y YWNsZSBhbmQvb3IgaXRzIGFmZmlsaWF0ZXMuIEFsbCByaWdodHMgcmVzZXJ2ZWQuCisgKiBD b3B5cmlnaHQgKGMpIDIwMTIgYnkgRGVscGhpeC4gQWxsIHJpZ2h0cyByZXNlcnZlZC4KICAq LwogCiAvKgpAQCAtNjAzLDcgKzYwNCw3IEBAIG1haW4oaW50IGFyZ2MsIGNoYXIgKiphcmd2 KQogCX0KIAogCXdoaWxlICgoYyA9IGdldG9wdChhcmdjLCBhcmd2LAotCSAgICAiOmFBOmI6 ZDpmOkZnOnFoSWM6dDpUOmw6bXI6czplOnVMOnA6IikpICE9IC0xKSB7CisJICAgICI6YUE6 YjpkOkQ6ZjpGZzpxaEljOnQ6VDpsOm1yOnM6ZTp1TDpwOiIpKSAhPSAtMSkgewogCQlzd2l0 Y2ggKGMpIHsKIAkJY2FzZSAnYSc6CiAJCQlmbGFncyB8PSBaSU5KRUNUX0ZMVVNIX0FSQzsK QEAgLTYyOSw2ICs2MzAsMTUgQEAgbWFpbihpbnQgYXJnYywgY2hhciAqKmFyZ3YpCiAJCWNh c2UgJ2QnOgogCQkJZGV2aWNlID0gb3B0YXJnOwogCQkJYnJlYWs7CisJCWNhc2UgJ0QnOgor CQkJcmVjb3JkLnppX3RpbWVyID0gc3RydG91bGwob3B0YXJnLCAmZW5kLCAxMCk7CisJCQlp ZiAoZXJybm8gIT0gMCB8fCAqZW5kICE9ICdcMCcpIHsKKwkJCQkodm9pZCkgZnByaW50Zihz dGRlcnIsICJpbnZhbGlkIGkvbyBkZWxheSAiCisJCQkJICAgICJ2YWx1ZTogJyVzJ1xuIiwg b3B0YXJnKTsKKwkJCQl1c2FnZSgpOworCQkJCXJldHVybiAoMSk7CisJCQl9CisJCQlicmVh azsKIAkJY2FzZSAnZSc6CiAJCQlpZiAoc3RyY2FzZWNtcChvcHRhcmcsICJpbyIpID09IDAp IHsKIAkJCQllcnJvciA9IEVJTzsKQEAgLTY5Myw2ICs3MDMsNyBAQCBtYWluKGludCBhcmdj LCBjaGFyICoqYXJndikKIAkJY2FzZSAncCc6CiAJCQkodm9pZCkgc3RybGNweShyZWNvcmQu emlfZnVuYywgb3B0YXJnLAogCQkJICAgIHNpemVvZiAocmVjb3JkLnppX2Z1bmMpKTsKKwkJ CXJlY29yZC56aV9jbWQgPSBaSU5KRUNUX1BBTklDOwogCQkJYnJlYWs7CiAJCWNhc2UgJ3En OgogCQkJcXVpZXQgPSAxOwpAQCAtNzY2LDEzICs3NzcsMTUgQEAgbWFpbihpbnQgYXJnYywg Y2hhciAqKmFyZ3YpCiAJYXJnYyAtPSBvcHRpbmQ7CiAJYXJndiArPSBvcHRpbmQ7CiAKKwlp ZiAocmVjb3JkLnppX2R1cmF0aW9uICE9IDApCisJCXJlY29yZC56aV9jbWQgPSBaSU5KRUNU X0lHTk9SRURfV1JJVEVTOworCiAJaWYgKGNhbmNlbCAhPSBOVUxMKSB7CiAJCS8qCiAJCSAq ICctYycgaXMgaW52YWxpZCB3aXRoIGFueSBvdGhlciBvcHRpb25zLgogCQkgKi8KIAkJaWYg KHJhdyAhPSBOVUxMIHx8IHJhbmdlICE9IE5VTEwgfHwgdHlwZSAhPSBUWVBFX0lOVkFMIHx8 Ci0JCSAgICBsZXZlbCAhPSAwIHx8IHJlY29yZC56aV9mdW5jWzBdICE9ICdcMCcgfHwKLQkJ ICAgIHJlY29yZC56aV9kdXJhdGlvbiAhPSAwKSB7CisJCSAgICBsZXZlbCAhPSAwIHx8IHJl Y29yZC56aV9jbWQgIT0gWklOSkVDVF9VTklOSVRJQUxJWkVEKSB7CiAJCQkodm9pZCkgZnBy aW50ZihzdGRlcnIsICJjYW5jZWwgKC1jKSBpbmNvbXBhdGlibGUgd2l0aCAiCiAJCQkgICAg ImFueSBvdGhlciBvcHRpb25zXG4iKTsKIAkJCXVzYWdlKCk7CkBAIC04MDQsOCArODE3LDcg QEAgbWFpbihpbnQgYXJnYywgY2hhciAqKmFyZ3YpCiAJCSAqIGZvciBkb2luZyBpbmplY3Rp b24sIHNvIGhhbmRsZSBpdCBzZXBhcmF0ZWx5IGhlcmUuCiAJCSAqLwogCQlpZiAocmF3ICE9 IE5VTEwgfHwgcmFuZ2UgIT0gTlVMTCB8fCB0eXBlICE9IFRZUEVfSU5WQUwgfHwKLQkJICAg IGxldmVsICE9IDAgfHwgcmVjb3JkLnppX2Z1bmNbMF0gIT0gJ1wwJyB8fAotCQkgICAgcmVj b3JkLnppX2R1cmF0aW9uICE9IDApIHsKKwkJICAgIGxldmVsICE9IDAgfHwgcmVjb3JkLnpp X2NtZCAhPSBaSU5KRUNUX1VOSU5JVElBTElaRUQpIHsKIAkJCSh2b2lkKSBmcHJpbnRmKHN0 ZGVyciwgImRldmljZSAoLWQpIGluY29tcGF0aWJsZSB3aXRoICIKIAkJCSAgICAiZGF0YSBl cnJvciBpbmplY3Rpb25cbiIpOwogCQkJdXNhZ2UoKTsKQEAgLTgzOSw3ICs4NTEsNyBAQCBt YWluKGludCBhcmdjLCBjaGFyICoqYXJndikKIAogCX0gZWxzZSBpZiAocmF3ICE9IE5VTEwp IHsKIAkJaWYgKHJhbmdlICE9IE5VTEwgfHwgdHlwZSAhPSBUWVBFX0lOVkFMIHx8IGxldmVs ICE9IDAgfHwKLQkJICAgIHJlY29yZC56aV9mdW5jWzBdICE9ICdcMCcgfHwgcmVjb3JkLnpp X2R1cmF0aW9uICE9IDApIHsKKwkJICAgIHJlY29yZC56aV9jbWQgIT0gWklOSkVDVF9VTklO SVRJQUxJWkVEKSB7CiAJCQkodm9pZCkgZnByaW50ZihzdGRlcnIsICJyYXcgKC1iKSBmb3Jt YXQgd2l0aCAiCiAJCQkgICAgImFueSBvdGhlciBvcHRpb25zXG4iKTsKIAkJCXVzYWdlKCk7 CkBAIC04NjIsMTMgKzg3NCwxNCBAQCBtYWluKGludCBhcmdjLCBjaGFyICoqYXJndikKIAkJ CXJldHVybiAoMSk7CiAJCX0KIAorCQlyZWNvcmQuemlfY21kID0gWklOSkVDVF9EQVRBX0ZB VUxUOwogCQlpZiAodHJhbnNsYXRlX3JhdyhyYXcsICZyZWNvcmQpICE9IDApCiAJCQlyZXR1 cm4gKDEpOwogCQlpZiAoIWVycm9yKQogCQkJZXJyb3IgPSBFSU87Ci0JfSBlbHNlIGlmIChy ZWNvcmQuemlfZnVuY1swXSAhPSAnXDAnKSB7CisJfSBlbHNlIGlmIChyZWNvcmQuemlfY21k ID09IFpJTkpFQ1RfUEFOSUMpIHsKIAkJaWYgKHJhdyAhPSBOVUxMIHx8IHJhbmdlICE9IE5V TEwgfHwgdHlwZSAhPSBUWVBFX0lOVkFMIHx8Ci0JCSAgICBsZXZlbCAhPSAwIHx8IGRldmlj ZSAhPSBOVUxMIHx8IHJlY29yZC56aV9kdXJhdGlvbiAhPSAwKSB7CisJCSAgICBsZXZlbCAh PSAwIHx8IGRldmljZSAhPSBOVUxMKSB7CiAJCQkodm9pZCkgZnByaW50ZihzdGRlcnIsICJw YW5pYyAoLXApIGluY29tcGF0aWJsZSB3aXRoICIKIAkJCSAgICAib3RoZXIgb3B0aW9uc1xu Iik7CiAJCQl1c2FnZSgpOwpAQCAtODg2LDcgKzg5OSw3IEBAIG1haW4oaW50IGFyZ2MsIGNo YXIgKiphcmd2KQogCQlpZiAoYXJndlsxXSAhPSBOVUxMKQogCQkJcmVjb3JkLnppX3R5cGUg PSBhdG9pKGFyZ3ZbMV0pOwogCQlkYXRhc2V0WzBdID0gJ1wwJzsKLQl9IGVsc2UgaWYgKHJl Y29yZC56aV9kdXJhdGlvbiAhPSAwKSB7CisJfSBlbHNlIGlmIChyZWNvcmQuemlfY21kID09 IFpJTkpFQ1RfSUdOT1JFRF9XUklURVMpIHsKIAkJaWYgKG5vd3JpdGVzID09IDApIHsKIAkJ CSh2b2lkKSBmcHJpbnRmKHN0ZGVyciwgIi1zIG9yIC1nIG1lYW5pbmdsZXNzICIKIAkJCSAg ICAid2l0aG91dCAtSSAoaWdub3JlIHdyaXRlcylcbiIpOwpAQCAtOTQwLDYgKzk1Myw3IEBA IG1haW4oaW50IGFyZ2MsIGNoYXIgKiphcmd2KQogCQkJcmV0dXJuICgxKTsKIAkJfQogCisJ CXJlY29yZC56aV9jbWQgPSBaSU5KRUNUX0RBVEFfRkFVTFQ7CiAJCWlmICh0cmFuc2xhdGVf cmVjb3JkKHR5cGUsIGFyZ3ZbMF0sIHJhbmdlLCBsZXZlbCwgJnJlY29yZCwgcG9vbCwKIAkJ ICAgIGRhdGFzZXQpICE9IDApCiAJCQlyZXR1cm4gKDEpOwpJbmRleDogY2RkbC9jb250cmli L29wZW5zb2xhcmlzL2xpYi9saWJ6cG9vbC9jb21tb24va2VybmVsLmMKPT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PQotLS0gY2RkbC9jb250cmliL29wZW5zb2xhcmlzL2xpYi9saWJ6cG9vbC9jb21tb24va2Vy bmVsLmMJKHJldmlzaW9uIDI0NzE4NykKKysrIGNkZGwvY29udHJpYi9vcGVuc29sYXJpcy9s aWIvbGlienBvb2wvY29tbW9uL2tlcm5lbC5jCSh3b3JraW5nIGNvcHkpCkBAIC00NSw2ICs0 NSw5IEBAIGludCBhb2s7CiB1aW50NjRfdCBwaHlzbWVtOwogdm5vZGVfdCAqcm9vdGRpciA9 ICh2bm9kZV90ICopMHhhYmNkMTIzNDsKIGNoYXIgaHdfc2VyaWFsW0hXX0hPU1RJRF9MRU5d OworI2lmZGVmIGlsbHVtb3MKK2ttdXRleF90IGNwdV9sb2NrOworI2VuZGlmCiAKIHN0cnVj dCB1dHNuYW1lIHV0c25hbWUgPSB7CiAJInVzZXJsYW5kIiwgImxpYnpwb29sIiwgIjEiLCAi MSIsICJuYSIKQEAgLTg0Miw2ICs4NDUsMjggQEAgZGRpX3N0cnRvdWxsKGNvbnN0IGNoYXIg KnN0ciwgY2hhciAqKm5wdHIsIGludCBiYXMKIAlyZXR1cm4gKDApOwogfQogCisjaWZkZWYg aWxsdW1vcworLyogQVJHU1VTRUQgKi8KK2N5Y2xpY19pZF90CitjeWNsaWNfYWRkKGN5Y19o YW5kbGVyX3QgKmhkbHIsIGN5Y190aW1lX3QgKndoZW4pCit7CisJcmV0dXJuICgxKTsKK30K KworLyogQVJHU1VTRUQgKi8KK3ZvaWQKK2N5Y2xpY19yZW1vdmUoY3ljbGljX2lkX3QgaWQp Cit7Cit9CisKKy8qIEFSR1NVU0VEICovCitpbnQKK2N5Y2xpY19yZXByb2dyYW0oY3ljbGlj X2lkX3QgaWQsIGhydGltZV90IGV4cGlyYXRpb24pCit7CisJcmV0dXJuICgxKTsKK30KKyNl bmRpZgorCiAvKgogICogPT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PQogICoga2VybmVsIGVtdWxhdGlv biBzZXR1cCAmIHRlYXJkb3duCkBAIC04NzUsNiArOTAwLDEwIEBAIGtlcm5lbF9pbml0KGlu dCBtb2RlKQogCiAJc3lzdGVtX3Rhc2txX2luaXQoKTsKIAorI2lmZGVmIGlsbHVtb3MKKwlt dXRleF9pbml0KCZjcHVfbG9jaywgTlVMTCwgTVVURVhfREVGQVVMVCwgTlVMTCk7CisjZW5k aWYKKwogCXNwYV9pbml0KG1vZGUpOwogfQogCkluZGV4OiBjZGRsL2NvbnRyaWIvb3BlbnNv bGFyaXMvbGliL2xpYnpwb29sL2NvbW1vbi9zeXMvemZzX2NvbnRleHQuaAo9PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09Ci0tLSBjZGRsL2NvbnRyaWIvb3BlbnNvbGFyaXMvbGliL2xpYnpwb29sL2NvbW1vbi9z eXMvemZzX2NvbnRleHQuaAkocmV2aXNpb24gMjQ3MTg3KQorKysgY2RkbC9jb250cmliL29w ZW5zb2xhcmlzL2xpYi9saWJ6cG9vbC9jb21tb24vc3lzL3pmc19jb250ZXh0LmgJKHdvcmtp bmcgY29weSkKQEAgLTQ1Nyw2ICs0NTcsOSBAQCBleHRlcm4gdm5vZGVfdCAqcm9vdGRpcjsK IAogZXh0ZXJuIHZvaWQgZGVsYXkoY2xvY2tfdCB0aWNrcyk7CiAKKyNkZWZpbmUJU0VDX1RP X1RJQ0soc2VjKQkoKHNlYykgKiBoeikKKyNkZWZpbmUJTlNFQ19UT19USUNLKHVzZWMpCSgo dXNlYykgLyAoTkFOT1NFQyAvIGh6KSkKKwogI2RlZmluZQlnZXRocmVzdGltZV9zZWMoKSB0 aW1lKE5VTEwpCiAjZGVmaW5lCWdldGhyZXN0aW1lKHQpIFwKIAlkbyB7XApAQCAtNjI0LDYg KzYyNywzNiBAQCB0eXBlZGVmCXVpbnQzMl90CWlkbWFwX3JpZF90OwogI2RlZmluZQlFUkVT VEFSVAkoLTEpCiAjZW5kaWYKIAorI2lmZGVmIGlsbHVtb3MKKy8qCisgKiBDeWNsaWMgaW5m b3JtYXRpb24KKyAqLworZXh0ZXJuIGttdXRleF90IGNwdV9sb2NrOworCit0eXBlZGVmIHVp bnRwdHJfdCBjeWNsaWNfaWRfdDsKK3R5cGVkZWYgdWludDE2X3QgY3ljX2xldmVsX3Q7Cit0 eXBlZGVmIHZvaWQgKCpjeWNfZnVuY190KSh2b2lkICopOworCisjZGVmaW5lCUNZX0xPV19M RVZFTAkwCisjZGVmaW5lCUNZX0lORklOSVRZCUlOVDY0X01BWAorI2RlZmluZQlDWUNMSUNf Tk9ORQkoKGN5Y2xpY19pZF90KTApCisKK3R5cGVkZWYgc3RydWN0IGN5Y190aW1lIHsKKwlo cnRpbWVfdCBjeXRfd2hlbjsKKwlocnRpbWVfdCBjeXRfaW50ZXJ2YWw7Cit9IGN5Y190aW1l X3Q7CisKK3R5cGVkZWYgc3RydWN0IGN5Y19oYW5kbGVyIHsKKwljeWNfZnVuY190IGN5aF9m dW5jOworCXZvaWQgKmN5aF9hcmc7CisJY3ljX2xldmVsX3QgY3loX2xldmVsOworfSBjeWNf aGFuZGxlcl90OworCitleHRlcm4gY3ljbGljX2lkX3QgY3ljbGljX2FkZChjeWNfaGFuZGxl cl90ICosIGN5Y190aW1lX3QgKik7CitleHRlcm4gdm9pZCBjeWNsaWNfcmVtb3ZlKGN5Y2xp Y19pZF90KTsKK2V4dGVybiBpbnQgY3ljbGljX3JlcHJvZ3JhbShjeWNsaWNfaWRfdCwgaHJ0 aW1lX3QpOworI2VuZGlmCS8qIGlsbHVtb3MgKi8KKwogI2lmZGVmCV9fY3BsdXNwbHVzCiB9 CiAjZW5kaWYKSW5kZXg6IHN5cy9jZGRsL2NvbXBhdC9vcGVuc29sYXJpcy9zeXMvdGltZS5o Cj09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT0KLS0tIHN5cy9jZGRsL2NvbXBhdC9vcGVuc29sYXJpcy9zeXMvdGlt ZS5oCShyZXZpc2lvbiAyNDcxODcpCisrKyBzeXMvY2RkbC9jb21wYXQvb3BlbnNvbGFyaXMv c3lzL3RpbWUuaAkod29ya2luZyBjb3B5KQpAQCAtNDYsNiArNDYsOSBAQCB0eXBlZGVmIGxv bmdsb25nX3QJaHJ0aW1lX3Q7CiAJKCh0cyktPnR2X3NlYyA8IElOVDY0X01JTiB8fCAodHMp LT50dl9zZWMgPiBJTlQ2NF9NQVgpCiAjZW5kaWYKIAorI2RlZmluZQlTRUNfVE9fVElDSyhz ZWMpCSgoc2VjKSAqIGh6KQorI2RlZmluZQlOU0VDX1RPX1RJQ0sodXNlYykJKCh1c2VjKSAv IChOQU5PU0VDIC8gaHopKQorCiAjaWZkZWYgX0tFUk5FTAogc3RhdGljIF9faW5saW5lIGhy dGltZV90CiBnZXRocnRpbWUodm9pZCkgewpJbmRleDogc3lzL2NkZGwvY29udHJpYi9vcGVu c29sYXJpcwo9PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09Ci0tLSBzeXMvY2RkbC9jb250cmliL29wZW5zb2xhcmlz CShyZXZpc2lvbiAyNDcxODcpCisrKyBzeXMvY2RkbC9jb250cmliL29wZW5zb2xhcmlzCSh3 b3JraW5nIGNvcHkpCgpQcm9wZXJ0eSBjaGFuZ2VzIG9uOiBzeXMvY2RkbC9jb250cmliL29w ZW5zb2xhcmlzCl9fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX18KTW9kaWZpZWQ6IHN2bjptZXJnZWluZm8KICAgTWVy Z2VkIC92ZW5kb3Itc3lzL2lsbHVtb3MvZGlzdDpyMjQyNzMyCkluZGV4OiBzeXMvY2RkbC9j b250cmliL29wZW5zb2xhcmlzL3V0cy9jb21tb24vZnMvemZzL3NwYS5jCj09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT0KLS0tIHN5cy9jZGRsL2NvbnRyaWIvb3BlbnNvbGFyaXMvdXRzL2NvbW1vbi9mcy96ZnMv c3BhLmMJKHJldmlzaW9uIDI0NzE4NykKKysrIHN5cy9jZGRsL2NvbnRyaWIvb3BlbnNvbGFy aXMvdXRzL2NvbW1vbi9mcy96ZnMvc3BhLmMJKHdvcmtpbmcgY29weSkKQEAgLTE0MSw2ICsx NDEsMTAgQEAgdWludF90CQl6aW9fdGFza3FfYmFzZWRjID0gODA7CQkvKiBiYXNlIGR1dHkg Y3ljbGUKIGJvb2xlYW5fdAlzcGFfY3JlYXRlX3Byb2Nlc3MgPSBCX1RSVUU7CS8qIG5vIHBy b2Nlc3MgPT0+IG5vIHN5c2RjICovCiBleHRlcm4gaW50CXpmc19zeW5jX3Bhc3NfZGVmZXJy ZWRfZnJlZTsKIAorI2lmbmRlZiBpbGx1bW9zCitleHRlcm4gdm9pZCBzcGFfZGVhZG1hbih2 b2lkICphcmcpOworI2VuZGlmCisKIC8qCiAgKiBUaGlzIChpbGxlZ2FsKSBwb29sIG5hbWUg aXMgdXNlZCB3aGVuIHRlbXBvcmFyaWx5IGltcG9ydGluZyBhIHNwYV90IGluIG9yZGVyCiAg KiB0byBnZXQgdGhlIHZkZXYgc3RhdHMgYXNzb2NpYXRlZCB3aXRoIHRoZSBpbXBvcnRlZCBk ZXZpY2VzLgpAQCAtNjI1OCw2ICs2MjYyLDE3IEBAIHNwYV9zeW5jKHNwYV90ICpzcGEsIHVp bnQ2NF90IHR4ZykKIAogCXR4ID0gZG11X3R4X2NyZWF0ZV9hc3NpZ25lZChkcCwgdHhnKTsK IAorCXNwYS0+c3BhX3N5bmNfc3RhcnR0aW1lID0gZ2V0aHJ0aW1lKCk7CisjaWZkZWYgaWxs dW1vcworCVZFUklGWShjeWNsaWNfcmVwcm9ncmFtKHNwYS0+c3BhX2RlYWRtYW5fY3ljaWQs CisJICAgIHNwYS0+c3BhX3N5bmNfc3RhcnR0aW1lICsgc3BhLT5zcGFfZGVhZG1hbl9zeW5j dGltZSkpOworI2Vsc2UJLyogRnJlZUJTRCAqLworI2lmZGVmIF9LRVJORUwKKwljYWxsb3V0 X3Jlc2V0KCZzcGEtPnNwYV9kZWFkbWFuX2N5Y2lkLAorCSAgICBoeiAqIHNwYS0+c3BhX2Rl YWRtYW5fc3luY3RpbWUgLyBOQU5PU0VDLCBzcGFfZGVhZG1hbiwgc3BhKTsKKyNlbmRpZgor I2VuZGlmCisKIAkvKgogCSAqIElmIHdlIGFyZSB1cGdyYWRpbmcgdG8gU1BBX1ZFUlNJT05f UkFJRFpfREVGTEFURSB0aGlzIHR4ZywKIAkgKiBzZXQgc3BhX2RlZmxhdGUgaWYgd2UgaGF2 ZSBubyByYWlkLXogdmRldnMuCkBAIC02Mzg2LDYgKzY0MDEsMTQgQEAgc3BhX3N5bmMoc3Bh X3QgKnNwYSwgdWludDY0X3QgdHhnKQogCX0KIAlkbXVfdHhfY29tbWl0KHR4KTsKIAorI2lm ZGVmIGlsbHVtb3MKKwlWRVJJRlkoY3ljbGljX3JlcHJvZ3JhbShzcGEtPnNwYV9kZWFkbWFu X2N5Y2lkLCBDWV9JTkZJTklUWSkpOworI2Vsc2UJLyogRnJlZUJTRCAqLworI2lmZGVmIF9L RVJORUwKKwljYWxsb3V0X2RyYWluKCZzcGEtPnNwYV9kZWFkbWFuX2N5Y2lkKTsKKyNlbmRp ZgorI2VuZGlmCisKIAkvKgogCSAqIENsZWFyIHRoZSBkaXJ0eSBjb25maWcgbGlzdC4KIAkg Ki8KSW5kZXg6IHN5cy9jZGRsL2NvbnRyaWIvb3BlbnNvbGFyaXMvdXRzL2NvbW1vbi9mcy96 ZnMvc3BhX21pc2MuYwo9PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09Ci0tLSBzeXMvY2RkbC9jb250cmliL29wZW5z b2xhcmlzL3V0cy9jb21tb24vZnMvemZzL3NwYV9taXNjLmMJKHJldmlzaW9uIDI0NzE4NykK KysrIHN5cy9jZGRsL2NvbnRyaWIvb3BlbnNvbGFyaXMvdXRzL2NvbW1vbi9mcy96ZnMvc3Bh X21pc2MuYwkod29ya2luZyBjb3B5KQpAQCAtMjMsOSArMjMsMTMgQEAKICAqIENvcHlyaWdo dCAoYykgMjAxMiBieSBEZWxwaGl4LiBBbGwgcmlnaHRzIHJlc2VydmVkLgogICogQ29weXJp Z2h0IDIwMTEgTmV4ZW50YSBTeXN0ZW1zLCBJbmMuICBBbGwgcmlnaHRzIHJlc2VydmVkLgog ICovCisvKgorICogUG9ydGlvbnMgQ29weXJpZ2h0IDIwMTIgTWFydGluIE1hdHVza2EgPG1t QEZyZWVCU0Qub3JnPgorICovCiAKICNpbmNsdWRlIDxzeXMvemZzX2NvbnRleHQuaD4KICNp bmNsdWRlIDxzeXMvc3BhX2ltcGwuaD4KKyNpbmNsdWRlIDxzeXMvc3BhX2Jvb3QuaD4KICNp bmNsdWRlIDxzeXMvemlvLmg+CiAjaW5jbHVkZSA8c3lzL3ppb19jaGVja3N1bS5oPgogI2lu Y2x1ZGUgPHN5cy96aW9fY29tcHJlc3MuaD4KQEAgLTI1Myw4ICsyNTcsMzQgQEAgVFVOQUJM RV9JTlQoInZmcy56ZnMucmVjb3ZlciIsICZ6ZnNfcmVjb3Zlcik7CiBTWVNDVExfSU5UKF92 ZnNfemZzLCBPSURfQVVUTywgcmVjb3ZlciwgQ1RMRkxBR19SRFRVTiwgJnpmc19yZWNvdmVy LCAwLAogICAgICJUcnkgdG8gcmVjb3ZlciBmcm9tIG90aGVyd2lzZS1mYXRhbCBlcnJvcnMu Iik7CiAKK2V4dGVybiBpbnQgemZzX3R4Z19zeW5jdGltZV9tczsKIAogLyoKKyAqIEV4cGly YXRpb24gdGltZSBpbiB1bml0cyBvZiB6ZnNfdHhnX3N5bmN0aW1lX21zLiBUaGlzIHZhbHVl IGhhcyB0d28KKyAqIG1lYW5pbmdzLiBGaXJzdCBpdCBpcyB1c2VkIHRvIGRldGVybWluZSB3 aGVuIHRoZSBzcGFfZGVhZG1hbiBsb2dpYworICogc2hvdWxkIGZpcmUuIEJ5IGRlZmF1bHQg dGhlIHNwYV9kZWFkbWFuIHdpbGwgZmlyZSBpZiBzcGFfc3luYyBoYXMKKyAqIG5vdCBjb21w bGV0ZWQgaW4gMTAwMCAqIHpmc190eGdfc3luY3RpbWVfbXMgKGkuZS4gMTAwMCBzZWNvbmRz KS4KKyAqIFNlY29uZGx5LCB0aGUgdmFsdWUgZGV0ZXJtaW5lcyBpZiBhbiBJL08gaXMgY29u c2lkZXJlZCAiaHVuZyIuCisgKiBBbnkgSS9PIHRoYXQgaGFzIG5vdCBjb21wbGV0ZWQgaW4g emZzX2RlYWRtYW5fc3luY3RpbWUgaXMgY29uc2lkZXJlZAorICogImh1bmciIHJlc3VsdGlu ZyBpbiBhIHN5c3RlbSBwYW5pYy4KKyAqIDEwMDAgemZzX3R4Z19zeW5jdGltZV9tcyAoaS5l LiAxMDAwIHNlY29uZHMpLgorICovCit1aW50NjRfdCB6ZnNfZGVhZG1hbl9zeW5jdGltZSA9 IDEwMDBVTEw7CitUVU5BQkxFX1FVQUQoInZmcy56ZnMuZGVhZG1hbl9zeW5jdGltZSIsICZ6 ZnNfZGVhZG1hbl9zeW5jdGltZSk7CitTWVNDVExfVVFVQUQoX3Zmc196ZnMsIE9JRF9BVVRP LCBkZWFkbWFuX3N5bmN0aW1lLCBDVExGTEFHX1JEVFVOLAorICAgICZ6ZnNfZGVhZG1hbl9z eW5jdGltZSwgMCwKKyAgICAiU3RhbGxlZCBaRlMgSS9PIGV4cGlyYXRpb24gdGltZSBpbiB1 bml0cyBvZiB2ZnMuemZzLnR4Z19zeW5jdGltZV9tcyIpOworCisvKgorICogT3ZlcnJpZGUg dGhlIHpmcyBkZWFkbWFuIGJlaGF2aW9yIHZpYSAvZXRjL3N5c3RlbS4gQnkgZGVmYXVsdCB0 aGUKKyAqIGRlYWRtYW4gaXMgZW5hYmxlZCBleGNlcHQgb24gVk13YXJlIGFuZCBzcGFyYyBk ZXBsb3ltZW50cy4KKyAqLworaW50IHpmc19kZWFkbWFuX2VuYWJsZWQgPSAwOworVFVOQUJM RV9JTlQoInZmcy56ZnMuZGVhZG1hbl9lbmFibGVkIiwgJnpmc19kZWFkbWFuX2VuYWJsZWQp OworU1lTQ1RMX0lOVChfdmZzX3pmcywgT0lEX0FVVE8sIGRlYWRtYW5fZW5hYmxlZCwgQ1RM RkxBR19SRFRVTiwKKyAgICAmemZzX2RlYWRtYW5fZW5hYmxlZCwgMCwgIktlcm5lbCBwYW5p YyBvbiBzdGFsbGVkIFpGUyBJL08iKTsKKworLyoKICAqID09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 CiAgKiBTUEEgY29uZmlnIGxvY2tpbmcKICAqID09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09CkBAIC00 MjIsNiArNDUyLDIzIEBAIHNwYV9sb29rdXAoY29uc3QgY2hhciAqbmFtZSkKIH0KIAogLyoK KyAqIEZpcmVzIHdoZW4gc3BhX3N5bmMgaGFzIG5vdCBjb21wbGV0ZWQgd2l0aGluIHpmc19k ZWFkbWFuX3N5bmN0aW1lX21zLgorICogSWYgdGhlIHpmc19kZWFkbWFuX2VuYWJsZWQgZmxh ZyBpcyBzZXQgdGhlbiBpdCBpbnNwZWN0cyBhbGwgdmRldiBxdWV1ZXMKKyAqIGxvb2tpbmcg Zm9yIHBvdGVudGlhbGx5IGh1bmcgSS9Pcy4KKyAqLwordm9pZAorc3BhX2RlYWRtYW4odm9p ZCAqYXJnKQoreworCXNwYV90ICpzcGEgPSBhcmc7CisKKwl6ZnNfZGJnbXNnKCJzbG93IHNw YV9zeW5jOiBzdGFydGVkICVsbHUgc2Vjb25kcyBhZ28sIGNhbGxzICVsbHUiLAorCSAgICAo Z2V0aHJ0aW1lKCkgLSBzcGEtPnNwYV9zeW5jX3N0YXJ0dGltZSkgLyBOQU5PU0VDLAorCSAg ICArK3NwYS0+c3BhX2RlYWRtYW5fY2FsbHMpOworCWlmICh6ZnNfZGVhZG1hbl9lbmFibGVk KQorCQl2ZGV2X2RlYWRtYW4oc3BhLT5zcGFfcm9vdF92ZGV2KTsKK30KKworLyoKICAqIENy ZWF0ZSBhbiB1bmluaXRpYWxpemVkIHNwYV90IHdpdGggdGhlIGdpdmVuIG5hbWUuICBSZXF1 aXJlcwogICogc3BhX25hbWVzcGFjZV9sb2NrLiAgVGhlIGNhbGxlciBtdXN0IGVuc3VyZSB0 aGF0IHRoZSBzcGFfdCBkb2Vzbid0IGFscmVhZHkKICAqIGV4aXN0IGJ5IGNhbGxpbmcgc3Bh X2xvb2t1cCgpIGZpcnN0LgpAQCAtNDMxLDYgKzQ3OCwxMCBAQCBzcGFfYWRkKGNvbnN0IGNo YXIgKm5hbWUsIG52bGlzdF90ICpjb25maWcsIGNvbnN0CiB7CiAJc3BhX3QgKnNwYTsKIAlz cGFfY29uZmlnX2RpcmVudF90ICpkcDsKKyNpZmRlZiBpbGx1bW9zCisJY3ljX2hhbmRsZXJf dCBoZGxyOworCWN5Y190aW1lX3Qgd2hlbjsKKyNlbmRpZgogCiAJQVNTRVJUKE1VVEVYX0hF TEQoJnNwYV9uYW1lc3BhY2VfbG9jaykpOwogCkBAIC00NjIsNiArNTEzLDMyIEBAIHNwYV9h ZGQoY29uc3QgY2hhciAqbmFtZSwgbnZsaXN0X3QgKmNvbmZpZywgY29uc3QKIAlzcGEtPnNw YV9wcm9jID0gJnAwOwogCXNwYS0+c3BhX3Byb2Nfc3RhdGUgPSBTUEFfUFJPQ19OT05FOwog CisjaWZkZWYgaWxsdW1vcworCWhkbHIuY3loX2Z1bmMgPSBzcGFfZGVhZG1hbjsKKwloZGxy LmN5aF9hcmcgPSBzcGE7CisJaGRsci5jeWhfbGV2ZWwgPSBDWV9MT1dfTEVWRUw7CisjZW5k aWYKKworCXNwYS0+c3BhX2RlYWRtYW5fc3luY3RpbWUgPSB6ZnNfZGVhZG1hbl9zeW5jdGlt ZSAqCisJICAgIHpmc190eGdfc3luY3RpbWVfbXMgKiBNSUNST1NFQzsKKworI2lmZGVmIGls bHVtb3MKKwkvKgorCSAqIFRoaXMgZGV0ZXJtaW5lcyBob3cgb2Z0ZW4gd2UgbmVlZCB0byBj aGVjayBmb3IgaHVuZyBJL09zIGFmdGVyCisJICogdGhlIGN5Y2xpYyBoYXMgYWxyZWFkeSBm aXJlZC4gU2luY2UgY2hlY2tpbmcgZm9yIGh1bmcgSS9PcyBpcworCSAqIGFuIGV4cGVuc2l2 ZSBvcGVyYXRpb24gd2UgZG9uJ3Qgd2FudCB0byBjaGVjayB0b28gZnJlcXVlbnRseS4KKwkg KiBJbnN0ZWFkIHdhaXQgZm9yIDUgc3luY3RpbWVzIGJlZm9yZSBjaGVja2luZyBhZ2Fpbi4K KwkgKi8KKwl3aGVuLmN5dF9pbnRlcnZhbCA9IDVVTEwgKiB6ZnNfdHhnX3N5bmN0aW1lX21z ICogTUlDUk9TRUM7CisJd2hlbi5jeXRfd2hlbiA9IENZX0lORklOSVRZOworCW11dGV4X2Vu dGVyKCZjcHVfbG9jayk7CisJc3BhLT5zcGFfZGVhZG1hbl9jeWNpZCA9IGN5Y2xpY19hZGQo JmhkbHIsICZ3aGVuKTsKKwltdXRleF9leGl0KCZjcHVfbG9jayk7CisjZWxzZQkvKiBGcmVl QlNEICovCisjaWZkZWYgX0tFUk5FTAorCWNhbGxvdXRfaW5pdCgmc3BhLT5zcGFfZGVhZG1h bl9jeWNpZCwgQ0FMTE9VVF9NUFNBRkUpOworI2VuZGlmCisjZW5kaWYKIAlyZWZjb3VudF9j cmVhdGUoJnNwYS0+c3BhX3JlZmNvdW50KTsKIAlzcGFfY29uZmlnX2xvY2tfaW5pdChzcGEp OwogCkBAIC01NDQsNiArNjIxLDE4IEBAIHNwYV9yZW1vdmUoc3BhX3QgKnNwYSkKIAludmxp c3RfZnJlZShzcGEtPnNwYV9sb2FkX2luZm8pOwogCXNwYV9jb25maWdfc2V0KHNwYSwgTlVM TCk7CiAKKyNpZmRlZiBpbGx1bW9zCisJbXV0ZXhfZW50ZXIoJmNwdV9sb2NrKTsKKwlpZiAo c3BhLT5zcGFfZGVhZG1hbl9jeWNpZCAhPSBDWUNMSUNfTk9ORSkKKwkJY3ljbGljX3JlbW92 ZShzcGEtPnNwYV9kZWFkbWFuX2N5Y2lkKTsKKwltdXRleF9leGl0KCZjcHVfbG9jayk7CisJ c3BhLT5zcGFfZGVhZG1hbl9jeWNpZCA9IENZQ0xJQ19OT05FOworI2Vsc2UJLyogRnJlZUJT RCAqLworI2lmZGVmIF9LRVJORUwKKwljYWxsb3V0X2RyYWluKCZzcGEtPnNwYV9kZWFkbWFu X2N5Y2lkKTsKKyNlbmRpZgorI2VuZGlmCisKIAlyZWZjb3VudF9kZXN0cm95KCZzcGEtPnNw YV9yZWZjb3VudCk7CiAKIAlzcGFfY29uZmlnX2xvY2tfZGVzdHJveShzcGEpOwpAQCAtMTUx MSw2ICsxNjAwLDEyIEBAIHNwYV9wcmV2X3NvZnR3YXJlX3ZlcnNpb24oc3BhX3QgKnNwYSkK IH0KIAogdWludDY0X3QKK3NwYV9kZWFkbWFuX3N5bmN0aW1lKHNwYV90ICpzcGEpCit7CisJ cmV0dXJuIChzcGEtPnNwYV9kZWFkbWFuX3N5bmN0aW1lKTsKK30KKwordWludDY0X3QKIGR2 YV9nZXRfZHNpemVfc3luYyhzcGFfdCAqc3BhLCBjb25zdCBkdmFfdCAqZHZhKQogewogCXVp bnQ2NF90IGFzaXplID0gRFZBX0dFVF9BU0laRShkdmEpOwpAQCAtMTYwNSw3ICsxNzAwLDkg QEAgc3BhX2luaXQoaW50IG1vZGUpCiAJc3BhX21vZGVfZ2xvYmFsID0gbW9kZTsKIAogI2lm ZGVmIGlsbHVtb3MKLSNpZm5kZWYgX0tFUk5FTAorI2lmZGVmIF9LRVJORUwKKwlzcGFfYXJj aF9pbml0KCk7CisjZWxzZQogCWlmIChzcGFfbW9kZV9nbG9iYWwgIT0gRlJFQUQgJiYgZHBy aW50Zl9maW5kX3N0cmluZygid2F0Y2giKSkgewogCQlhcmNfcHJvY2ZkID0gb3BlbigiL3By b2Mvc2VsZi9jdGwiLCBPX1dST05MWSk7CiAJCWlmIChhcmNfcHJvY2ZkID09IC0xKSB7Cklu ZGV4OiBzeXMvY2RkbC9jb250cmliL29wZW5zb2xhcmlzL3V0cy9jb21tb24vZnMvemZzL3N5 cy9zcGEuaAo9PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09Ci0tLSBzeXMvY2RkbC9jb250cmliL29wZW5zb2xhcmlz L3V0cy9jb21tb24vZnMvemZzL3N5cy9zcGEuaAkocmV2aXNpb24gMjQ3MTg3KQorKysgc3lz L2NkZGwvY29udHJpYi9vcGVuc29sYXJpcy91dHMvY29tbW9uL2ZzL3pmcy9zeXMvc3BhLmgJ KHdvcmtpbmcgY29weSkKQEAgLTU5OSw2ICs1OTksNyBAQCBleHRlcm4gYm9vbGVhbl90IHNw YV9zdXNwZW5kZWQoc3BhX3QgKnNwYSk7CiBleHRlcm4gdWludDY0X3Qgc3BhX2Jvb3Rmcyhz cGFfdCAqc3BhKTsKIGV4dGVybiB1aW50NjRfdCBzcGFfZGVsZWdhdGlvbihzcGFfdCAqc3Bh KTsKIGV4dGVybiBvYmpzZXRfdCAqc3BhX21ldGFfb2Jqc2V0KHNwYV90ICpzcGEpOworZXh0 ZXJuIHVpbnQ2NF90IHNwYV9kZWFkbWFuX3N5bmN0aW1lKHNwYV90ICpzcGEpOwogCiAvKiBN aXNjZWxsYW5lb3VzIHN1cHBvcnQgcm91dGluZXMgKi8KIGV4dGVybiB2b2lkIHNwYV9hY3Rp dmF0ZV9tb3NfZmVhdHVyZShzcGFfdCAqc3BhLCBjb25zdCBjaGFyICpmZWF0dXJlKTsKSW5k ZXg6IHN5cy9jZGRsL2NvbnRyaWIvb3BlbnNvbGFyaXMvdXRzL2NvbW1vbi9mcy96ZnMvc3lz L3NwYV9ib290LmgKPT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PQotLS0gc3lzL2NkZGwvY29udHJpYi9vcGVuc29s YXJpcy91dHMvY29tbW9uL2ZzL3pmcy9zeXMvc3BhX2Jvb3QuaAkocmV2aXNpb24gMjQ3MTg3 KQorKysgc3lzL2NkZGwvY29udHJpYi9vcGVuc29sYXJpcy91dHMvY29tbW9uL2ZzL3pmcy9z eXMvc3BhX2Jvb3QuaAkod29ya2luZyBjb3B5KQpAQCAtMjMsNiArMjMsMTAgQEAKICAqIFVz ZSBpcyBzdWJqZWN0IHRvIGxpY2Vuc2UgdGVybXMuCiAgKi8KIAorLyoKKyAqIENvcHlyaWdo dCAoYykgMjAxMiBieSBEZWxwaGl4LiBBbGwgcmlnaHRzIHJlc2VydmVkLgorICovCisKICNp Zm5kZWYgX1NZU19TUEFfQk9PVF9ICiAjZGVmaW5lCV9TWVNfU1BBX0JPT1RfSAogCkBAIC0z NSw2ICszOSw4IEBAIGV4dGVybiAiQyIgewogZXh0ZXJuIGNoYXIgKnNwYV9nZXRfYm9vdHBy b3AoY2hhciAqcHJvcCk7CiBleHRlcm4gdm9pZCBzcGFfZnJlZV9ib290cHJvcChjaGFyICpw cm9wKTsKIAorZXh0ZXJuIHZvaWQgc3BhX2FyY2hfaW5pdCh2b2lkKTsKKwogI2lmZGVmCV9f Y3BsdXNwbHVzCiB9CiAjZW5kaWYKSW5kZXg6IHN5cy9jZGRsL2NvbnRyaWIvb3BlbnNvbGFy aXMvdXRzL2NvbW1vbi9mcy96ZnMvc3lzL3NwYV9pbXBsLmgKPT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PQotLS0g c3lzL2NkZGwvY29udHJpYi9vcGVuc29sYXJpcy91dHMvY29tbW9uL2ZzL3pmcy9zeXMvc3Bh X2ltcGwuaAkocmV2aXNpb24gMjQ3MTg3KQorKysgc3lzL2NkZGwvY29udHJpYi9vcGVuc29s YXJpcy91dHMvY29tbW9uL2ZzL3pmcy9zeXMvc3BhX2ltcGwuaAkod29ya2luZyBjb3B5KQpA QCAtMjMwLDYgKzIzMCwxNiBAQCBzdHJ1Y3Qgc3BhIHsKIAl1aW50NjRfdAlzcGFfZmVhdF9m b3Jfd3JpdGVfb2JqOwkvKiByZXF1aXJlZCB0byB3cml0ZSB0byBwb29sICovCiAJdWludDY0 X3QJc3BhX2ZlYXRfZm9yX3JlYWRfb2JqOwkvKiByZXF1aXJlZCB0byByZWFkIGZyb20gcG9v bCAqLwogCXVpbnQ2NF90CXNwYV9mZWF0X2Rlc2Nfb2JqOwkvKiBGZWF0dXJlIGRlc2NyaXB0 aW9ucyAqLworI2lmZGVmIGlsbHVtb3MKKwljeWNsaWNfaWRfdAlzcGFfZGVhZG1hbl9jeWNp ZDsJLyogY3ljbGljIGlkICovCisjZWxzZQkvKiBGcmVlQlNEICovCisjaWZkZWYgX0tFUk5F TAorCXN0cnVjdCBjYWxsb3V0CXNwYV9kZWFkbWFuX2N5Y2lkOwkvKiBjYWxsb3V0IGlkICov CisjZW5kaWYKKyNlbmRpZgkvKiBpbGx1bW9zICovCisJdWludDY0X3QJc3BhX2RlYWRtYW5f Y2FsbHM7CS8qIG51bWJlciBvZiBkZWFkbWFuIGNhbGxzICovCisJdWludDY0X3QJc3BhX3N5 bmNfc3RhcnR0aW1lOwkvKiBzdGFydGluZyB0aW1lIGZvIHNwYV9zeW5jICovCisJdWludDY0 X3QJc3BhX2RlYWRtYW5fc3luY3RpbWU7CS8qIGRlYWRtYW4gZXhwaXJhdGlvbiB0aW1lciAq LwogCS8qCiAJICogc3BhX3JlZmNudCAmIHNwYV9jb25maWdfbG9jayBtdXN0IGJlIHRoZSBs YXN0IGVsZW1lbnRzCiAJICogYmVjYXVzZSByZWZjb3VudF90IGNoYW5nZXMgc2l6ZSBiYXNl ZCBvbiBjb21waWxhdGlvbiBvcHRpb25zLgpJbmRleDogc3lzL2NkZGwvY29udHJpYi9vcGVu c29sYXJpcy91dHMvY29tbW9uL2ZzL3pmcy9zeXMvdmRldi5oCj09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT0KLS0t IHN5cy9jZGRsL2NvbnRyaWIvb3BlbnNvbGFyaXMvdXRzL2NvbW1vbi9mcy96ZnMvc3lzL3Zk ZXYuaAkocmV2aXNpb24gMjQ3MTg3KQorKysgc3lzL2NkZGwvY29udHJpYi9vcGVuc29sYXJp cy91dHMvY29tbW9uL2ZzL3pmcy9zeXMvdmRldi5oCSh3b3JraW5nIGNvcHkpCkBAIC04MCw2 ICs4MCw3IEBAIGV4dGVybiB2b2lkIHZkZXZfbWV0YXNsYWJfZmluaSh2ZGV2X3QgKnZkKTsK IGV4dGVybiB2b2lkIHZkZXZfbWV0YXNsYWJfc2V0X3NpemUodmRldl90ICopOwogZXh0ZXJu IHZvaWQgdmRldl9leHBhbmQodmRldl90ICp2ZCwgdWludDY0X3QgdHhnKTsKIGV4dGVybiB2 b2lkIHZkZXZfc3BsaXQodmRldl90ICp2ZCk7CitleHRlcm4gdm9pZCB2ZGV2X2RlYWRtYW4o dmRldl90ICp2ZCk7CiAKIAogZXh0ZXJuIHZvaWQgdmRldl9nZXRfc3RhdHModmRldl90ICp2 ZCwgdmRldl9zdGF0X3QgKnZzKTsKSW5kZXg6IHN5cy9jZGRsL2NvbnRyaWIvb3BlbnNvbGFy aXMvdXRzL2NvbW1vbi9mcy96ZnMvc3lzL3ZkZXZfaW1wbC5oCj09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT0KLS0t IHN5cy9jZGRsL2NvbnRyaWIvb3BlbnNvbGFyaXMvdXRzL2NvbW1vbi9mcy96ZnMvc3lzL3Zk ZXZfaW1wbC5oCShyZXZpc2lvbiAyNDcxODcpCisrKyBzeXMvY2RkbC9jb250cmliL29wZW5z b2xhcmlzL3V0cy9jb21tb24vZnMvemZzL3N5cy92ZGV2X2ltcGwuaAkod29ya2luZyBjb3B5 KQpAQCAtMTA0LDYgKzEwNCw4IEBAIHN0cnVjdCB2ZGV2X3F1ZXVlIHsKIAlhdmxfdHJlZV90 CXZxX3JlYWRfdHJlZTsKIAlhdmxfdHJlZV90CXZxX3dyaXRlX3RyZWU7CiAJYXZsX3RyZWVf dAl2cV9wZW5kaW5nX3RyZWU7CisJdWludDY0X3QJdnFfaW9fY29tcGxldGVfdHM7CisJdWlu dDY0X3QJdnFfaW9fZGVsdGFfdHM7CiAJa211dGV4X3QJdnFfbG9jazsKIH07CiAKSW5kZXg6 IHN5cy9jZGRsL2NvbnRyaWIvb3BlbnNvbGFyaXMvdXRzL2NvbW1vbi9mcy96ZnMvc3lzL3pm c19jb250ZXh0LmgKPT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PQotLS0gc3lzL2NkZGwvY29udHJpYi9vcGVuc29s YXJpcy91dHMvY29tbW9uL2ZzL3pmcy9zeXMvemZzX2NvbnRleHQuaAkocmV2aXNpb24gMjQ3 MTg3KQorKysgc3lzL2NkZGwvY29udHJpYi9vcGVuc29sYXJpcy91dHMvY29tbW9uL2ZzL3pm cy9zeXMvemZzX2NvbnRleHQuaAkod29ya2luZyBjb3B5KQpAQCAtMjIsNiArMjIsMTAgQEAK ICAqIENvcHlyaWdodCAyMDA5IFN1biBNaWNyb3N5c3RlbXMsIEluYy4gIEFsbCByaWdodHMg cmVzZXJ2ZWQuCiAgKiBVc2UgaXMgc3ViamVjdCB0byBsaWNlbnNlIHRlcm1zLgogICovCisv KgorICogQ29weXJpZ2h0IDIwMTEgTmV4ZW50YSBTeXN0ZW1zLCBJbmMuICBBbGwgcmlnaHRz IHJlc2VydmVkLgorICogQ29weXJpZ2h0IChjKSAyMDEyIGJ5IERlbHBoaXguIEFsbCByaWdo dHMgcmVzZXJ2ZWQuCisgKi8KIAogI2lmbmRlZiBfU1lTX1pGU19DT05URVhUX0gKICNkZWZp bmUJX1NZU19aRlNfQ09OVEVYVF9ICkBAIC04OCw2ICs5MiwxMSBAQCBleHRlcm4gIkMiIHsK ICNpbmNsdWRlIDxzeXMvdThfdGV4dHByZXAuaD4KICNpbmNsdWRlIDxzeXMvZm0vdXRpbC5o PgogI2luY2x1ZGUgPHN5cy9zdW5kZGkuaD4KKyNpZmRlZiBpbGx1bW9zCisjaW5jbHVkZSA8 c3lzL2N5Y2xpYy5oPgorI2Vsc2UJLyogRnJlZUJTRCAqLworI2luY2x1ZGUgPHN5cy9jYWxs b3V0Lmg+CisjZW5kaWYKIAogI2luY2x1ZGUgPG1hY2hpbmUvc3RkYXJnLmg+CiAKSW5kZXg6 IHN5cy9jZGRsL2NvbnRyaWIvb3BlbnNvbGFyaXMvdXRzL2NvbW1vbi9mcy96ZnMvc3lzL3pm c19pb2N0bC5oCj09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT0KLS0tIHN5cy9jZGRsL2NvbnRyaWIvb3BlbnNvbGFy aXMvdXRzL2NvbW1vbi9mcy96ZnMvc3lzL3pmc19pb2N0bC5oCShyZXZpc2lvbiAyNDcxODcp CisrKyBzeXMvY2RkbC9jb250cmliL29wZW5zb2xhcmlzL3V0cy9jb21tb24vZnMvemZzL3N5 cy96ZnNfaW9jdGwuaAkod29ya2luZyBjb3B5KQpAQCAtMjQ2LDEyICsyNDYsMjQgQEAgdHlw ZWRlZiBzdHJ1Y3QgemluamVjdF9yZWNvcmQgewogCXVpbnQzMl90CXppX2lvdHlwZTsKIAlp bnQzMl90CQl6aV9kdXJhdGlvbjsKIAl1aW50NjRfdAl6aV90aW1lcjsKKwl1aW50MzJfdAl6 aV9jbWQ7CisJdWludDMyX3QJemlfcGFkOwogfSB6aW5qZWN0X3JlY29yZF90OwogCiAjZGVm aW5lCVpJTkpFQ1RfTlVMTAkJMHgxCiAjZGVmaW5lCVpJTkpFQ1RfRkxVU0hfQVJDCTB4Mgog I2RlZmluZQlaSU5KRUNUX1VOTE9BRF9TUEEJMHg0CiAKK3R5cGVkZWYgZW51bSB6aW5qZWN0 X3R5cGUgeworCVpJTkpFQ1RfVU5JTklUSUFMSVpFRCwKKwlaSU5KRUNUX0RBVEFfRkFVTFQs CisJWklOSkVDVF9ERVZJQ0VfRkFVTFQsCisJWklOSkVDVF9MQUJFTF9GQVVMVCwKKwlaSU5K RUNUX0lHTk9SRURfV1JJVEVTLAorCVpJTkpFQ1RfUEFOSUMsCisJWklOSkVDVF9ERUxBWV9J TywKK30gemluamVjdF90eXBlX3Q7CisKIHR5cGVkZWYgc3RydWN0IHpmc19zaGFyZSB7CiAJ dWludDY0X3QJel9leHBvcnRkYXRhOwogCXVpbnQ2NF90CXpfc2hhcmVkYXRhOwpJbmRleDog c3lzL2NkZGwvY29udHJpYi9vcGVuc29sYXJpcy91dHMvY29tbW9uL2ZzL3pmcy9zeXMvemlv LmgKPT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PQotLS0gc3lzL2NkZGwvY29udHJpYi9vcGVuc29sYXJpcy91dHMv Y29tbW9uL2ZzL3pmcy9zeXMvemlvLmgJKHJldmlzaW9uIDI0NzE4NykKKysrIHN5cy9jZGRs L2NvbnRyaWIvb3BlbnNvbGFyaXMvdXRzL2NvbW1vbi9mcy96ZnMvc3lzL3ppby5oCSh3b3Jr aW5nIGNvcHkpCkBAIC00NDMsNiArNDQzLDcgQEAgc3RydWN0IHppbyB7CiAKIAl1aW50NjRf dAlpb19vZmZzZXQ7CiAJdWludDY0X3QJaW9fZGVhZGxpbmU7CisJdWludDY0X3QJaW9fdGlt ZXN0YW1wOwogCWF2bF9ub2RlX3QJaW9fb2Zmc2V0X25vZGU7CiAJYXZsX25vZGVfdAlpb19k ZWFkbGluZV9ub2RlOwogCWF2bF90cmVlX3QJKmlvX3ZkZXZfdHJlZTsKQEAgLTU5Niw2ICs1 OTcsNyBAQCBleHRlcm4gaW50IHppb19oYW5kbGVfZmF1bHRfaW5qZWN0aW9uKHppb190ICp6 aW8sCiBleHRlcm4gaW50IHppb19oYW5kbGVfZGV2aWNlX2luamVjdGlvbih2ZGV2X3QgKnZk LCB6aW9fdCAqemlvLCBpbnQgZXJyb3IpOwogZXh0ZXJuIGludCB6aW9faGFuZGxlX2xhYmVs X2luamVjdGlvbih6aW9fdCAqemlvLCBpbnQgZXJyb3IpOwogZXh0ZXJuIHZvaWQgemlvX2hh bmRsZV9pZ25vcmVkX3dyaXRlcyh6aW9fdCAqemlvKTsKK2V4dGVybiB1aW50NjRfdCB6aW9f aGFuZGxlX2lvX2RlbGF5KHppb190ICp6aW8pOwogCiAvKgogICogQ2hlY2tzdW0gZXJlcG9y dCBmdW5jdGlvbnMKSW5kZXg6IHN5cy9jZGRsL2NvbnRyaWIvb3BlbnNvbGFyaXMvdXRzL2Nv bW1vbi9mcy96ZnMvdmRldi5jCj09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT0KLS0tIHN5cy9jZGRsL2NvbnRyaWIv b3BlbnNvbGFyaXMvdXRzL2NvbW1vbi9mcy96ZnMvdmRldi5jCShyZXZpc2lvbiAyNDcxODcp CisrKyBzeXMvY2RkbC9jb250cmliL29wZW5zb2xhcmlzL3V0cy9jb21tb24vZnMvemZzL3Zk ZXYuYwkod29ya2luZyBjb3B5KQpAQCAtMzE3MywzICszMTczLDQxIEBAIHZkZXZfc3BsaXQo dmRldl90ICp2ZCkKIAl9CiAJdmRldl9wcm9wYWdhdGVfc3RhdGUoY3ZkKTsKIH0KKwordm9p ZAordmRldl9kZWFkbWFuKHZkZXZfdCAqdmQpCit7CisJZm9yIChpbnQgYyA9IDA7IGMgPCB2 ZC0+dmRldl9jaGlsZHJlbjsgYysrKSB7CisJCXZkZXZfdCAqY3ZkID0gdmQtPnZkZXZfY2hp bGRbY107CisKKwkJdmRldl9kZWFkbWFuKGN2ZCk7CisJfQorCisJaWYgKHZkLT52ZGV2X29w cy0+dmRldl9vcF9sZWFmKSB7CisJCXZkZXZfcXVldWVfdCAqdnEgPSAmdmQtPnZkZXZfcXVl dWU7CisKKwkJbXV0ZXhfZW50ZXIoJnZxLT52cV9sb2NrKTsKKwkJaWYgKGF2bF9udW1ub2Rl cygmdnEtPnZxX3BlbmRpbmdfdHJlZSkgPiAwKSB7CisJCQlzcGFfdCAqc3BhID0gdmQtPnZk ZXZfc3BhOworCQkJemlvX3QgKmZpbzsKKwkJCXVpbnQ2NF90IGRlbHRhOworCisJCQkvKgor CQkJICogTG9vayBhdCB0aGUgaGVhZCBvZiBhbGwgdGhlIHBlbmRpbmcgcXVldWVzLAorCQkJ ICogaWYgYW55IEkvTyBoYXMgYmVlbiBvdXRzdGFuZGluZyBmb3IgbG9uZ2VyIHRoYW4KKwkJ CSAqIHRoZSBzcGFfZGVhZG1hbl9zeW5jdGltZSB3ZSBwYW5pYyB0aGUgc3lzdGVtLgorCQkJ ICovCisJCQlmaW8gPSBhdmxfZmlyc3QoJnZxLT52cV9wZW5kaW5nX3RyZWUpOworCQkJZGVs dGEgPSBkZGlfZ2V0X2xib2x0NjQoKSAtIGZpby0+aW9fdGltZXN0YW1wOworCQkJaWYgKGRl bHRhID4gTlNFQ19UT19USUNLKHNwYV9kZWFkbWFuX3N5bmN0aW1lKHNwYSkpKSB7CisJCQkJ emZzX2RiZ21zZygiU0xPVyBJTzogemlvIHRpbWVzdGFtcCAlbGx1LCAiCisJCQkJICAgICJk ZWx0YSAlbGx1LCBsYXN0IGlvICVsbHUiLAorCQkJCSAgICBmaW8tPmlvX3RpbWVzdGFtcCwg ZGVsdGEsCisJCQkJICAgIHZxLT52cV9pb19jb21wbGV0ZV90cyk7CisJCQkJZm1fcGFuaWMo IkkvTyB0byBwb29sICclcycgYXBwZWFycyB0byBiZSAiCisJCQkJICAgICJodW5nLiIsIHNw YV9uYW1lKHNwYSkpOworCQkJfQorCQl9CisJCW11dGV4X2V4aXQoJnZxLT52cV9sb2NrKTsK Kwl9Cit9CkluZGV4OiBzeXMvY2RkbC9jb250cmliL29wZW5zb2xhcmlzL3V0cy9jb21tb24v ZnMvemZzL3ZkZXZfcXVldWUuYwo9PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09Ci0tLSBzeXMvY2RkbC9jb250cmli L29wZW5zb2xhcmlzL3V0cy9jb21tb24vZnMvemZzL3ZkZXZfcXVldWUuYwkocmV2aXNpb24g MjQ3MTg3KQorKysgc3lzL2NkZGwvY29udHJpYi9vcGVuc29sYXJpcy91dHMvY29tbW9uL2Zz L3pmcy92ZGV2X3F1ZXVlLmMJKHdvcmtpbmcgY29weSkKQEAgLTIzLDYgKzIzLDEwIEBACiAg KiBVc2UgaXMgc3ViamVjdCB0byBsaWNlbnNlIHRlcm1zLgogICovCiAKKy8qCisgKiBDb3B5 cmlnaHQgKGMpIDIwMTIgYnkgRGVscGhpeC4gQWxsIHJpZ2h0cyByZXNlcnZlZC4KKyAqLwor CiAjaW5jbHVkZSA8c3lzL3pmc19jb250ZXh0Lmg+CiAjaW5jbHVkZSA8c3lzL3ZkZXZfaW1w bC5oPgogI2luY2x1ZGUgPHN5cy96aW8uaD4KQEAgLTMxNSw2ICszMTksNyBAQCBhZ2FpbjoK IAkJICAgIHppb19idWZfYWxsb2Moc2l6ZSksIHNpemUsIGZpby0+aW9fdHlwZSwgWklPX1BS SU9SSVRZX0FHRywKIAkJICAgIGZsYWdzIHwgWklPX0ZMQUdfRE9OVF9DQUNIRSB8IFpJT19G TEFHX0RPTlRfUVVFVUUsCiAJCSAgICB2ZGV2X3F1ZXVlX2FnZ19pb19kb25lLCBOVUxMKTsK KwkJYWlvLT5pb190aW1lc3RhbXAgPSBmaW8tPmlvX3RpbWVzdGFtcDsKIAogCQluaW8gPSBm aW87CiAJCWRvIHsKQEAgLTM4Niw3ICszOTEsOCBAQCB2ZGV2X3F1ZXVlX2lvKHppb190ICp6 aW8pCiAKIAltdXRleF9lbnRlcigmdnEtPnZxX2xvY2spOwogCi0JemlvLT5pb19kZWFkbGlu ZSA9IChkZGlfZ2V0X2xib2x0NjQoKSA+PiB6ZnNfdmRldl90aW1lX3NoaWZ0KSArCisJemlv LT5pb190aW1lc3RhbXAgPSBkZGlfZ2V0X2xib2x0NjQoKTsKKwl6aW8tPmlvX2RlYWRsaW5l ID0gKHppby0+aW9fdGltZXN0YW1wID4+IHpmc192ZGV2X3RpbWVfc2hpZnQpICsKIAkgICAg emlvLT5pb19wcmlvcml0eTsKIAogCXZkZXZfcXVldWVfaW9fYWRkKHZxLCB6aW8pOwpAQCAt NDExLDEwICs0MTcsMTYgQEAgdmRldl9xdWV1ZV9pb19kb25lKHppb190ICp6aW8pCiB7CiAJ dmRldl9xdWV1ZV90ICp2cSA9ICZ6aW8tPmlvX3ZkLT52ZGV2X3F1ZXVlOwogCisJaWYgKHpp b19pbmplY3Rpb25fZW5hYmxlZCkKKwkJZGVsYXkoU0VDX1RPX1RJQ0soemlvX2hhbmRsZV9p b19kZWxheSh6aW8pKSk7CisKIAltdXRleF9lbnRlcigmdnEtPnZxX2xvY2spOwogCiAJYXZs X3JlbW92ZSgmdnEtPnZxX3BlbmRpbmdfdHJlZSwgemlvKTsKIAorCXZxLT52cV9pb19jb21w bGV0ZV90cyA9IGRkaV9nZXRfbGJvbHQ2NCgpOworCXZxLT52cV9pb19kZWx0YV90cyA9IHZx LT52cV9pb19jb21wbGV0ZV90cyAtIHppby0+aW9fdGltZXN0YW1wOworCiAJZm9yIChpbnQg aSA9IDA7IGkgPCB6ZnNfdmRldl9yYW1wX3JhdGU7IGkrKykgewogCQl6aW9fdCAqbmlvID0g dmRldl9xdWV1ZV9pb190b19pc3N1ZSh2cSwgemZzX3ZkZXZfbWF4X3BlbmRpbmcpOwogCQlp ZiAobmlvID09IE5VTEwpCkluZGV4OiBzeXMvY2RkbC9jb250cmliL29wZW5zb2xhcmlzL3V0 cy9jb21tb24vZnMvemZzL3ppb19pbmplY3QuYwo9PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09Ci0tLSBzeXMvY2Rk bC9jb250cmliL29wZW5zb2xhcmlzL3V0cy9jb21tb24vZnMvemZzL3ppb19pbmplY3QuYwko cmV2aXNpb24gMjQ3MTg3KQorKysgc3lzL2NkZGwvY29udHJpYi9vcGVuc29sYXJpcy91dHMv Y29tbW9uL2ZzL3pmcy96aW9faW5qZWN0LmMJKHdvcmtpbmcgY29weSkKQEAgLTIwLDYgKzIw LDcgQEAKICAqLwogLyoKICAqIENvcHlyaWdodCAoYykgMjAwNSwgMjAxMCwgT3JhY2xlIGFu ZC9vciBpdHMgYWZmaWxpYXRlcy4gQWxsIHJpZ2h0cyByZXNlcnZlZC4KKyAqIENvcHlyaWdo dCAoYykgMjAxMiBieSBEZWxwaGl4LiBBbGwgcmlnaHRzIHJlc2VydmVkLgogICovCiAKIC8q CkBAIC0xNDcsMTYgKzE0OCwxMCBAQCB6aW9faGFuZGxlX2ZhdWx0X2luamVjdGlvbih6aW9f dCAqemlvLCBpbnQgZXJyb3IpCiAJZm9yIChoYW5kbGVyID0gbGlzdF9oZWFkKCZpbmplY3Rf aGFuZGxlcnMpOyBoYW5kbGVyICE9IE5VTEw7CiAJICAgIGhhbmRsZXIgPSBsaXN0X25leHQo JmluamVjdF9oYW5kbGVycywgaGFuZGxlcikpIHsKIAotCQkvKiBJZ25vcmUgZXJyb3JzIG5v dCBkZXN0aW5lZCBmb3IgdGhpcyBwb29sICovCi0JCWlmICh6aW8tPmlvX3NwYSAhPSBoYW5k bGVyLT56aV9zcGEpCisJCWlmICh6aW8tPmlvX3NwYSAhPSBoYW5kbGVyLT56aV9zcGEgfHwK KwkJICAgIGhhbmRsZXItPnppX3JlY29yZC56aV9jbWQgIT0gWklOSkVDVF9EQVRBX0ZBVUxU KQogCQkJY29udGludWU7CiAKLQkJLyogSWdub3JlIGRldmljZSBlcnJvcnMgYW5kIHBhbmlj IGluamVjdGlvbiAqLwotCQlpZiAoaGFuZGxlci0+emlfcmVjb3JkLnppX2d1aWQgIT0gMCB8 fAotCQkgICAgaGFuZGxlci0+emlfcmVjb3JkLnppX2Z1bmNbMF0gIT0gJ1wwJyB8fAotCQkg ICAgaGFuZGxlci0+emlfcmVjb3JkLnppX2R1cmF0aW9uICE9IDApCi0JCQljb250aW51ZTsK LQogCQkvKiBJZiB0aGlzIGhhbmRsZXIgbWF0Y2hlcywgcmV0dXJuIEVJTyAqLwogCQlpZiAo emlvX21hdGNoX2hhbmRsZXIoJnppby0+aW9fbG9naWNhbC0+aW9fYm9va21hcmssCiAJCSAg ICB6aW8tPmlvX2JwID8gQlBfR0VUX1RZUEUoemlvLT5pb19icCkgOiBETVVfT1RfTk9ORSwK QEAgLTE5NywxMCArMTkyLDcgQEAgemlvX2hhbmRsZV9sYWJlbF9pbmplY3Rpb24oemlvX3Qg KnppbywgaW50IGVycm9yKQogCQl1aW50NjRfdCBzdGFydCA9IGhhbmRsZXItPnppX3JlY29y ZC56aV9zdGFydDsKIAkJdWludDY0X3QgZW5kID0gaGFuZGxlci0+emlfcmVjb3JkLnppX2Vu ZDsKIAotCQkvKiBJZ25vcmUgZGV2aWNlIG9ubHkgZmF1bHRzIG9yIHBhbmljIGluamVjdGlv biAqLwotCQlpZiAoaGFuZGxlci0+emlfcmVjb3JkLnppX3N0YXJ0ID09IDAgfHwKLQkJICAg IGhhbmRsZXItPnppX3JlY29yZC56aV9mdW5jWzBdICE9ICdcMCcgfHwKLQkJICAgIGhhbmRs ZXItPnppX3JlY29yZC56aV9kdXJhdGlvbiAhPSAwKQorCQlpZiAoaGFuZGxlci0+emlfcmVj b3JkLnppX2NtZCAhPSBaSU5KRUNUX0xBQkVMX0ZBVUxUKQogCQkJY29udGludWU7CiAKIAkJ LyoKQEAgLTI0NiwxMyArMjM4LDcgQEAgemlvX2hhbmRsZV9kZXZpY2VfaW5qZWN0aW9uKHZk ZXZfdCAqdmQsIHppb190ICp6aW8KIAlmb3IgKGhhbmRsZXIgPSBsaXN0X2hlYWQoJmluamVj dF9oYW5kbGVycyk7IGhhbmRsZXIgIT0gTlVMTDsKIAkgICAgaGFuZGxlciA9IGxpc3RfbmV4 dCgmaW5qZWN0X2hhbmRsZXJzLCBoYW5kbGVyKSkgewogCi0JCS8qCi0JCSAqIElnbm9yZSBs YWJlbCBzcGVjaWZpYyBmYXVsdHMsIHBhbmljIGluamVjdGlvbgotCQkgKiBvciBmYWtlIHdy aXRlcwotCQkgKi8KLQkJaWYgKGhhbmRsZXItPnppX3JlY29yZC56aV9zdGFydCAhPSAwIHx8 Ci0JCSAgICBoYW5kbGVyLT56aV9yZWNvcmQuemlfZnVuY1swXSAhPSAnXDAnIHx8Ci0JCSAg ICBoYW5kbGVyLT56aV9yZWNvcmQuemlfZHVyYXRpb24gIT0gMCkKKwkJaWYgKGhhbmRsZXIt PnppX3JlY29yZC56aV9jbWQgIT0gWklOSkVDVF9ERVZJQ0VfRkFVTFQpCiAJCQljb250aW51 ZTsKIAogCQlpZiAodmQtPnZkZXZfZ3VpZCA9PSBoYW5kbGVyLT56aV9yZWNvcmQuemlfZ3Vp ZCkgewpAQCAtMzE2LDEyICszMDIsMTAgQEAgemlvX2hhbmRsZV9pZ25vcmVkX3dyaXRlcyh6 aW9fdCAqemlvKQogCSAgICBoYW5kbGVyID0gbGlzdF9uZXh0KCZpbmplY3RfaGFuZGxlcnMs IGhhbmRsZXIpKSB7CiAKIAkJLyogSWdub3JlIGVycm9ycyBub3QgZGVzdGluZWQgZm9yIHRo aXMgcG9vbCAqLwotCQlpZiAoemlvLT5pb19zcGEgIT0gaGFuZGxlci0+emlfc3BhKQorCQlp ZiAoemlvLT5pb19zcGEgIT0gaGFuZGxlci0+emlfc3BhIHx8CisJCSAgICBoYW5kbGVyLT56 aV9yZWNvcmQuemlfY21kICE9IFpJTkpFQ1RfSUdOT1JFRF9XUklURVMpCiAJCQljb250aW51 ZTsKIAotCQlpZiAoaGFuZGxlci0+emlfcmVjb3JkLnppX2R1cmF0aW9uID09IDApCi0JCQlj b250aW51ZTsKLQogCQkvKgogCQkgKiBQb3NpdGl2ZSBkdXJhdGlvbiBpbXBsaWVzICMgb2Yg c2Vjb25kcywgbmVnYXRpdmUKIAkJICogYSBudW1iZXIgb2YgdHhncwpAQCAtMzU1LDEzICsz MzksMTAgQEAgc3BhX2hhbmRsZV9pZ25vcmVkX3dyaXRlcyhzcGFfdCAqc3BhKQogCWZvciAo aGFuZGxlciA9IGxpc3RfaGVhZCgmaW5qZWN0X2hhbmRsZXJzKTsgaGFuZGxlciAhPSBOVUxM OwogCSAgICBoYW5kbGVyID0gbGlzdF9uZXh0KCZpbmplY3RfaGFuZGxlcnMsIGhhbmRsZXIp KSB7CiAKLQkJLyogSWdub3JlIGVycm9ycyBub3QgZGVzdGluZWQgZm9yIHRoaXMgcG9vbCAq LwotCQlpZiAoc3BhICE9IGhhbmRsZXItPnppX3NwYSkKKwkJaWYgKHNwYSAhPSBoYW5kbGVy LT56aV9zcGEgfHwKKwkJICAgIGhhbmRsZXItPnppX3JlY29yZC56aV9jbWQgIT0gWklOSkVD VF9JR05PUkVEX1dSSVRFUykKIAkJCWNvbnRpbnVlOwogCi0JCWlmIChoYW5kbGVyLT56aV9y ZWNvcmQuemlfZHVyYXRpb24gPT0gMCkKLQkJCWNvbnRpbnVlOwotCiAJCWlmIChoYW5kbGVy LT56aV9yZWNvcmQuemlfZHVyYXRpb24gPiAwKSB7CiAJCQlWRVJJRlkoaGFuZGxlci0+emlf cmVjb3JkLnppX3RpbWVyID09IDAgfHwKIAkJCSAgICBoYW5kbGVyLT56aV9yZWNvcmQuemlf dGltZXIgKwpAQCAtMzc5LDYgKzM2MCwzNCBAQCBzcGFfaGFuZGxlX2lnbm9yZWRfd3JpdGVz KHNwYV90ICpzcGEpCiAJcndfZXhpdCgmaW5qZWN0X2xvY2spOwogfQogCit1aW50NjRfdAor emlvX2hhbmRsZV9pb19kZWxheSh6aW9fdCAqemlvKQoreworCXZkZXZfdCAqdmQgPSB6aW8t PmlvX3ZkOworCWluamVjdF9oYW5kbGVyX3QgKmhhbmRsZXI7CisJdWludDY0X3Qgc2Vjb25k cyA9IDA7CisKKwlpZiAoemlvX2luamVjdGlvbl9lbmFibGVkID09IDApCisJCXJldHVybiAo MCk7CisKKwlyd19lbnRlcigmaW5qZWN0X2xvY2ssIFJXX1JFQURFUik7CisKKwlmb3IgKGhh bmRsZXIgPSBsaXN0X2hlYWQoJmluamVjdF9oYW5kbGVycyk7IGhhbmRsZXIgIT0gTlVMTDsK KwkgICAgaGFuZGxlciA9IGxpc3RfbmV4dCgmaW5qZWN0X2hhbmRsZXJzLCBoYW5kbGVyKSkg eworCisJCWlmIChoYW5kbGVyLT56aV9yZWNvcmQuemlfY21kICE9IFpJTkpFQ1RfREVMQVlf SU8pCisJCQljb250aW51ZTsKKworCQlpZiAodmQtPnZkZXZfZ3VpZCA9PSBoYW5kbGVyLT56 aV9yZWNvcmQuemlfZ3VpZCkgeworCQkJc2Vjb25kcyA9IGhhbmRsZXItPnppX3JlY29yZC56 aV90aW1lcjsKKwkJCWJyZWFrOworCQl9CisKKwl9CisJcndfZXhpdCgmaW5qZWN0X2xvY2sp OworCXJldHVybiAoc2Vjb25kcyk7Cit9CisKIC8qCiAgKiBDcmVhdGUgYSBuZXcgaGFuZGxl ciBmb3IgdGhlIGdpdmVuIHJlY29yZC4gIFdlIGFkZCBpdCB0byB0aGUgbGlzdCwgYWRkaW5n CiAgKiBhIHJlZmVyZW5jZSB0byB0aGUgc3BhX3QgaW4gdGhlIHByb2Nlc3MuICBXZSBpbmNy ZW1lbnQgemlvX2luamVjdGlvbl9lbmFibGVkLAo= --------------090105020004000108000805-- From owner-zfs-devel@FreeBSD.ORG Sat Feb 23 13:44:25 2013 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 232FDB88; Sat, 23 Feb 2013 13:44:25 +0000 (UTC) (envelope-from davide.italiano@gmail.com) Received: from mail-vb0-f54.google.com (mail-vb0-f54.google.com [209.85.212.54]) by mx1.freebsd.org (Postfix) with ESMTP id A33BBE1F; Sat, 23 Feb 2013 13:44:24 +0000 (UTC) Received: by mail-vb0-f54.google.com with SMTP id l1so967317vba.41 for ; Sat, 23 Feb 2013 05:44:18 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; bh=oxtgyFJVEFjZvJnyHaPd51HPoWyCYSQQ938poWyCYwI=; b=GCB1234ItazQhfJdwhJiskNdI3IQffi6KpuIln1vUG0IqI4X95LV/igVjFEqSWFbXI SAbny7g9UgvW/pPlNM5tSc4h5wqTzG9rLanAB5GaJPvAA10tSydBA7Ii/tNO304st1ZK rvD6dE4hUd90LM6MX8R2y7hfvbp19SB2yDd9oP3QIBCjXMyVJ1XL4T+H4ugBq6ZuUr1P c5Vg1XbUpWAYAowlU+5/+zW3A9jsDtAVcQnefLTBEBriaEeBd7Hdr64LmNOTgyCRfmr+ xVGEuSAsP90PbpO6HdoC+srsspHz3WBWwAKCd71FY72CLia5hepS74aGa+pTyv86HI+C ijAQ== MIME-Version: 1.0 X-Received: by 10.52.94.17 with SMTP id cy17mr6405648vdb.68.1361626562074; Sat, 23 Feb 2013 05:36:02 -0800 (PST) Sender: davide.italiano@gmail.com Received: by 10.220.34.203 with HTTP; Sat, 23 Feb 2013 05:36:01 -0800 (PST) In-Reply-To: <5128BCA9.6050801@FreeBSD.org> References: <5128BCA9.6050801@FreeBSD.org> Date: Sat, 23 Feb 2013 14:36:01 +0100 X-Google-Sender-Auth: puP-LGDWlUrgFNcAAyXrM91JS_Y Message-ID: Subject: Re: Vendor merge review: ZFS I/O deadman thread From: Davide Italiano To: Martin Matuska Content-Type: text/plain; charset=ISO-8859-1 Cc: zfs-devel@freebsd.org X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 23 Feb 2013 13:44:25 -0000 On Sat, Feb 23, 2013 at 1:57 PM, Martin Matuska wrote: > Please review the following proposed merge from vendor. > > The patch uses callout_drain(), I am not sure if callout_stop() is safe in this case. Assuming you want to maintain the same semantic as Illumos code, and also assuming the Illumos code is correct, I think the right choice here is callout_drain(). In fact, if I understand correctly the changes, Illumos code rely on cyclic_remove(), which guarantees that the removed cyclic handler has completed execution (blocking, if necessary). Also, I see the code rely on cyclic which provides high-resolution timers rather than coarse-grained callout. I'm not really sure if you need such high granularity, but in case do, you could consider migrating the patch to the new callout_* functions once they're committed (hopefully, next week or something like). > As of the patch, the deadman thread is disabled by default. Should we enable it? > > Thank you. > > http://people.freebsd.org/~mm/patches/zfs/zfs_deadman.patch > > Link to vendor ZFS issue: > 3246 ZFS I/O deadman thread > https://www.illumos.org/issues/3246 > > -- > Martin Matuska > FreeBSD committer > http://blog.vx.sk > > > _______________________________________________ > zfs-devel@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/zfs-devel > To unsubscribe, send any mail to "zfs-devel-unsubscribe@freebsd.org" Thanks, Davide From owner-zfs-devel@FreeBSD.ORG Sun Feb 24 22:22:21 2013 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id D47735F9; Sun, 24 Feb 2013 22:22:21 +0000 (UTC) (envelope-from delphij@delphij.net) Received: from anubis.delphij.net (anubis.delphij.net [IPv6:2001:470:1:117::25]) by mx1.freebsd.org (Postfix) with ESMTP id BC27B90D; Sun, 24 Feb 2013 22:22:21 +0000 (UTC) Received: from Xins-MacBook-Pro-2.local (c-67-188-85-47.hsd1.ca.comcast.net [67.188.85.47]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by anubis.delphij.net (Postfix) with ESMTPSA id 525012446D; Sun, 24 Feb 2013 14:22:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=delphij.net; s=anubis; t=1361744541; bh=Q1OUDkrcXJc7wy3Jw61nPFTxDhCXeHCDi+848iSwNSY=; h=Date:From:Reply-To:To:CC:Subject:References:In-Reply-To; b=KHuTMKFo2+t0bNmTSzysyNswoRIe/jC/dmUSabhchUwxDvsJfvkzQa97KgGr/V7PS 50KiWAAD5R/HqEn/JnmgclRZ7GQyhRunOZiEgvqYxp2Oulp6dng0VZEsw+Pw5tPijU QavgWSmYQNl0qAqSwFs1TGzFGuIv2lV9zP7U1Ffk= Message-ID: <512A929D.1050100@delphij.net> Date: Sun, 24 Feb 2013 14:22:21 -0800 From: Xin Li Organization: The FreeBSD Project MIME-Version: 1.0 To: Martin Matuska Subject: Re: Vendor merge review: ZFS I/O deadman thread References: <5128BCA9.6050801@FreeBSD.org> In-Reply-To: <5128BCA9.6050801@FreeBSD.org> X-Enigmail-Version: 1.5 Content-Type: text/plain; charset=windows-1250 Content-Transfer-Encoding: 7bit Cc: zfs-devel@FreeBSD.org X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list Reply-To: d@delphij.net List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 24 Feb 2013 22:22:21 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 On 2/23/13 4:57 AM, Martin Matuska wrote: > Please review the following proposed merge from vendor. > > The patch uses callout_drain(), I am not sure if callout_stop() is > safe in this case. As of the patch, the deadman thread is disabled > by default. Should we enable it? > > Thank you. > > http://people.freebsd.org/~mm/patches/zfs/zfs_deadman.patch > > Link to vendor ZFS issue: 3246 ZFS I/O deadman thread > https://www.illumos.org/issues/3246 (In the patch I didn't see callout_stop, I think you have already changed them to callout_drain per Davide's comments? I don't think this requires high resolution callout as it's a watchdog type thing) I think this should only be enabled when the system is on physical system so I think we could probably leave the default there and check it on module load (something like, if (zfs_deadman_enabled == -1) zfs_deadman_enabled = (vm_guest == 0)? 1:0;) The other parts seems fine. Cheers, -----BEGIN PGP SIGNATURE----- iQEcBAEBCAAGBQJRKpKcAAoJEG80Jeu8UPuz3M0H/RDaXuocJ3DYiqomGCnuwBwd 5XBjtpCKTdo2JfVPi0Xz4miqk5Cxd6ET27clmNgohgGGOrbBgretr9mwiVboO+nf V1Qt67ULabtAtOCElMO9ZaI5T2autt0ecrlMiHxXkMs29gpRAvREU7gPqoL16p8r LeCNBpbOIiXJXWxRJyY8x4FbmbgFI4SK4Co7t5dXz+UQJM5MfA5S4qGHBhqJfJXx x3+m2J99QNKeCvUDrp18DI2Y7xpv4nTiX7Xx81mfFE8PbMG0ItVjwkknQElxF8ZL H4CZq/OWB6tEcF9K3hMvOI4yodO9sIG5+s+jKUwKJU/l5TYGvBPc140bL9JisWk= =CMDj -----END PGP SIGNATURE----- From owner-zfs-devel@FreeBSD.ORG Fri Mar 8 02:59:57 2013 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 4A3EC4BA for ; Fri, 8 Mar 2013 02:59:57 +0000 (UTC) (envelope-from delphij@delphij.net) Received: from anubis.delphij.net (anubis.delphij.net [IPv6:2001:470:1:117::25]) by mx1.freebsd.org (Postfix) with ESMTP id D6A03273 for ; Fri, 8 Mar 2013 02:59:56 +0000 (UTC) Received: from epsilon.delphij.net (drawbridge.ixsystems.com [206.40.55.65]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by anubis.delphij.net (Postfix) with ESMTPSA id 4B0CE24265; Thu, 7 Mar 2013 18:59:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=delphij.net; s=anubis; t=1362711596; bh=UYHtII2eG8/CqgZwEsNkHmXgzCEolZDCDpAwdJjP2Qw=; h=Date:From:Reply-To:To:Subject; b=sWuA1fkO9ouqgooohMXyV2SNZkzn7pNYd1co+gyk7R6/CNAj4UrFbXLSJNlBGETS/ N/WShOn+11WRm1Yfq4LJeLxJB3P8WgznPW6FGo8/CS3ryZXlIxyLItYbBpfiE5o0zD cXTCd2j987we+Pi5HZW8zzCQHaGKj67nNBlUpyRc= Message-ID: <5139542B.6070905@delphij.net> Date: Thu, 07 Mar 2013 18:59:55 -0800 From: Xin Li Organization: The FreeBSD Project MIME-Version: 1.0 To: zfs-devel@freebsd.org Subject: space map corruption? X-Enigmail-Version: 1.5.1 Content-Type: multipart/mixed; boundary="------------010604030801070703070900" X-Content-Filtered-By: Mailman/MimeDel 2.1.14 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list Reply-To: d@delphij.net List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 Mar 2013 02:59:57 -0000 This is a multi-part message in MIME format. --------------010604030801070703070900 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 Hi, We have seen this panic this morning. The system refuses to import the pool read/write, but I was able to import with -F -f -o readonly=on -R /mnt. We have backups but can't keep this system down too long. Is there anything that can benefit tracing down the bug? (The system was running a slightly patched FreeBSD 9.1-RELEASE). Cheers, - -- Xin LI https://www.delphij.net/ FreeBSD - The Power to Serve! Live free or die -----BEGIN PGP SIGNATURE----- iQEcBAEBCgAGBQJROVQrAAoJEG80Jeu8UPuzIvYH/AsAo8CT3qSViWYMtNB/Mcdh d4t+LanjV6oMfKF85eZtlngjb3J4GRejqf2lJs3APbC2N6BL+7St734O8TLDJG+D sDbOTVoVb6+rd/EivwXGP2dw1c3fpIKnN+P01TqEiaH5OkwaN672X3bPx8S3DkeI 9bmDVxtjSDZ0C9SAvGNhug1XqnnLxbFL+oUEEex0bxjZ3LdcTpy8PnNGWA9UaZNq hy4CQ6GRZu91wiXwZI8FCc9JVHlvQQy+E+O3ZnJsEfFX0kOQP6JIEWJ5KiDiYEgd aFhvSUCmpmV5nmUmJ94eIitFp/9MKfCt5WCwZ+vjEzV10gujmABEWHkX25QWXOk= =oq3J -----END PGP SIGNATURE----- --------------010604030801070703070900-- From owner-zfs-devel@FreeBSD.ORG Wed Mar 13 21:21:22 2013 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 46BD3ABA for ; Wed, 13 Mar 2013 21:21:22 +0000 (UTC) (envelope-from will@firepipe.net) Received: from mail-ie0-x22e.google.com (mail-ie0-x22e.google.com [IPv6:2607:f8b0:4001:c03::22e]) by mx1.freebsd.org (Postfix) with ESMTP id 0C2C8B6F for ; Wed, 13 Mar 2013 21:21:22 +0000 (UTC) Received: by mail-ie0-f174.google.com with SMTP id k10so2094863iea.5 for ; Wed, 13 Mar 2013 14:21:21 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:x-received:date:message-id:subject:from:to :content-type:x-gm-message-state; bh=11JquZLT1HdicQ/cKvqEQ8JgFTDjw+iC45tPFQxg3v4=; b=DRP6wBmt0x1R/NGIS1pYqDPMg1hIptu7WB78NVD6h2PS7Tu7nRoi3zMdBWJkaozJrs w8WGYoV3x6vvOTpALKptskgRhscs0YT+l+Adfs+EkCwwdZiQVWo4blQC70o5530majLs eKeQ+PbB0tFbVbLUCDB7QtUDI4OXdQGwIOrPKzaYaHIl2omEy3a43N2J1Fm51b8j6C7Z /tldGsildDR8kX/SDSRfAflWkPScSdsCtKCtkMCGP+Ffcxy3KhxWVvhLul73P/PDI+4V cf8IX8XOkKiOyK9XxB4ZGmWdj2fwGxWKHlhotWprGmkee97QjC8L+c3vkmfZj1eSuq+c cjJg== MIME-Version: 1.0 X-Received: by 10.50.34.231 with SMTP id c7mr17258152igj.95.1363209681716; Wed, 13 Mar 2013 14:21:21 -0700 (PDT) Received: by 10.231.141.10 with HTTP; Wed, 13 Mar 2013 14:21:21 -0700 (PDT) Date: Wed, 13 Mar 2013 15:21:21 -0600 Message-ID: Subject: CFR: Fix panic in zfs_umount() due to dropped root vnode refcount From: Will Andrews To: zfs-devel@freebsd.org X-Gm-Message-State: ALoCoQkq1CwWTFlQOdxjTMFFB4RWEcL6uWHODl3VW5xBSNicoETKzh6Y/6Yu0b6Z5o1VleOs9r1v Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Mar 2013 21:21:22 -0000 Hi, Diff: http://people.freebsd.org/~will/zfs/zfs_vfsops.c-fix-unmount-panic.diff Change 660985 by willa@willa_SpectraBSD on 2013/03/08 17:43:49 Fix the zfs unmount panic that is due to a missed refcount drop. When checking for activity in a non-forced unmount, remember that we called vflush() releasing the refcount that zfs_mount() placed on the filesystem's root vnode using VFS_ROOT, and restore it. Without this change, the next time around, the missing refcount led to a second release of the root vnode when its refcount was already zero, and thus the panic occurred while trying to grab the vnode's interlock, which had been destroyed when the vnode's refcount dropped to zero. While I'm here, refactor the unmount failure code to ensure that the zfs ctldir (i.e. .zfs/.snapshots) is always restored, along with, if needed, zfsvfs->z_unmounted = B_FALSE. In short, if the unmount fails, restore all original functionality prior to unmounting. This particular panic manifests easily if you have active NFS I/O on a ZFS while trying to do a non-forced unmount. We've also seen it happen on shutdown occasionally. I would like to commit this or a version of it, to FreeBSD/head next week. Thanks, --Will. From owner-zfs-devel@FreeBSD.ORG Wed Mar 13 21:57:14 2013 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 0181CBDA; Wed, 13 Mar 2013 21:57:14 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 24416F68; Wed, 13 Mar 2013 21:57:12 +0000 (UTC) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id XAA04823; Wed, 13 Mar 2013 23:57:09 +0200 (EET) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1UFtfZ-000Faf-Fx; Wed, 13 Mar 2013 23:57:09 +0200 Message-ID: <5140F632.7070305@FreeBSD.org> Date: Wed, 13 Mar 2013 23:57:06 +0200 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:17.0) Gecko/20130220 Thunderbird/17.0.3 MIME-Version: 1.0 To: Will Andrews Subject: Re: CFR: Fix panic in zfs_umount() due to dropped root vnode refcount References: In-Reply-To: X-Enigmail-Version: 1.5.1 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: zfs-devel@FreeBSD.org X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Mar 2013 21:57:14 -0000 on 13/03/2013 23:21 Will Andrews said the following: > Hi, > > Diff: > http://people.freebsd.org/~will/zfs/zfs_vfsops.c-fix-unmount-panic.diff > > Change 660985 by willa@willa_SpectraBSD on 2013/03/08 17:43:49 > > Fix the zfs unmount panic that is due to a missed refcount drop. > > When checking for activity in a non-forced unmount, remember that > we called vflush() releasing the refcount that zfs_mount() placed > on the filesystem's root vnode using VFS_ROOT, and restore it. > > Without this change, the next time around, the missing refcount > led to a second release of the root vnode when its refcount was > already zero, and thus the panic occurred while trying to grab > the vnode's interlock, which had been destroyed when the vnode's > refcount dropped to zero. > > While I'm here, refactor the unmount failure code to ensure that > the zfs ctldir (i.e. .zfs/.snapshots) is always restored, along > with, > if needed, zfsvfs->z_unmounted = B_FALSE. In short, if the unmount > fails, restore all original functionality prior to unmounting. > > This particular panic manifests easily if you have active NFS I/O on a ZFS > while trying to do a non-forced unmount. We've also seen it happen on > shutdown occasionally. > > I would like to commit this or a version of it, to FreeBSD/head next week. I propose to put the whole troublesome if (!(fflag & MS_FORCE)) {... } block under #ifdef sun -- that's what I have in my tree. Solaris uses its own mechanism to decide whether unmounting is allowed, but FreeBSD has its own -- vflush success is sufficient. References: http://code.metager.de/source/xref/freebsd/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vfsops.c#1942 http://code.metager.de/source/search?q=&defs=&refs=mnt_ref&path=%2Ffreebsd%2F&hist=&project=freebsd http://src.illumos.org/source/xref/illumos-gate/usr/src/uts/common/fs/zfs/zfs_vfsops.c#1866 Thank you for catching this edge case. -- Andriy Gapon From owner-zfs-devel@FreeBSD.ORG Sat Mar 23 16:21:15 2013 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 17E4F858 for ; Sat, 23 Mar 2013 16:21:15 +0000 (UTC) (envelope-from will@firepipe.net) Received: from mail-ia0-x231.google.com (mail-ia0-x231.google.com [IPv6:2607:f8b0:4001:c02::231]) by mx1.freebsd.org (Postfix) with ESMTP id E1697BFC for ; Sat, 23 Mar 2013 16:21:14 +0000 (UTC) Received: by mail-ia0-f177.google.com with SMTP id w33so1316002iag.36 for ; Sat, 23 Mar 2013 09:21:14 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:x-received:in-reply-to:references:date:message-id :subject:from:to:cc:content-type:x-gm-message-state; bh=5nXkbnfJyFE3wFur/XjT7bk788ueHbMlRGl6eoUc3fk=; b=ReaY0wDB7cY2VprHncRbfPZnVC7NGMDobfeUYg6KBBVMWUXD1gbRDaBh142unz8mpO h2FbFxbk6ZqSmspWaatQ2FZxndGU21GSRpkigPKiN3IdKycPDOBAQiHE3cK53mxKdyZI 2P62kRc2ybrYPaGF/yOVuDDdthMHur+FJhHRymzZGG+GL+ECWIlkxFJtSL54+P0I4Wbt XSsKov6by8DvwYQSTbkDdEmYtVSf3JR7aoRmotSJhAsXHS00QQ7oxrG0amyugpBcEcgF msAQLqP8HwRccalWlPZ+2Y65V5jAcDbzfrP7+6qjktydXWNTC069T0fGmDmAbDYJaeim owfw== MIME-Version: 1.0 X-Received: by 10.50.194.229 with SMTP id hz5mr3831424igc.111.1364055674522; Sat, 23 Mar 2013 09:21:14 -0700 (PDT) Received: by 10.231.247.74 with HTTP; Sat, 23 Mar 2013 09:21:14 -0700 (PDT) In-Reply-To: <5140F632.7070305@FreeBSD.org> References: <5140F632.7070305@FreeBSD.org> Date: Sat, 23 Mar 2013 10:21:14 -0600 Message-ID: Subject: Re: CFR: Fix panic in zfs_umount() due to dropped root vnode refcount From: Will Andrews To: Andriy Gapon X-Gm-Message-State: ALoCoQnEnDI6MKtpGHVlfoGjDMa4dqP8nq7fQQo8R75CpB4OT6InsPLTjv0XPlAZB1ESeHQrg75I Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: zfs-devel@freebsd.org X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 23 Mar 2013 16:21:15 -0000 Sounds good to me. I tried your approach, and it also fixes the problem. If you'd like, I'll commit it for you. Thanks, --Will. On Wed, Mar 13, 2013 at 3:57 PM, Andriy Gapon wrote: > on 13/03/2013 23:21 Will Andrews said the following: > > Hi, > > > > Diff: > > http://people.freebsd.org/~will/zfs/zfs_vfsops.c-fix-unmount-panic.diff > > > > Change 660985 by willa@willa_SpectraBSD on 2013/03/08 17:43:49 > > > > Fix the zfs unmount panic that is due to a missed refcount drop. > > > > When checking for activity in a non-forced unmount, remember that > > we called vflush() releasing the refcount that zfs_mount() placed > > on the filesystem's root vnode using VFS_ROOT, and restore it. > > > > Without this change, the next time around, the missing refcount > > led to a second release of the root vnode when its refcount was > > already zero, and thus the panic occurred while trying to grab > > the vnode's interlock, which had been destroyed when the vnode's > > refcount dropped to zero. > > > > While I'm here, refactor the unmount failure code to ensure that > > the zfs ctldir (i.e. .zfs/.snapshots) is always restored, along > > with, > > if needed, zfsvfs->z_unmounted = B_FALSE. In short, if the > unmount > > fails, restore all original functionality prior to unmounting. > > > > This particular panic manifests easily if you have active NFS I/O on a > ZFS > > while trying to do a non-forced unmount. We've also seen it happen on > > shutdown occasionally. > > > > I would like to commit this or a version of it, to FreeBSD/head next > week. > > I propose to put the whole troublesome if (!(fflag & MS_FORCE)) {... } > block > under #ifdef sun -- that's what I have in my tree. > Solaris uses its own mechanism to decide whether unmounting is allowed, but > FreeBSD has its own -- vflush success is sufficient. > > References: > > http://code.metager.de/source/xref/freebsd/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vfsops.c#1942 > > http://code.metager.de/source/search?q=&defs=&refs=mnt_ref&path=%2Ffreebsd%2F&hist=&project=freebsd > > http://src.illumos.org/source/xref/illumos-gate/usr/src/uts/common/fs/zfs/zfs_vfsops.c#1866 > > Thank you for catching this edge case. > -- > Andriy Gapon > From owner-zfs-devel@FreeBSD.ORG Tue Apr 2 22:10:28 2013 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id E054DF98; Tue, 2 Apr 2013 22:10:28 +0000 (UTC) (envelope-from delphij@delphij.net) Received: from anubis.delphij.net (anubis.delphij.net [IPv6:2001:470:1:117::25]) by mx1.freebsd.org (Postfix) with ESMTP id C66C1E37; Tue, 2 Apr 2013 22:10:28 +0000 (UTC) Received: from zeta.ixsystems.com (drawbridge.ixsystems.com [206.40.55.65]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by anubis.delphij.net (Postfix) with ESMTPSA id 1C8AF6BE3; Tue, 2 Apr 2013 15:10:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=delphij.net; s=anubis; t=1364940628; bh=BoQQMnTUua06xNz/4tuC4hFvWZV8DSMezg19afhf8BE=; h=Date:From:Reply-To:To:Subject; b=meqjofCtFZdt/y5tVUYYEQLVxdvuq7u/17uaMlZg+pFKHbkpVQ9s7WGQYqqmybsi8 3UPiGIcoQDQGiwHvr+Zt//TBFgH527ugwijOGzsQu5RTCdPZeaCR0ARA/wYOzm3+QI nZLEUu0FG45PERfbW6U777Z2qMZjU8P91Dt00tds= Message-ID: <515B5754.3020104@delphij.net> Date: Tue, 02 Apr 2013 15:10:28 -0700 From: Xin Li Organization: The FreeBSD Project MIME-Version: 1.0 To: zfs-devel@freebsd.org Subject: [PATCH FOR REVIEW] (For stable/8 only) change default zpool creation version to 28 X-Enigmail-Version: 1.5.1 Content-Type: multipart/mixed; boundary="------------060505050309030807010401" X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list Reply-To: d@delphij.net List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Apr 2013 22:10:28 -0000 This is a multi-part message in MIME format. --------------060505050309030807010401 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 Hi, The release engineer team have some concerns of having zpool default creation version leave at 5000 (feature flags) because they are not supported by "higher" versions of FreeBSD, specifically, 9.0-RELEASE (EoL'ed last month) and 9.1-RELEASE. A user may be surprised if he or she "upgrades" a FreeBSD 8.4-RELEASE system to a 9.1-RELEASE system. In order to solve this, there is a proposal that changes the default creation version to 28 instead, as it's supported universally. The attached patch implements this idea. If there would be no objections in the next 24 hours, I will commit this change directly to stable/8 and take care for the releng/8.4 merge should I get approval for that. If you have better proposal, please do speak up and just go ahead with your version. Thanks in advance! Cheers, - -- Xin LI https://www.delphij.net/ FreeBSD - The Power to Serve! Live free or die -----BEGIN PGP SIGNATURE----- iQEcBAEBCgAGBQJRW1dUAAoJEG80Jeu8UPuzGCMH/jFEX86BR3ozZDS5TPs5XSJI MvjEf3wLllaA+HG2NtVCbKVhMxZvnt4RyQSCWlNHietefiG2uizHu/wf7PXkgTtQ Z1yqElo2pXbLVY1jMkp91yVNAsE1hwoyUASc+V3LgZijXTvw10AWqpQI54yOyNfj nLn7ic8Tnwa1a05JPoAabsCLAQ9YNStaEt6bo6180gXiD7gBHLibTaD0OuRIVgej wOnSfI1o1XK2veYo04PVZLdGEEHeK+mbF1EPna7K6K6EwZ7vfsk6NGp9882lpoKj LsW8ctnANUNv8hjfwOCUDQUW1NpjUDzkNvnt4Nfkh9TFfBsZjvbfxfEgS6W62Qk= =UGCv -----END PGP SIGNATURE----- --------------060505050309030807010401 Content-Type: text/plain; charset=UTF-8; name="zpool.diff" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="zpool.diff" Index: cddl/contrib/opensolaris/cmd/zpool/zpool_main.c =================================================================== --- cddl/contrib/opensolaris/cmd/zpool/zpool_main.c (revision 249032) +++ cddl/contrib/opensolaris/cmd/zpool/zpool_main.c (working copy) @@ -856,6 +856,17 @@ zpool_do_create(int argc, char **argv) } } + /* Compatiblity with 9.0 and 9.1: Use version 28 if unspecified */ + if (nvlist_lookup_string(props, + zpool_prop_to_name(ZPOOL_PROP_VERSION), + &propval) != 0) { + if (add_prop_list(zpool_prop_to_name( + ZPOOL_PROP_VERSION), "28", &props, B_TRUE)) + goto errout; + enable_all_pool_feat = B_FALSE; + } else if (enable_all_pool_feat) + nvlist_remove_all(props, zpool_prop_to_name(ZPOOL_PROP_VERSION)); + argc -= optind; argv += optind; --------------060505050309030807010401-- From owner-zfs-devel@FreeBSD.ORG Wed Apr 3 05:51:22 2013 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id EB52BA2E for ; Wed, 3 Apr 2013 05:51:22 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 29E6B62E for ; Wed, 3 Apr 2013 05:51:21 +0000 (UTC) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id IAA18241; Wed, 03 Apr 2013 08:51:12 +0300 (EEST) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1UNGbH-000FEA-PB; Wed, 03 Apr 2013 08:51:11 +0300 Message-ID: <515BC34E.40309@FreeBSD.org> Date: Wed, 03 Apr 2013 08:51:10 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:17.0) Gecko/20130321 Thunderbird/17.0.4 MIME-Version: 1.0 To: d@delphij.net Subject: Re: [PATCH FOR REVIEW] (For stable/8 only) change default zpool creation version to 28 References: <515B5754.3020104@delphij.net> In-Reply-To: <515B5754.3020104@delphij.net> X-Enigmail-Version: 1.5.1 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: zfs-devel@FreeBSD.org X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Apr 2013 05:51:23 -0000 on 03/04/2013 01:10 Xin Li said the following: > If you have better proposal, > please do speak up and just go ahead with your version. To RE team: please release 9.2. -- Andriy Gapon From owner-zfs-devel@FreeBSD.ORG Wed Apr 3 16:22:16 2013 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 70C62D72; Wed, 3 Apr 2013 16:22:16 +0000 (UTC) (envelope-from delphij@delphij.net) Received: from anubis.delphij.net (anubis.delphij.net [64.62.153.212]) by mx1.freebsd.org (Postfix) with ESMTP id 5C9A5E1F; Wed, 3 Apr 2013 16:22:16 +0000 (UTC) Received: from delphij-macbook.local (unknown [IPv6:2001:470:83bf:0:580d:5ee8:5035:2a67]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by anubis.delphij.net (Postfix) with ESMTPSA id AE21C636D; Wed, 3 Apr 2013 09:22:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=delphij.net; s=anubis; t=1365006130; bh=DOVwWGNTZVr2HrRV4Ov1HxP/OoohFGc0YD8CNTbw9IY=; h=Date:From:Reply-To:To:CC:Subject:References:In-Reply-To; b=KhAx/i8RHTrabNmerdWw1w4tOEyIg6mX3Yg0CBDIXL0NEf+idDZjg2ApvIVOafJ9U 00/IBiWRi0kX8Q5h379mS0Vq1GlWqWcne/fxmLzhNOsEu8RvCh9GFBliMd9lDmyqCQ wPfcudspTjQIUE5PtJsBBy4Ap1gGtz/e/0ysd34U= Message-ID: <515C5734.8090304@delphij.net> Date: Wed, 03 Apr 2013 09:22:12 -0700 From: Xin Li Organization: The FreeBSD Project MIME-Version: 1.0 To: Andriy Gapon Subject: Re: [PATCH FOR REVIEW] (For stable/8 only) change default zpool creation version to 28 References: <515B5754.3020104@delphij.net> <515BC34E.40309@FreeBSD.org> In-Reply-To: <515BC34E.40309@FreeBSD.org> X-Enigmail-Version: 1.5.1 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: zfs-devel@FreeBSD.org, d@delphij.net X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list Reply-To: d@delphij.net List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Apr 2013 16:22:16 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 On 4/2/13 10:51 PM, Andriy Gapon wrote: > on 03/04/2013 01:10 Xin Li said the following: >> If you have better proposal, please do speak up and just go ahead >> with your version. > > To RE team: please release 9.2. Even if they do that anytime soon, it's already too late to release it before 8.4... Cheers, -----BEGIN PGP SIGNATURE----- iQEcBAEBCAAGBQJRXFczAAoJEG80Jeu8UPuzXu8H/2wE9dqVcg0l5COF01mpqH8H UXQJXk9mOOjn1IxcgZKdqGDlkktZLzoJjVzQm9K0T0u/6pVBoottrp8hdFJ07Fyz VFg2jjNS/RcR1XTCEn3uurdFoQDNmoVdSoC6euYkepjS5samQSYfu3kumjPIHAds CsdYT7+AVOL8ULDsMViRF4PoY+fxAeETehygxC0PkytpantXI+nB3vXRQcTfXiB6 OlnGQIewKJ4i5Cg1KKfl5QGVxyGXRcvltWqOcDuRfy6ibCVo+pvQfEU50xij1NM8 hbjepgzpevLj90rZuHmJ6fhkHZYKkqJKLU3owxbBNZDC6q/KCPXE+n1XfJ8vN14= =xhKK -----END PGP SIGNATURE----- From owner-zfs-devel@FreeBSD.ORG Wed Apr 3 21:04:44 2013 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 3A356278 for ; Wed, 3 Apr 2013 21:04:44 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from mail.vx.sk (mail.vx.sk [176.9.45.25]) by mx1.freebsd.org (Postfix) with ESMTP id BB654EDB for ; Wed, 3 Apr 2013 21:04:43 +0000 (UTC) Received: from core.vx.sk (localhost [127.0.0.2]) by mail.vx.sk (Postfix) with ESMTP id 819643596D; Wed, 3 Apr 2013 23:04:42 +0200 (CEST) X-Virus-Scanned: amavisd-new at mail.vx.sk Received: from mail.vx.sk by core.vx.sk (amavisd-new, unix socket) with LMTP id 0UC14wf4tWRq; Wed, 3 Apr 2013 23:04:37 +0200 (CEST) Received: from [10.9.8.1] (chello085216226145.chello.sk [85.216.226.145]) by mail.vx.sk (Postfix) with ESMTPSA id 3BF073595F; Wed, 3 Apr 2013 23:04:37 +0200 (CEST) Message-ID: <515C9966.6020003@FreeBSD.org> Date: Wed, 03 Apr 2013 23:04:38 +0200 From: Martin Matuska User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:17.0) Gecko/20130328 Thunderbird/17.0.5 MIME-Version: 1.0 To: d@delphij.net Subject: Re: [PATCH FOR REVIEW] (For stable/8 only) change default zpool creation version to 28 References: <515B5754.3020104@delphij.net> In-Reply-To: <515B5754.3020104@delphij.net> X-Enigmail-Version: 1.5.1 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: zfs-devel@freebsd.org X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Apr 2013 21:04:44 -0000 Looks ok to me. On 3.4.2013 0:10, Xin Li wrote: > Hi, > > The release engineer team have some concerns of having zpool default > creation version leave at 5000 (feature flags) because they are not > supported by "higher" versions of FreeBSD, specifically, 9.0-RELEASE > (EoL'ed last month) and 9.1-RELEASE. A user may be surprised if he or > she "upgrades" a FreeBSD 8.4-RELEASE system to a 9.1-RELEASE system. > > In order to solve this, there is a proposal that changes the default > creation version to 28 instead, as it's supported universally. The > attached patch implements this idea. > > If there would be no objections in the next 24 hours, I will commit > this change directly to stable/8 and take care for the releng/8.4 > merge should I get approval for that. If you have better proposal, > please do speak up and just go ahead with your version. > > Thanks in advance! > > Cheers, > > > _______________________________________________ > zfs-devel@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/zfs-devel > To unsubscribe, send any mail to "zfs-devel-unsubscribe@freebsd.org" -- Martin Matuska FreeBSD committer http://blog.vx.sk From owner-zfs-devel@FreeBSD.ORG Wed Apr 3 23:37:37 2013 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id DDBD126C for ; Wed, 3 Apr 2013 23:37:37 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: from mail-lb0-f182.google.com (mail-lb0-f182.google.com [209.85.217.182]) by mx1.freebsd.org (Postfix) with ESMTP id 63B56845 for ; Wed, 3 Apr 2013 23:37:37 +0000 (UTC) Received: by mail-lb0-f182.google.com with SMTP id z13so2163533lbh.27 for ; Wed, 03 Apr 2013 16:37:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=delphix.com; s=google; h=mime-version:x-received:in-reply-to:references:date:message-id :subject:from:to:cc:content-type; bh=VNTbpwamTX59wVajxnABoveGxi223+X8e8Nr0cnglaU=; b=FwySC9hsuPvT9Feja7AGG4MUd1hbiCni3pjH4rDj1+HN6olVU7TlD7miV5SlpHVave BlrUL7BuY+b+6xioPw+2iRGJHNjK9khGQctCXyPg6g7nXixpmPbAK77tdNl8e1cp/OAO UeAu9mN0//bnXQwCdHusRaxoAxXYPWm/AfdEI= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:x-received:in-reply-to:references:date:message-id :subject:from:to:cc:content-type:x-gm-message-state; bh=VNTbpwamTX59wVajxnABoveGxi223+X8e8Nr0cnglaU=; b=lFQp9DqCD59hyDUGqxUOucLJu0eCn73pBfq4VtqwDQxUFCSWq8kAUs/+yKi+v9Bgcs WPtKrYM+XPqR2ZIdbeeUVFFxcIJ44LhOJfnFhqMt5wG4t5aQcmfGmRLruxc36NYZoyMG lrhIz8jYJ5ul+BS/QKHNPmbdsOQsdzijKelQ76TmVkP5mwD6brT0AubrT+ilHFBkJAkY gYhEaDXLgcQn0uYUlYrST8pSHnkFVz8WK5tp/gD0KZJ87XAmVHEUh8ra9eBKMfZpJd3s Sklkrp3W8krApxi6o1jC7wnL0+uBI09AVyJypL1sisJUrQnowplCDTDfMpd+xqsXpeVA df/g== MIME-Version: 1.0 X-Received: by 10.152.29.232 with SMTP id n8mr1933072lah.55.1365032255930; Wed, 03 Apr 2013 16:37:35 -0700 (PDT) Received: by 10.114.26.202 with HTTP; Wed, 3 Apr 2013 16:37:35 -0700 (PDT) In-Reply-To: <515B5754.3020104@delphij.net> References: <515B5754.3020104@delphij.net> Date: Wed, 3 Apr 2013 16:37:35 -0700 Message-ID: Subject: Re: [PATCH FOR REVIEW] (For stable/8 only) change default zpool creation version to 28 From: Matthew Ahrens To: d@delphij.net X-Gm-Message-State: ALoCoQmIlItk2oGr45UeKYpgnnsGKl173AVaNqz8m37i0+ubHfvpDuqdvXHuLKdxPKybzkz/4FVz Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: zfs-devel@freebsd.org X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Apr 2013 23:37:37 -0000 What if they specify "-d -o feature@...=enabled"? It seems like your code will not find the "version" property set, and then try to set "version=28", which is not what the user intended. Also, I'm not sure why you'd need to remove the "version" property if you are enabling all features. They shouldn't have specified a version, and I believe that this error is checked later on. On Tue, Apr 2, 2013 at 3:10 PM, Xin Li wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA512 > > Hi, > > The release engineer team have some concerns of having zpool default > creation version leave at 5000 (feature flags) because they are not > supported by "higher" versions of FreeBSD, specifically, 9.0-RELEASE > (EoL'ed last month) and 9.1-RELEASE. A user may be surprised if he or > she "upgrades" a FreeBSD 8.4-RELEASE system to a 9.1-RELEASE system. > > In order to solve this, there is a proposal that changes the default > creation version to 28 instead, as it's supported universally. The > attached patch implements this idea. > > If there would be no objections in the next 24 hours, I will commit > this change directly to stable/8 and take care for the releng/8.4 > merge should I get approval for that. If you have better proposal, > please do speak up and just go ahead with your version. > > Thanks in advance! > > Cheers, > - -- > Xin LI https://www.delphij.net/ > FreeBSD - The Power to Serve! Live free or die > -----BEGIN PGP SIGNATURE----- > > iQEcBAEBCgAGBQJRW1dUAAoJEG80Jeu8UPuzGCMH/jFEX86BR3ozZDS5TPs5XSJI > MvjEf3wLllaA+HG2NtVCbKVhMxZvnt4RyQSCWlNHietefiG2uizHu/wf7PXkgTtQ > Z1yqElo2pXbLVY1jMkp91yVNAsE1hwoyUASc+V3LgZijXTvw10AWqpQI54yOyNfj > nLn7ic8Tnwa1a05JPoAabsCLAQ9YNStaEt6bo6180gXiD7gBHLibTaD0OuRIVgej > wOnSfI1o1XK2veYo04PVZLdGEEHeK+mbF1EPna7K6K6EwZ7vfsk6NGp9882lpoKj > LsW8ctnANUNv8hjfwOCUDQUW1NpjUDzkNvnt4Nfkh9TFfBsZjvbfxfEgS6W62Qk= > =UGCv > -----END PGP SIGNATURE----- > > _______________________________________________ > zfs-devel@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/zfs-devel > To unsubscribe, send any mail to "zfs-devel-unsubscribe@freebsd.org" > From owner-zfs-devel@FreeBSD.ORG Thu Apr 4 00:39:56 2013 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id BD4CF311 for ; Thu, 4 Apr 2013 00:39:56 +0000 (UTC) (envelope-from delphij@delphij.net) Received: from anubis.delphij.net (anubis.delphij.net [64.62.153.212]) by mx1.freebsd.org (Postfix) with ESMTP id A6902CB5 for ; Thu, 4 Apr 2013 00:39:56 +0000 (UTC) Received: from zeta.ixsystems.com (drawbridge.ixsystems.com [206.40.55.65]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by anubis.delphij.net (Postfix) with ESMTPSA id 98E733B2C; Wed, 3 Apr 2013 17:39:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=delphij.net; s=anubis; t=1365035995; bh=UChvidoXmvcGXiM5dA+rXqYjPHEG75U/QbBHeOH5pbE=; h=Date:From:Reply-To:To:CC:Subject:References:In-Reply-To; b=zv0JFQh2v3yzqaLDumjSbBbZcqkw6Jj85FNZQNnxSdFYh5JRXz95/WLLH4Ls3aUqC 0ZHIAqT+Ovjn1kxtcYv9JZwDPVS0jWkxkrw3aUTPgxE8V67H8+W9PKY//qpy+VM/46 DuLdlS75FQSeP4F9inpt/1ocvPWUUta/cLmFi7L4= Message-ID: <515CCBDC.7010304@delphij.net> Date: Wed, 03 Apr 2013 17:39:56 -0700 From: Xin Li Organization: The FreeBSD Project MIME-Version: 1.0 To: Matthew Ahrens Subject: Re: [PATCH FOR REVIEW] (For stable/8 only) change default zpool creation version to 28 References: <515B5754.3020104@delphij.net> In-Reply-To: X-Enigmail-Version: 1.5.1 Content-Type: multipart/mixed; boundary="------------010002070703090301020302" Cc: zfs-devel@freebsd.org, d@delphij.net X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list Reply-To: d@delphij.net List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Apr 2013 00:39:56 -0000 This is a multi-part message in MIME format. --------------010002070703090301020302 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 Hi, Matthew, Thanks for your comments! See my replies inline. On 04/03/13 16:37, Matthew Ahrens wrote: > What if they specify "-d -o feature@...=enabled"? It seems like > your code will not find the "version" property set, and then try to > set "version=28", which is not what the user intended. This would be caught later (feature flags can not be specified with version) but actually that's a good question for this case: > Also, I'm not sure why you'd need to remove the "version" property > if you are enabling all features. They shouldn't have specified a > version, and I believe that this error is checked later on. I think you are right that the case -d -o feature@...=enabled can not be used with version=5000 case. I haven't come to a better idea yet, the attached patch would fix that issue but is ugly. In the other thread you have mentioned an alternative idea that just have the user to explicitly do an "zpool upgrade", I'll talk with the re@ to figure out if that's acceptable. Again, thanks for your time and your comments! > On Tue, Apr 2, 2013 at 3:10 PM, Xin Li > wrote: > > Hi, > > The release engineer team have some concerns of having zpool > default creation version leave at 5000 (feature flags) because they > are not supported by "higher" versions of FreeBSD, specifically, > 9.0-RELEASE (EoL'ed last month) and 9.1-RELEASE. A user may be > surprised if he or she "upgrades" a FreeBSD 8.4-RELEASE system to a > 9.1-RELEASE system. > > In order to solve this, there is a proposal that changes the > default creation version to 28 instead, as it's supported > universally. The attached patch implements this idea. > > If there would be no objections in the next 24 hours, I will > commit this change directly to stable/8 and take care for the > releng/8.4 merge should I get approval for that. If you have > better proposal, please do speak up and just go ahead with your > version. > > Thanks in advance! > > Cheers, >> >> _______________________________________________ >> zfs-devel@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/zfs-devel To >> unsubscribe, send any mail to >> "zfs-devel-unsubscribe@freebsd.org" >> > _______________________________________________ > zfs-devel@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/zfs-devel To unsubscribe, > send any mail to "zfs-devel-unsubscribe@freebsd.org" > - -- Xin LI https://www.delphij.net/ FreeBSD - The Power to Serve! Live free or die -----BEGIN PGP SIGNATURE----- iQEcBAEBCgAGBQJRXMvcAAoJEG80Jeu8UPuzKFEIAIst/amXnTAyCoz5PFp8Y7mO mnGQF0EicY/WYrxSmPQkVM8LheqFg7efT8As7Weq6B1PQ6Fqcw7WHxdDYad21ywQ rTeB/R4l0f459jOKrgjuWiIg/t27riK/xPwwLOxpcFZ8eh86kLN6n2xxW348hZ13 ma4g1wbU4dcvhPtHS4BT6/lWSEkrfpbPDKLPjbuwPuU6NhiOW1jJIr1hc8W1z+Ep roSnN+RNKfya/Ur2MI7eN15py1BLFqzPbDrLukx2wf6i+Z9rmwd0lvMx8lZKDcEe /9dEjWueEYYCh3i7rRM6/r4PoTbVBKimpeSEbtDtDyKpHOlu7C2ZmnE+Ej/Q+58= =KOnh -----END PGP SIGNATURE----- --------------010002070703090301020302 Content-Type: text/plain; charset=UTF-8; name="zpool.1.diff" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="zpool.1.diff" Index: cddl/contrib/opensolaris/cmd/zpool/zpool_main.c =================================================================== --- cddl/contrib/opensolaris/cmd/zpool/zpool_main.c (revision 249071) +++ cddl/contrib/opensolaris/cmd/zpool/zpool_main.c (working copy) @@ -768,6 +768,7 @@ boolean_t force = B_FALSE; boolean_t dryrun = B_FALSE; boolean_t enable_all_pool_feat = B_TRUE; + boolean_t spa_want_features = B_FALSE; int c; nvlist_t *nvroot = NULL; char *poolname; @@ -815,9 +816,6 @@ *propval = '\0'; propval++; - if (add_prop_list(optarg, propval, &props, B_TRUE)) - goto errout; - /* * If the user is creating a pool that doesn't support * feature flags, don't enable any features. @@ -830,8 +828,16 @@ if (*end == '\0' && ver < SPA_VERSION_FEATURES) { enable_all_pool_feat = B_FALSE; + } else if (*end == '\0' && + ver == SPA_VERSION_FEATURES) { + spa_want_features = B_TRUE; + break; } } + + if (add_prop_list(optarg, propval, &props, B_TRUE)) + goto errout; + break; case 'O': if ((propval = strchr(optarg, '=')) == NULL) { @@ -856,6 +862,19 @@ } } +#ifdef __FreeBSD__ + /* Compatiblity with FreeBSD 9.0 and 9.1: Use version 28 if unspecified */ + if (nvlist_lookup_string(props, + zpool_prop_to_name(ZPOOL_PROP_VERSION), + &propval) != 0 && !spa_want_features && + !prop_list_contains_feature(props)) { + if (add_prop_list(zpool_prop_to_name( + ZPOOL_PROP_VERSION), "28", &props, B_TRUE)) + goto errout; + enable_all_pool_feat = B_FALSE; + } +#endif /* __FreeBSD__ */ + argc -= optind; argv += optind; --------------010002070703090301020302-- From owner-zfs-devel@FreeBSD.ORG Sun Apr 7 19:29:19 2013 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id EB533F0D for ; Sun, 7 Apr 2013 19:29:19 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from mail.vx.sk (mail.vx.sk [176.9.45.25]) by mx1.freebsd.org (Postfix) with ESMTP id 4DAC3C14 for ; Sun, 7 Apr 2013 19:29:19 +0000 (UTC) Received: from core.vx.sk (localhost [127.0.0.2]) by mail.vx.sk (Postfix) with ESMTP id 3814F358C5 for ; Sun, 7 Apr 2013 21:29:12 +0200 (CEST) X-Virus-Scanned: amavisd-new at mail.vx.sk Received: from mail.vx.sk by core.vx.sk (amavisd-new, unix socket) with LMTP id OhExrX_k2L3P for ; Sun, 7 Apr 2013 21:29:08 +0200 (CEST) Received: from [10.9.8.1] (chello085216226145.chello.sk [85.216.226.145]) by mail.vx.sk (Postfix) with ESMTPSA id 99504358BA for ; Sun, 7 Apr 2013 21:29:08 +0200 (CEST) Message-ID: <5161C904.8060600@FreeBSD.org> Date: Sun, 07 Apr 2013 21:29:08 +0200 From: Martin Matuska User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:17.0) Gecko/20130328 Thunderbird/17.0.5 MIME-Version: 1.0 To: zfs-devel@FreeBSD.org Subject: [REVIEW] patch for copyout on ioctl error X-Enigmail-Version: 1.5.1 Content-Type: multipart/mixed; boundary="------------040107010104050106040805" X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 07 Apr 2013 19:29:20 -0000 This is a multi-part message in MIME format. --------------040107010104050106040805 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Hi all, ZFS expects a copyout of zfs_cmd_t on a ioctl error. Our sys_ioctl() doesn't copyout in this case. I am suggesting the attached patch for review to fix (or workaround) this situation. The idea is easy - zfs ioctl's won't send zfs_cmd_t anymore but a new struct zfs_iocparm_t consisting of just three elements: pointer to zfs_cmd_t size of zfs_cmd_t zfs_ioctl_version Size of zfs_cmd_t is intended for the kernel to re-verify if talking to the correct version. zfs_ioctl_version is for future purposes to make it easier for the kernel to identify ioctl changes. The kernel module will do the copyin/copyout of zfs_cmd_t itself (like illumos does). This makes future compatibility more simple and zfsdev_ioctl() behaves much more like in illumos. The patch retains compatibility with all previous states. Cheers, mm -- Martin Matuska FreeBSD committer http://blog.vx.sk --------------040107010104050106040805 Content-Type: text/plain; charset=windows-1250; name="zfs_ioctl_copyout.patch" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="zfs_ioctl_copyout.patch" SW5kZXg6IGNkZGwvY29udHJpYi9vcGVuc29sYXJpcy9saWIvbGliemZzL2NvbW1vbi9saWJ6 ZnNfY29tcGF0LmMKPT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PQotLS0gY2RkbC9jb250cmliL29wZW5zb2xhcmlz L2xpYi9saWJ6ZnMvY29tbW9uL2xpYnpmc19jb21wYXQuYwkocmV2aXNpb24gMjQ5MjI2KQor KysgY2RkbC9jb250cmliL29wZW5zb2xhcmlzL2xpYi9saWJ6ZnMvY29tbW9uL2xpYnpmc19j b21wYXQuYwkod29ya2luZyBjb3B5KQpAQCAtNzIsNyArNzIsOSBAQCB6Y21kX2lvY3RsKGlu dCBmZCwgaW50IHJlcXVlc3QsIHpmc19jbWRfdCAqemMpCiAJaWYgKHpmc19pb2N0bF92ZXJz aW9uID09IFpGU19JT0NWRVJfVU5ERUYpCiAJCXpmc19pb2N0bF92ZXJzaW9uID0gZ2V0X3pm c19pb2N0bF92ZXJzaW9uKCk7CiAKLQlpZiAoemZzX2lvY3RsX3ZlcnNpb24gPT0gWkZTX0lP Q1ZFUl9ERUFETUFOKQorCWlmICh6ZnNfaW9jdGxfdmVyc2lvbiA9PSBaRlNfSU9DVkVSX0xa QykKKwkJY2ZsYWcgPSBaRlNfQ01EX0NPTVBBVF9MWkM7CisJZWxzZSBpZiAoemZzX2lvY3Rs X3ZlcnNpb24gPT0gWkZTX0lPQ1ZFUl9ERUFETUFOKQogCQljZmxhZyA9IFpGU19DTURfQ09N UEFUX0RFQURNQU47CiAKIAkvKgpJbmRleDogc3lzL2NkZGwvY29udHJpYi9vcGVuc29sYXJp cy9jb21tb24vemZzL3pmc19pb2N0bF9jb21wYXQuYwo9PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09Ci0tLSBzeXMv Y2RkbC9jb250cmliL29wZW5zb2xhcmlzL2NvbW1vbi96ZnMvemZzX2lvY3RsX2NvbXBhdC5j CShyZXZpc2lvbiAyNDkyMjYpCisrKyBzeXMvY2RkbC9jb250cmliL29wZW5zb2xhcmlzL2Nv bW1vbi96ZnMvemZzX2lvY3RsX2NvbXBhdC5jCSh3b3JraW5nIGNvcHkpCkBAIC01NzUsMTEg KzU3NSwyMSBAQCBpbnQKIHpjbWRfaW9jdGxfY29tcGF0KGludCBmZCwgaW50IHJlcXVlc3Qs IHpmc19jbWRfdCAqemMsIGNvbnN0IGludCBjZmxhZykKIHsKIAlpbnQgbmMsIHJldDsKKwl6 ZnNfaW9jcGFybV90ICp6ZnNfaW9jcGFybTsKIAl2b2lkICp6Y19jOwogCXVuc2lnbmVkIGxv bmcgbmNtZDsKIAogCXN3aXRjaCAoY2ZsYWcpIHsKIAljYXNlIFpGU19DTURfQ09NUEFUX05P TkU6CisJCW5jbWQgPSBfSU9XUignWicsIHJlcXVlc3QsIHN0cnVjdCB6ZnNfaW9jcGFybSk7 CisJCXpmc19pb2NwYXJtID0gbWFsbG9jKHNpemVvZih6ZnNfaW9jcGFybV90KSk7CisJCXpm c19pb2NwYXJtLT56ZnNfY21kID0gemM7CisJCXpmc19pb2NwYXJtLT56ZnNfY21kX3NpemUg PSBzaXplb2YoemZzX2NtZF90KTsKKwkJemZzX2lvY3Bhcm0tPnpmc19pb2N0bF92ZXJzaW9u ID0gWkZTX0lPQ1ZFUl9DVVJSRU5UOworCQlyZXQgPSBpb2N0bChmZCwgbmNtZCwgemZzX2lv Y3Bhcm0pOworCQlmcmVlKHpmc19pb2NwYXJtKTsKKwkJcmV0dXJuIChyZXQpOworCWNhc2Ug WkZTX0NNRF9DT01QQVRfTFpDOgogCQluY21kID0gX0lPV1IoJ1onLCByZXF1ZXN0LCBzdHJ1 Y3QgemZzX2NtZCk7CiAJCXJldCA9IGlvY3RsKGZkLCBuY21kLCB6Yyk7CiAJCXJldHVybiAo cmV0KTsKQEAgLTY3Nyw3ICs2ODcsNyBAQCB6ZnNfaW9jdGxfY29tcGF0X2lubnZsKHpmc19j bWRfdCAqemMsIG52bGlzdF90ICogaQogCWNoYXIgKnBvb2xuYW1lLCAqc25hcG5hbWU7CiAJ aW50IGVycjsKIAotCWlmIChjZmxhZyA9PSBaRlNfQ01EX0NPTVBBVF9OT05FKQorCWlmIChj ZmxhZyA9PSBaRlNfQ01EX0NPTVBBVF9OT05FIHx8IGNmbGFnID09IFpGU19DTURfQ09NUEFU X0xaQykKIAkJZ290byBvdXQ7CiAKIAlzd2l0Y2ggKHZlYykgewpAQCAtODI4LDcgKzgzOCw3 IEBAIHpmc19pb2N0bF9jb21wYXRfb3V0bnZsKHpmc19jbWRfdCAqemMsIG52bGlzdF90ICoK IHsKIAludmxpc3RfdCAqdG1wbnZsOwogCi0JaWYgKGNmbGFnID09IFpGU19DTURfQ09NUEFU X05PTkUpCisJaWYgKGNmbGFnID09IFpGU19DTURfQ09NUEFUX05PTkUgfHwgY2ZsYWcgPT0g WkZTX0NNRF9DT01QQVRfTFpDKQogCQlyZXR1cm4gKG91dG52bCk7CiAKIAlzd2l0Y2ggKHZl YykgewpJbmRleDogc3lzL2NkZGwvY29udHJpYi9vcGVuc29sYXJpcy9jb21tb24vemZzL3pm c19pb2N0bF9jb21wYXQuaAo9PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09Ci0tLSBzeXMvY2RkbC9jb250cmliL29w ZW5zb2xhcmlzL2NvbW1vbi96ZnMvemZzX2lvY3RsX2NvbXBhdC5oCShyZXZpc2lvbiAyNDky MjYpCisrKyBzeXMvY2RkbC9jb250cmliL29wZW5zb2xhcmlzL2NvbW1vbi96ZnMvemZzX2lv Y3RsX2NvbXBhdC5oCSh3b3JraW5nIGNvcHkpCkBAIC00OSwxOSArNDksMjcgQEAgZXh0ZXJu ICJDIiB7CiAjZGVmaW5lCVpGU19JT0NWRVJfTk9ORQkJMAogI2RlZmluZQlaRlNfSU9DVkVS X0RFQURNQU4JMQogI2RlZmluZQlaRlNfSU9DVkVSX0xaQwkJMgotI2RlZmluZQlaRlNfSU9D VkVSX0NVUlJFTlQJWkZTX0lPQ1ZFUl9MWkMKKyNkZWZpbmUJWkZTX0lPQ1ZFUl9aQ01ECQkz CisjZGVmaW5lCVpGU19JT0NWRVJfQ1VSUkVOVAlaRlNfSU9DVkVSX1pDTUQKIAogLyogY29t cGF0aWJpbGl0eSBjb252ZXJzaW9uIGZsYWcgKi8KICNkZWZpbmUJWkZTX0NNRF9DT01QQVRf Tk9ORQkwCiAjZGVmaW5lCVpGU19DTURfQ09NUEFUX1YxNQkxCiAjZGVmaW5lCVpGU19DTURf Q09NUEFUX1YyOAkyCiAjZGVmaW5lCVpGU19DTURfQ09NUEFUX0RFQURNQU4JMworI2RlZmlu ZQlaRlNfQ01EX0NPTVBBVF9MWkMJNAogCiAjZGVmaW5lCVpGU19JT0NfQ09NUEFUX1BBU1MJ MjU0CiAjZGVmaW5lCVpGU19JT0NfQ09NUEFUX0ZBSUwJMjU1CiAKICNkZWZpbmUJWkZTX0lP Q1JFUShpb3JlcSkJKChpb3JlcSkgJiAweGZmKQogCit0eXBlZGVmIHN0cnVjdCB6ZnNfaW9j cGFybSB7CisJdWludDY0X3QJemZzX2NtZDsKKwl1aW50NjRfdAl6ZnNfY21kX3NpemU7CisJ dWludDE2X3QJemZzX2lvY3RsX3ZlcnNpb247Cit9IHpmc19pb2NwYXJtX3Q7CisKIHR5cGVk ZWYgc3RydWN0IHppbmplY3RfcmVjb3JkX3YxNSB7CiAJdWludDY0X3QJemlfb2Jqc2V0Owog CXVpbnQ2NF90CXppX29iamVjdDsKSW5kZXg6IHN5cy9jZGRsL2NvbnRyaWIvb3BlbnNvbGFy aXMvdXRzL2NvbW1vbi9mcy96ZnMvemZzX2lvY3RsLmMKPT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PQotLS0gc3lz L2NkZGwvY29udHJpYi9vcGVuc29sYXJpcy91dHMvY29tbW9uL2ZzL3pmcy96ZnNfaW9jdGwu YwkocmV2aXNpb24gMjQ5MjI2KQorKysgc3lzL2NkZGwvY29udHJpYi9vcGVuc29sYXJpcy91 dHMvY29tbW9uL2ZzL3pmcy96ZnNfaW9jdGwuYwkod29ya2luZyBjb3B5KQpAQCAtNTcxMywx MSArNTcxMywxMyBAQCB6ZnNkZXZfaW9jdGwoc3RydWN0IGNkZXYgKmRldiwgdV9sb25nIHpj bWQsIGNhZGRyXwogewogCXpmc19jbWRfdCAqemM7CiAJdWludF90IHZlY251bTsKKwlpbnQg ZXJyb3IsIHJjLCBsZW47CiAjaWZkZWYgaWxsdW1vcwotCWludCBlcnJvciwgcmMsIGxlbjsK IAltaW5vcl90IG1pbm9yID0gZ2V0bWlub3IoZGV2KTsKICNlbHNlCi0JaW50IGVycm9yLCBs ZW4sIGNmbGFnLCBjbWQsIG9sZHZlY251bTsKKwl6ZnNfaW9jcGFybV90ICp6Y19pb2NwYXJt OworCWludCBjZmxhZywgY21kLCBvbGR2ZWNudW07CisJYm9vbGVhbl90IG5ld2lvYywgY29t cGF0OwogCWNyZWRfdCAqY3IgPSB0ZC0+dGRfdWNyZWQ7CiAjZW5kaWYKIAljb25zdCB6ZnNf aW9jX3ZlY190ICp2ZWM7CkBAIC01NzI1LDYgKzU3MjcsOSBAQCB6ZnNkZXZfaW9jdGwoc3Ry dWN0IGNkZXYgKmRldiwgdV9sb25nIHpjbWQsIGNhZGRyXwogCW52bGlzdF90ICppbm52bCA9 IE5VTEw7CiAKIAljZmxhZyA9IFpGU19DTURfQ09NUEFUX05PTkU7CisJY29tcGF0ID0gQl9G QUxTRTsKKwluZXdpb2MgPSBCX1RSVUU7CisKIAlsZW4gPSBJT0NQQVJNX0xFTih6Y21kKTsK IAljbWQgPSB6Y21kICYgMHhmZjsKIApAQCAtNTczMiwxOSArNTczNywyNiBAQCB6ZnNkZXZf aW9jdGwoc3RydWN0IGNkZXYgKmRldiwgdV9sb25nIHpjbWQsIGNhZGRyXwogCSAqIENoZWNr IGlmIHdlIGFyZSB0YWxraW5nIHRvIHN1cHBvcnRlZCBvbGRlciBiaW5hcmllcwogCSAqIGFu ZCB0cmFuc2xhdGUgemZzX2NtZCBpZiBuZWNlc3NhcnkKIAkgKi8KLQlpZiAobGVuICE9IHNp emVvZih6ZnNfY21kX3QpKQotCQlpZiAobGVuID09IHNpemVvZih6ZnNfY21kX2RlYWRtYW5f dCkpIHsKKwlpZiAobGVuICE9IHNpemVvZih6ZnNfaW9jcGFybV90KSkgeworCQluZXdpb2Mg PSBCX0ZBTFNFOworCQlpZiAobGVuID09IHNpemVvZih6ZnNfY21kX3QpKSB7CisJCQljZmxh ZyA9IFpGU19DTURfQ09NUEFUX0xaQzsKKwkJCXZlY251bSA9IGNtZDsKKwkJfSBlbHNlIGlm IChsZW4gPT0gc2l6ZW9mKHpmc19jbWRfZGVhZG1hbl90KSkgewogCQkJY2ZsYWcgPSBaRlNf Q01EX0NPTVBBVF9ERUFETUFOOworCQkJY29tcGF0ID0gQl9UUlVFOwogCQkJdmVjbnVtID0g Y21kOwogCQl9IGVsc2UgaWYgKGxlbiA9PSBzaXplb2YoemZzX2NtZF92MjhfdCkpIHsKIAkJ CWNmbGFnID0gWkZTX0NNRF9DT01QQVRfVjI4OworCQkJY29tcGF0ID0gQl9UUlVFOwogCQkJ dmVjbnVtID0gY21kOwogCQl9IGVsc2UgaWYgKGxlbiA9PSBzaXplb2YoemZzX2NtZF92MTVf dCkpIHsKIAkJCWNmbGFnID0gWkZTX0NNRF9DT01QQVRfVjE1OworCQkJY29tcGF0ID0gQl9U UlVFOwogCQkJdmVjbnVtID0gemZzX2lvY3RsX3YxNV90b192MjhbY21kXTsKIAkJfSBlbHNl CiAJCQlyZXR1cm4gKEVJTlZBTCk7Ci0JZWxzZQorCX0gZWxzZQogCQl2ZWNudW0gPSBjbWQ7 CiAKICNpZmRlZiBpbGx1bW9zCkBAIC01NzUyLDcgKzU3NjQsNyBAQCB6ZnNkZXZfaW9jdGwo c3RydWN0IGNkZXYgKmRldiwgdV9sb25nIHpjbWQsIGNhZGRyXwogCUFTU0VSVDNVKGdldG1h am9yKGRldiksID09LCBkZGlfZHJpdmVyX21ham9yKHpmc19kaXApKTsKICNlbmRpZgogCi0J aWYgKGNmbGFnICE9IFpGU19DTURfQ09NUEFUX05PTkUpIHsKKwlpZiAoY29tcGF0KSB7CiAJ CWlmICh2ZWNudW0gPT0gWkZTX0lPQ19DT01QQVRfUEFTUykKIAkJCXJldHVybiAoMCk7CiAJ CWVsc2UgaWYgKHZlY251bSA9PSBaRlNfSU9DX0NPTVBBVF9GQUlMKQpAQCAtNTc3NywxMyAr NTc4OSwzMyBAQCB6ZnNkZXZfaW9jdGwoc3RydWN0IGNkZXYgKmRldiwgdV9sb25nIHpjbWQs IGNhZGRyXwogCQllcnJvciA9IFNFVF9FUlJPUihFRkFVTFQpOwogCQlnb3RvIG91dDsKIAl9 Ci0jZWxzZQotCWVycm9yID0gMDsKLSNlbmRpZgotCi0JaWYgKGNmbGFnICE9IFpGU19DTURf Q09NUEFUX05PTkUpIHsKKyNlbHNlCS8qICFpbGx1bW9zICovCisJLyoKKwkgKiBXZSBkb24n dCBhbGxvYy9mcmVlIHpjIG9ubHkgaWYgdGFsa2luZyB0byBsaWJyYXJ5IGlvY3RsIHZlcnNp b24gMgorCSAqLworCWlmIChjZmxhZyAhPSBaRlNfQ01EX0NPTVBBVF9MWkMpIHsKIAkJemMg PSBrbWVtX3phbGxvYyhzaXplb2YoemZzX2NtZF90KSwgS01fU0xFRVApOwogCQliemVybyh6 Yywgc2l6ZW9mKHpmc19jbWRfdCkpOworCX0gZWxzZSB7CisJCXpjID0gKHZvaWQgKilhcmc7 CisJCWVycm9yID0gMDsKKwl9CisKKwlpZiAobmV3aW9jKSB7CisJCXpjX2lvY3Bhcm0gPSAo dm9pZCAqKWFyZzsKKwkJaWYgKHpjX2lvY3Bhcm0tPnpmc19jbWRfc2l6ZSAhPSBzaXplb2Yo emZzX2NtZF90KSkgeworCQkJZXJyb3IgPSBTRVRfRVJST1IoRUZBVUxUKTsKKwkJCWdvdG8g b3V0OworCQl9CisJCWVycm9yID0gZGRpX2NvcHlpbigodm9pZCAqKXpjX2lvY3Bhcm0tPnpm c19jbWQsIHpjLAorCQkgICAgc2l6ZW9mKHpmc19jbWRfdCksIGZsYWcpOworCQlpZiAoZXJy b3IgIT0gMCkgeworCQkJZXJyb3IgPSBTRVRfRVJST1IoRUZBVUxUKTsKKwkJCWdvdG8gb3V0 OworCQl9CisJfQorCisJaWYgKGNvbXBhdCkgewogCQl6ZnNfY21kX2NvbXBhdF9nZXQoemMs IGFyZywgY2ZsYWcpOwogCQlvbGR2ZWNudW0gPSB2ZWNudW07CiAJCWVycm9yID0gemZzX2lv Y3RsX2NvbXBhdF9wcmUoemMsICZ2ZWNudW0sIGNmbGFnKTsKQEAgLTU3OTEsOCArNTgyMyw4 IEBAIHpmc2Rldl9pb2N0bChzdHJ1Y3QgY2RldiAqZGV2LCB1X2xvbmcgemNtZCwgY2FkZHJf CiAJCQlnb3RvIG91dDsKIAkJaWYgKG9sZHZlY251bSAhPSB2ZWNudW0pCiAJCQl2ZWMgPSAm emZzX2lvY192ZWNbdmVjbnVtXTsKLQl9IGVsc2UKLQkJemMgPSAodm9pZCAqKWFyZzsKKwl9 CisjZW5kaWYJLyogIWlsbHVtb3MgKi8KIAogCXpjLT56Y19pZmxhZ3MgPSBmbGFnICYgRktJ T0NUTDsKIAlpZiAoemMtPnpjX252bGlzdF9zcmNfc2l6ZSAhPSAwKSB7CkBAIC01ODAzLDcg KzU4MzUsNyBAQCB6ZnNkZXZfaW9jdGwoc3RydWN0IGNkZXYgKmRldiwgdV9sb25nIHpjbWQs IGNhZGRyXwogCX0KIAogCS8qIHJld3JpdGUgaW5udmwgZm9yIGJhY2t3YXJkcyBjb21wYXRp YmlsaXR5ICovCi0JaWYgKGNmbGFnICE9IFpGU19DTURfQ09NUEFUX05PTkUpCisJaWYgKGNv bXBhdCkKIAkJaW5udmwgPSB6ZnNfaW9jdGxfY29tcGF0X2lubnZsKHpjLCBpbm52bCwgdmVj bnVtLCBjZmxhZyk7CiAKIAkvKgpAQCAtNTg4MCw3ICs1OTEyLDcgQEAgemZzZGV2X2lvY3Rs KHN0cnVjdCBjZGV2ICpkZXYsIHVfbG9uZyB6Y21kLCBjYWRkcl8KIAkJZm52bGlzdF9mcmVl KGxvZ252KTsKIAogCQkvKiByZXdyaXRlIG91dG52bCBmb3IgYmFja3dhcmRzIGNvbXBhdGli aWxpdHkgKi8KLQkJaWYgKGNmbGFnICE9IFpGU19DTURfQ09NUEFUX05PTkUpCisJCWlmIChj ZmxhZyAhPSBaRlNfQ01EX0NPTVBBVF9OT05FICYmIGNmbGFnICE9IFpGU19DTURfQ09NUEFU X0xaQykKIAkJCW91dG52bCA9IHpmc19pb2N0bF9jb21wYXRfb3V0bnZsKHpjLCBvdXRudmws IHZlY251bSwKIAkJCSAgICBjZmxhZyk7CiAKQEAgLTU5MDQsMTAgKzU5MzYsMjMgQEAgemZz ZGV2X2lvY3RsKHN0cnVjdCBjZGV2ICpkZXYsIHVfbG9uZyB6Y21kLCBjYWRkcl8KIAogb3V0 OgogCW52bGlzdF9mcmVlKGlubnZsKTsKKworCWlmIChjb21wYXQpIHsKKwkJemZzX2lvY3Rs X2NvbXBhdF9wb3N0KHpjLCBjbWQsIGNmbGFnKTsKKwkJemZzX2NtZF9jb21wYXRfcHV0KHpj LCBhcmcsIHZlY251bSwgY2ZsYWcpOworCX0KKwogI2lmZGVmIGlsbHVtb3MKIAlyYyA9IGRk aV9jb3B5b3V0KHpjLCAodm9pZCAqKWFyZywgc2l6ZW9mICh6ZnNfY21kX3QpLCBmbGFnKTsK IAlpZiAoZXJyb3IgPT0gMCAmJiByYyAhPSAwKQogCQllcnJvciA9IFNFVF9FUlJPUihFRkFV TFQpOworI2Vsc2UKKwlpZiAobmV3aW9jKSB7CisJCXJjID0gZGRpX2NvcHlvdXQoemMsICh2 b2lkICopemNfaW9jcGFybS0+emZzX2NtZCwKKwkJICAgIHNpemVvZiAoemZzX2NtZF90KSwg ZmxhZyk7CisJCWlmIChlcnJvciA9PSAwICYmIHJjICE9IDApCisJCQllcnJvciA9IFNFVF9F UlJPUihFRkFVTFQpOworCX0KICNlbmRpZgogCWlmIChlcnJvciA9PSAwICYmIHZlYy0+enZl Y19hbGxvd19sb2cpIHsKIAkJY2hhciAqcyA9IHRzZF9nZXQoemZzX2FsbG93X2xvZ19rZXkp OwpAQCAtNTkxOSwxNCArNTk2NCwxNCBAQCBvdXQ6CiAJCQlzdHJmcmVlKHNhdmVkX3Bvb2xu YW1lKTsKIAl9CiAKLQlpZiAoY2ZsYWcgIT0gWkZTX0NNRF9DT01QQVRfTk9ORSkgewotCQl6 ZnNfaW9jdGxfY29tcGF0X3Bvc3QoemMsIGNtZCwgY2ZsYWcpOwotCQl6ZnNfY21kX2NvbXBh dF9wdXQoemMsIGFyZywgdmVjbnVtLCBjZmxhZyk7Ci0JCWttZW1fZnJlZSh6Yywgc2l6ZW9m ICh6ZnNfY21kX3QpKTsKLQl9Ci0KICNpZmRlZiBpbGx1bW9zCiAJa21lbV9mcmVlKHpjLCBz aXplb2YgKHpmc19jbWRfdCkpOworI2Vsc2UKKwkvKgorCSAqIFdlIGRvbid0IGFsbG9jL2Zy ZWUgemMgb25seSBpZiB0YWxraW5nIHRvIGxpYnJhcnkgaW9jdGwgdmVyc2lvbiAyCisJICov CisJaWYgKGNmbGFnICE9IFpGU19DTURfQ09NUEFUX0xaQykKKwkJa21lbV9mcmVlKHpjLCBz aXplb2YgKHpmc19jbWRfdCkpOwogI2VuZGlmCiAJcmV0dXJuIChlcnJvcik7CiB9Cg== --------------040107010104050106040805-- From owner-zfs-devel@FreeBSD.ORG Sun Apr 7 20:07:47 2013 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 815B4604; Sun, 7 Apr 2013 20:07:47 +0000 (UTC) (envelope-from mattjeet@gmail.com) Received: from mail-wg0-f46.google.com (mail-wg0-f46.google.com [74.125.82.46]) by mx1.freebsd.org (Postfix) with ESMTP id EBD62E04; Sun, 7 Apr 2013 20:07:46 +0000 (UTC) Received: by mail-wg0-f46.google.com with SMTP id l18so5061517wgh.1 for ; Sun, 07 Apr 2013 13:07:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:reply-to:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; bh=lFkinX7bwNCwc7c/fvIQOmNBBSYXCbVO1tIjMAICsTE=; b=DD/3fJlN0BYioRZsi9pM+Qd2HPbfyIiXbSwrJaUXs9Lrbro5tRTNa1b9T2PoG4EW1J bvm3jmQMFWR3uNf8kfIaNRPVO56PP5sbY5FOhNUp8aiVfisHvjymehbfKLwl5iMjuv4Y UxWqEWMmobStBGD3wTnwvxfUwF+YTOG93EY9wpUrVNAx8PaZIqhfrpiLyTmkEcNwWiyW 9CXvlkK+u7Fo2Zzv/oN3J1UUj7j5iKBtP5G/HMzh8zIvD3lVGHHBgRDwAfP+ZIAQLja6 R8D0cq8nA/BwEcXBlHaEeBBMJw7fa2hiBvb70advvC9EVEmakqd94LreyjfUUeZb1pUG Z1RA== MIME-Version: 1.0 X-Received: by 10.194.89.234 with SMTP id br10mr27468810wjb.43.1365365260542; Sun, 07 Apr 2013 13:07:40 -0700 (PDT) Sender: mattjeet@gmail.com Received: by 10.194.121.98 with HTTP; Sun, 7 Apr 2013 13:07:40 -0700 (PDT) In-Reply-To: <5161C904.8060600@FreeBSD.org> References: <5161C904.8060600@FreeBSD.org> Date: Sun, 7 Apr 2013 13:07:40 -0700 X-Google-Sender-Auth: hvCUGs_JTwfTtAYSdVSdn-WuCZQ Message-ID: Subject: Re: [REVIEW] patch for copyout on ioctl error From: Matt Olander To: Martin Matuska Content-Type: text/plain; charset=ISO-8859-1 Cc: "zfs-devel@freebsd.org" X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list Reply-To: matt@ixsystems.com List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 07 Apr 2013 20:07:47 -0000 On Sun, Apr 7, 2013 at 12:29 PM, Martin Matuska wrote: > Hi all, > > ZFS expects a copyout of zfs_cmd_t on a ioctl error. Our sys_ioctl() > doesn't copyout in this case. > > I am suggesting the attached patch for review to fix (or workaround) > this situation. Nice, thanks Martin! We will try and test it out this week ;) Cheers, -matt > > The idea is easy - zfs ioctl's won't send zfs_cmd_t anymore but a new > struct zfs_iocparm_t consisting of just three elements: > pointer to zfs_cmd_t > size of zfs_cmd_t > zfs_ioctl_version > > Size of zfs_cmd_t is intended for the kernel to re-verify if talking to > the correct version. zfs_ioctl_version is for future purposes to make it > easier for the > kernel to identify ioctl changes. > > The kernel module will do the copyin/copyout of zfs_cmd_t itself (like > illumos does). This makes future compatibility more simple and > zfsdev_ioctl() behaves much more like in illumos. The patch retains > compatibility with all previous states. > > Cheers, > mm > > -- > Martin Matuska > FreeBSD committer > http://blog.vx.sk > > > _______________________________________________ > zfs-devel@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/zfs-devel > To unsubscribe, send any mail to "zfs-devel-unsubscribe@freebsd.org" From owner-zfs-devel@FreeBSD.ORG Mon Apr 8 22:28:34 2013 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 546DCFE0; Mon, 8 Apr 2013 22:28:34 +0000 (UTC) (envelope-from pawel@dawidek.net) Received: from mail.dawidek.net (garage.dawidek.net [91.121.88.72]) by mx1.freebsd.org (Postfix) with ESMTP id 20ACDFBE; Mon, 8 Apr 2013 22:28:33 +0000 (UTC) Received: from localhost (89-73-195-149.dynamic.chello.pl [89.73.195.149]) by mail.dawidek.net (Postfix) with ESMTPSA id 5832DCBF; Tue, 9 Apr 2013 00:24:54 +0200 (CEST) Date: Tue, 9 Apr 2013 00:30:30 +0200 From: Pawel Jakub Dawidek To: Martin Matuska Subject: Re: [REVIEW] patch for copyout on ioctl error Message-ID: <20130408223030.GB1399@garage.freebsd.pl> References: <5161C904.8060600@FreeBSD.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="NDin8bjvE/0mNLFQ" Content-Disposition: inline In-Reply-To: <5161C904.8060600@FreeBSD.org> X-OS: FreeBSD 10.0-CURRENT amd64 User-Agent: Mutt/1.5.21 (2010-09-15) Cc: zfs-devel@FreeBSD.org X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Apr 2013 22:28:34 -0000 --NDin8bjvE/0mNLFQ Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sun, Apr 07, 2013 at 09:29:08PM +0200, Martin Matuska wrote: > Hi all, >=20 > ZFS expects a copyout of zfs_cmd_t on a ioctl error. Our sys_ioctl() > doesn't copyout in this case. >=20 > I am suggesting the attached patch for review to fix (or workaround) > this situation. >=20 > The idea is easy - zfs ioctl's won't send zfs_cmd_t anymore but a new > struct zfs_iocparm_t consisting of just three elements: > pointer to zfs_cmd_t > size of zfs_cmd_t > zfs_ioctl_version >=20 > Size of zfs_cmd_t is intended for the kernel to re-verify if talking to > the correct version. zfs_ioctl_version is for future purposes to make it > easier for the > kernel to identify ioctl changes. >=20 > The kernel module will do the copyin/copyout of zfs_cmd_t itself (like > illumos does). This makes future compatibility more simple and > zfsdev_ioctl() behaves much more like in illumos. The patch retains > compatibility with all previous states. I wonder if this is not the right time to add a IOC_ flag to natively support such behaviour in FreeBSD's ioctl. Not sure how much would that help to reduce complexity. If not much, then it is probably not worth it. The patch looks good, overall. One comment below. > Index: sys/cddl/contrib/opensolaris/common/zfs/zfs_ioctl_compat.c > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > --- sys/cddl/contrib/opensolaris/common/zfs/zfs_ioctl_compat.c (revision = 249226) > +++ sys/cddl/contrib/opensolaris/common/zfs/zfs_ioctl_compat.c (working c= opy) > @@ -575,11 +575,21 @@ int > zcmd_ioctl_compat(int fd, int request, zfs_cmd_t *zc, const int cflag) > { > int nc, ret; > + zfs_iocparm_t *zfs_iocparm; > void *zc_c; > unsigned long ncmd; > =20 > switch (cflag) { > case ZFS_CMD_COMPAT_NONE: > + ncmd =3D _IOWR('Z', request, struct zfs_iocparm); > + zfs_iocparm =3D malloc(sizeof(zfs_iocparm_t)); There is no check if malloc(3) succeeded, but actually would suggest eliminating malloc(3) altogether and allocating zfs_iocparm on the stack... > + zfs_iocparm->zfs_cmd_size =3D sizeof(zfs_cmd_t); > + zfs_iocparm->zfs_ioctl_version =3D ZFS_IOCVER_CURRENT; > + ret =3D ioctl(fd, ncmd, zfs_iocparm); > + free(zfs_iocparm); > + return (ret); =2E..Here you could just: return (ioctl(fd, ncmd, &zfs_iocparm)); PS. In case we decide to grow zfs_iocparam structure in the future it would be better to place zfs_ioctl_version as the first field. --=20 Pawel Jakub Dawidek http://www.wheelsystems.com FreeBSD committer http://www.FreeBSD.org Am I Evil? Yes, I Am! http://tupytaj.pl --NDin8bjvE/0mNLFQ Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (FreeBSD) iEYEARECAAYFAlFjRQYACgkQForvXbEpPzRUyQCeJgVsi2kAYs2VewSTy0giyaUP IgoAn14IZse4/103aozc8Yb7dlEkKeG3 =FVnm -----END PGP SIGNATURE----- --NDin8bjvE/0mNLFQ-- From owner-zfs-devel@FreeBSD.ORG Mon Apr 8 22:30:29 2013 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 39918B8; Mon, 8 Apr 2013 22:30:29 +0000 (UTC) (envelope-from delphij@delphij.net) Received: from anubis.delphij.net (anubis.delphij.net [64.62.153.212]) by mx1.freebsd.org (Postfix) with ESMTP id 2703FFE2; Mon, 8 Apr 2013 22:30:28 +0000 (UTC) Received: from zeta.ixsystems.com (drawbridge.ixsystems.com [206.40.55.65]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by anubis.delphij.net (Postfix) with ESMTPSA id 43C466199; Mon, 8 Apr 2013 15:30:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=delphij.net; s=anubis; t=1365460228; bh=KMhtM0K9I2lCZmEyHjo2HM0RZqcKAAxpdmIMWL+0nxc=; h=Date:From:Reply-To:To:CC:Subject:References:In-Reply-To; b=JNx9mXWKL/U6+bMibZTKLAh4gEzg8jA7/vzzCpTQ+Pc3T/8nHDZsVBFH4yUcfwy27 V/nuq7fxLG9aruMWFTpc5TH+dyfmxtQ49EeVHhelM+ebqTjAYoUmxcaOhAuB04vhex i2s5cvaBlBRUDCmePk3IfesV15VRUmXu0DF92DfI= Message-ID: <51634500.5000202@delphij.net> Date: Mon, 08 Apr 2013 15:30:24 -0700 From: Xin Li Organization: The FreeBSD Project MIME-Version: 1.0 To: Martin Matuska Subject: Re: [REVIEW] patch for copyout on ioctl error References: <5161C904.8060600@FreeBSD.org> In-Reply-To: <5161C904.8060600@FreeBSD.org> X-Enigmail-Version: 1.5.1 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: zfs-devel@FreeBSD.org X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list Reply-To: d@delphij.net List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Apr 2013 22:30:29 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 Hi, Martin, The patch seems reasonable to me, thanks for working on it! Cheers, - -- Xin LI https://www.delphij.net/ FreeBSD - The Power to Serve! Live free or die -----BEGIN PGP SIGNATURE----- iQEcBAEBCgAGBQJRY0UAAAoJEG80Jeu8UPuzl9cH/iGj1UT+zins1NSgdTD7F3FY YHLybbyVd+vKwtWbtHog4NqNgljSz82T8DqaH74xi1taXx+Tbi6JFwsd0ewebYQT 0U0TVqJgrhfXuIQFYLtQb3SPZfw5ablUCobcfAzi5GhCYWOA1i0bcDqAnBpz4nwr e41m/0CzG4bOHfj05sdJe1S52AIuip1fZwRfakJu1JgCuA6q8Fq016kYSrKt9wce c1gaYv24h+FI4jKVQAazNZ9bGzHsq2ZUVXZxCmv7riwoxtn7mmcl9fnqLC6J6N2I hAMuOw2DoewKeCGbNNpBnqlARBo0omx73SlemDRQszKMTxDudA6uRvTp4mdIWS4= =o72r -----END PGP SIGNATURE----- From owner-zfs-devel@FreeBSD.ORG Tue Apr 9 06:20:44 2013 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 33199A5C; Tue, 9 Apr 2013 06:20:44 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from mail.vx.sk (mail.vx.sk [176.9.45.25]) by mx1.freebsd.org (Postfix) with ESMTP id EAD3A2CE; Tue, 9 Apr 2013 06:20:43 +0000 (UTC) Received: from core.vx.sk (localhost [127.0.0.2]) by mail.vx.sk (Postfix) with ESMTP id DD1C73764B; Tue, 9 Apr 2013 08:20:42 +0200 (CEST) X-Virus-Scanned: amavisd-new at mail.vx.sk Received: from mail.vx.sk by core.vx.sk (amavisd-new, unix socket) with LMTP id owHbbbR2_XU7; Tue, 9 Apr 2013 08:20:41 +0200 (CEST) Received: from [127.0.0.1] (p5B16C953.dip0.t-ipconnect.de [91.22.201.83]) by mail.vx.sk (Postfix) with ESMTPSA id B3D2137641; Tue, 9 Apr 2013 08:20:40 +0200 (CEST) Message-ID: <5163B339.5080300@FreeBSD.org> Date: Tue, 09 Apr 2013 08:20:41 +0200 From: Martin Matuska User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:17.0) Gecko/20130328 Thunderbird/17.0.5 MIME-Version: 1.0 To: Pawel Jakub Dawidek Subject: Re: [REVIEW] patch for copyout on ioctl error References: <5161C904.8060600@FreeBSD.org> <20130408223030.GB1399@garage.freebsd.pl> In-Reply-To: <20130408223030.GB1399@garage.freebsd.pl> X-Enigmail-Version: 1.5.1 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Antivirus: avast! (VPS 130408-2, 08/04/2013), Outbound message X-Antivirus-Status: Clean Cc: zfs-devel@FreeBSD.org X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 09 Apr 2013 06:20:44 -0000 On 9. 4. 2013 0:30, Pawel Jakub Dawidek wrote: > On Sun, Apr 07, 2013 at 09:29:08PM +0200, Martin Matuska wrote: >> Hi all, >> >> ZFS expects a copyout of zfs_cmd_t on a ioctl error. Our sys_ioctl() >> doesn't copyout in this case. >> >> I am suggesting the attached patch for review to fix (or workaround) >> this situation. >> >> The idea is easy - zfs ioctl's won't send zfs_cmd_t anymore but a new >> struct zfs_iocparm_t consisting of just three elements: >> pointer to zfs_cmd_t >> size of zfs_cmd_t >> zfs_ioctl_version >> >> Size of zfs_cmd_t is intended for the kernel to re-verify if talking to >> the correct version. zfs_ioctl_version is for future purposes to make it >> easier for the >> kernel to identify ioctl changes. >> >> The kernel module will do the copyin/copyout of zfs_cmd_t itself (like >> illumos does). This makes future compatibility more simple and >> zfsdev_ioctl() behaves much more like in illumos. The patch retains >> compatibility with all previous states. > I wonder if this is not the right time to add a IOC_ flag to natively > support such behaviour in FreeBSD's ioctl. Not sure how much would that > help to reduce complexity. If not much, then it is probably not worth it. > > The patch looks good, overall. One comment below. > >> Index: sys/cddl/contrib/opensolaris/common/zfs/zfs_ioctl_compat.c >> =================================================================== >> --- sys/cddl/contrib/opensolaris/common/zfs/zfs_ioctl_compat.c (revision 249226) >> +++ sys/cddl/contrib/opensolaris/common/zfs/zfs_ioctl_compat.c (working copy) >> @@ -575,11 +575,21 @@ int >> zcmd_ioctl_compat(int fd, int request, zfs_cmd_t *zc, const int cflag) >> { >> int nc, ret; >> + zfs_iocparm_t *zfs_iocparm; >> void *zc_c; >> unsigned long ncmd; >> >> switch (cflag) { >> case ZFS_CMD_COMPAT_NONE: >> + ncmd = _IOWR('Z', request, struct zfs_iocparm); >> + zfs_iocparm = malloc(sizeof(zfs_iocparm_t)); > There is no check if malloc(3) succeeded, but actually would suggest > eliminating malloc(3) altogether and allocating zfs_iocparm on the > stack... > >> + zfs_iocparm->zfs_cmd_size = sizeof(zfs_cmd_t); >> + zfs_iocparm->zfs_ioctl_version = ZFS_IOCVER_CURRENT; >> + ret = ioctl(fd, ncmd, zfs_iocparm); >> + free(zfs_iocparm); >> + return (ret); > ...Here you could just: return (ioctl(fd, ncmd, &zfs_iocparm)); > > PS. In case we decide to grow zfs_iocparam structure in the future it > would be better to place zfs_ioctl_version as the first field. > Thanks for the review, I will integrate the recommendations. Martin -- Martin Matuska FreeBSD committer http://blog.vx.sk From owner-zfs-devel@FreeBSD.ORG Wed Apr 10 00:49:16 2013 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id AADFF774 for ; Wed, 10 Apr 2013 00:49:16 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: from mail-la0-x230.google.com (mail-la0-x230.google.com [IPv6:2a00:1450:4010:c03::230]) by mx1.freebsd.org (Postfix) with ESMTP id 32EDF936 for ; Wed, 10 Apr 2013 00:49:15 +0000 (UTC) Received: by mail-la0-f48.google.com with SMTP id fq13so6925823lab.21 for ; Tue, 09 Apr 2013 17:49:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=delphix.com; s=google; h=mime-version:x-received:in-reply-to:references:date:message-id :subject:from:to:cc:content-type; bh=kTZJQ/P6/5WQIBsCMr8zjoCkmIOEygVKj+AOTjFfEYk=; b=VXkODLFpkgec5OfjeNFFsePzSDiRcEe6VCYIMj5YicUQc08EodXfAW5yk+A7g/+qFL DTaj1AFLhc7MBLoHTY8ZmqFNcwWFtQLAesMThbw5ugTlBvvXnycSOfP9SxuNCpdlThB8 namTsMldx47rL9juKyxRcI9hcMEDi/8iXyjpM= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:x-received:in-reply-to:references:date:message-id :subject:from:to:cc:content-type:x-gm-message-state; bh=kTZJQ/P6/5WQIBsCMr8zjoCkmIOEygVKj+AOTjFfEYk=; b=omsEg469qt+qK76w2AqyT6BjViyQVLqZvGYRZ3YhGZS+WMMFS7WDJLnEa611+Y72UB AgtN6gzE/egihng+QfK5mrjFlynhIHdEBsjVonOA8Y1OcmRU/L/K5MbvdWX9BuUmG7sX D3rEO6o4P3vUHjTqZ16wFsxLlxxnKL/6HpgeBar+pcbcAqfbvOmsA+9a6Y4fkA5krwh1 XyhPmWflidOy2WNBppW5Q5hjtttfzGym0fn3JDjBX5DyVWB2DZSzQjyAR7WTkwUbk7rq vLKQt0g0aoerZFFq0IEHcD/XERWfrfOHs5Kleq+5QDIgHIBhBixKL4B/VhWcv5pop2Yo zk6A== MIME-Version: 1.0 X-Received: by 10.152.115.139 with SMTP id jo11mr13052751lab.57.1365554954891; Tue, 09 Apr 2013 17:49:14 -0700 (PDT) Received: by 10.114.18.72 with HTTP; Tue, 9 Apr 2013 17:49:14 -0700 (PDT) In-Reply-To: <5161C904.8060600@FreeBSD.org> References: <5161C904.8060600@FreeBSD.org> Date: Tue, 9 Apr 2013 17:49:14 -0700 Message-ID: Subject: Re: [REVIEW] patch for copyout on ioctl error From: Matthew Ahrens To: Martin Matuska X-Gm-Message-State: ALoCoQkSV+xzRSNQHecb71Hw8JK8tXUYbW4iKP4+LiAZA3gQtN7YMImlxM+Oh1dVFdU4DGQnHbfP Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: zfs-devel@freebsd.org X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 10 Apr 2013 00:49:16 -0000 On Sun, Apr 7, 2013 at 12:29 PM, Martin Matuska wrote: > zfs_ioctl_version is for future purposes to make it > easier for the kernel to identify ioctl changes. > Is this so that you can support old libzfs.so binaries with a new kernel module? Or to detect and fail if an old libzfs is used? FYI, in illumos we require that libzfs and the kernel be matched (likewise with other libraries, e.g. libc). So you will need to update the ioctl version # when merging changes from illumos. --matt From owner-zfs-devel@FreeBSD.ORG Wed Apr 10 06:19:56 2013 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 84CE18F7 for ; Wed, 10 Apr 2013 06:19:56 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from mail.vx.sk (mail.vx.sk [IPv6:2a01:4f8:150:6101::4]) by mx1.freebsd.org (Postfix) with ESMTP id 132866B9 for ; Wed, 10 Apr 2013 06:19:56 +0000 (UTC) Received: from core.vx.sk (localhost [127.0.0.2]) by mail.vx.sk (Postfix) with ESMTP id 3723935D72; Wed, 10 Apr 2013 08:19:52 +0200 (CEST) X-Virus-Scanned: amavisd-new at mail.vx.sk Received: from mail.vx.sk by core.vx.sk (amavisd-new, unix socket) with LMTP id ZlTk0WcxuspA; Wed, 10 Apr 2013 08:19:50 +0200 (CEST) Received: from [10.9.8.1] (chello085216226145.chello.sk [85.216.226.145]) by mail.vx.sk (Postfix) with ESMTPSA id 353EB35D66; Wed, 10 Apr 2013 08:19:49 +0200 (CEST) Message-ID: <51650484.30903@FreeBSD.org> Date: Wed, 10 Apr 2013 08:19:48 +0200 From: Martin Matuska User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:17.0) Gecko/20130328 Thunderbird/17.0.5 MIME-Version: 1.0 To: Matthew Ahrens Subject: Re: [REVIEW] patch for copyout on ioctl error References: <5161C904.8060600@FreeBSD.org> In-Reply-To: X-Enigmail-Version: 1.5.1 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: zfs-devel@freebsd.org X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 10 Apr 2013 06:19:56 -0000 Some time ago I have written a compatibility layer for FreeBSD for this purpose that translates and/or modifies necessary parts. This way we support non-matching kernel and utilities, with full backwards compatibility (new kernel, old utility) and to some degree forward compatibility (old kernel, new utility). So everytime the interface changes in an incompatible way I have to update this layer :-) Cheers, mm On 10.4.2013 2:49, Matthew Ahrens wrote: > On Sun, Apr 7, 2013 at 12:29 PM, Martin Matuska > wrote: > > zfs_ioctl_version is for future purposes to make it > easier for the kernel to identify ioctl changes. > > > Is this so that you can support old libzfs.so binaries with a new > kernel module? Or to detect and fail if an old libzfs is used? FYI, > in illumos we require that libzfs and the kernel be matched (likewise > with other libraries, e.g. libc). So you will need to update the > ioctl version # when merging changes from illumos. > > --matt -- Martin Matuska FreeBSD committer http://blog.vx.sk From owner-zfs-devel@FreeBSD.ORG Fri Apr 19 11:29:11 2013 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 329ADDF3; Fri, 19 Apr 2013 11:29:11 +0000 (UTC) (envelope-from davide.italiano@gmail.com) Received: from mail-vb0-x230.google.com (mail-vb0-x230.google.com [IPv6:2607:f8b0:400c:c02::230]) by mx1.freebsd.org (Postfix) with ESMTP id CABD6E91; Fri, 19 Apr 2013 11:29:10 +0000 (UTC) Received: by mail-vb0-f48.google.com with SMTP id p13so3490711vbe.35 for ; Fri, 19 Apr 2013 04:29:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:sender:date:x-google-sender-auth:message-id :subject:from:to:cc:content-type; bh=16gx/4S+OQX/8DmcFo8Ty6Nvs64zsQM2hJ/i1rpHSL0=; b=CBDsd/G9g4tI/KrfLEKynkgXdB0i2KTRyxJj8PV6/S0nmh8ykvhjRaGNa0vcxPSdn5 eI0KZPmABkBQXYLQCtjEDErnbIrZ6Y+SakoMrDyEUeqj99du7b4UiAkHTDQX+3Hw5Yc/ 7R4iwhRcI5vXZnb5flILJA/w0UFD8v8MelYGT779KqORWeWFpUKoGFNH7JlBAsrV71uT rBSQGTuNbBdRCevh88fjv11uvfkxtw3sIC0DJV1WBmRWHzHVNaJCy1c9w7jvSpFG5a6Z lRfRnLdqfLlS9dveAt2CZ3A/SGtcyhAQe5PXk0aAoIcexFX4LypjdG5ulxSL8nylHvth ex4A== MIME-Version: 1.0 X-Received: by 10.58.143.100 with SMTP id sd4mr11218938veb.39.1366370577440; Fri, 19 Apr 2013 04:22:57 -0700 (PDT) Sender: davide.italiano@gmail.com Received: by 10.220.203.199 with HTTP; Fri, 19 Apr 2013 04:22:57 -0700 (PDT) Date: Fri, 19 Apr 2013 13:22:57 +0200 X-Google-Sender-Auth: 5Qp9UDjtRFocH5sXFssbbClQZq0 Message-ID: Subject: zio unused UMA zones From: Davide Italiano To: zfs-devel@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 Cc: alc@freebsd.org X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Apr 2013 11:29:11 -0000 Hi. Looking at vmstat -z I saw zones zio_buf and zio_data_buf are created even if vfs.zfs.zio.use_uma is set to zero. If UMA is not used for allocating ZFS I/O buffers such zones remain unused. Is there any reason to create them I'm missing or is this just an oversight? In case it's the second choice, do you mind if I modify the code so that zones are created only if needed? Thanks, -- Davide From owner-zfs-devel@FreeBSD.ORG Sat Apr 20 21:12:03 2013 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 59C9B910; Sat, 20 Apr 2013 21:12:03 +0000 (UTC) (envelope-from pawel@dawidek.net) Received: from mail.dawidek.net (garage.dawidek.net [91.121.88.72]) by mx1.freebsd.org (Postfix) with ESMTP id 262F9F04; Sat, 20 Apr 2013 21:12:02 +0000 (UTC) Received: from localhost (89-73-195-149.dynamic.chello.pl [89.73.195.149]) by mail.dawidek.net (Postfix) with ESMTPSA id BA304370; Sat, 20 Apr 2013 23:01:14 +0200 (CEST) Date: Sat, 20 Apr 2013 23:07:13 +0200 From: Pawel Jakub Dawidek To: Davide Italiano Subject: Re: zio unused UMA zones Message-ID: <20130420210712.GB1391@garage.freebsd.pl> References: MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="p4qYPpj5QlsIQJ0K" Content-Disposition: inline In-Reply-To: X-OS: FreeBSD 10.0-CURRENT amd64 User-Agent: Mutt/1.5.21 (2010-09-15) Cc: alc@freebsd.org, zfs-devel@freebsd.org X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 20 Apr 2013 21:12:03 -0000 --p4qYPpj5QlsIQJ0K Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, Apr 19, 2013 at 01:22:57PM +0200, Davide Italiano wrote: > Hi. > Looking at vmstat -z I saw zones zio_buf and zio_data_buf are created > even if vfs.zfs.zio.use_uma is set to zero. > If UMA is not used for allocating ZFS I/O buffers such zones remain > unused. Is there any reason to create them I'm missing or is this just > an oversight? > In case it's the second choice, do you mind if I modify the code so > that zones are created only if needed? The only reason I can think of to leave those zone around is ability to switch between UMA-allocated buffers and malloc-allocated buffers at runtime, but currently I don't think this is possible (we would be able to tell how the given buffer was allocated), so most likely I first implemented this by using UMA, but when problems with KVA fragmentation appeared I added a switch to use malloc and haven't made UMA zones creations to depend on this switch. I'd be happy to review the patch. --=20 Pawel Jakub Dawidek http://www.wheelsystems.com FreeBSD committer http://www.FreeBSD.org Am I Evil? Yes, I Am! http://mobter.com --p4qYPpj5QlsIQJ0K Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (FreeBSD) iEYEARECAAYFAlFzA4AACgkQForvXbEpPzQufQCg9L3snxfJvQOdoQOyMRCOP+5q mcgAoNRPiRq51hhXq7eLBV425+pirPyD =ANCI -----END PGP SIGNATURE----- --p4qYPpj5QlsIQJ0K-- From owner-zfs-devel@FreeBSD.ORG Sat Apr 20 21:12:50 2013 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 6D9ED924; Sat, 20 Apr 2013 21:12:50 +0000 (UTC) (envelope-from pawel@dawidek.net) Received: from mail.dawidek.net (garage.dawidek.net [91.121.88.72]) by mx1.freebsd.org (Postfix) with ESMTP id 37CBFF08; Sat, 20 Apr 2013 21:12:50 +0000 (UTC) Received: from localhost (89-73-195-149.dynamic.chello.pl [89.73.195.149]) by mail.dawidek.net (Postfix) with ESMTPSA id B7AFF374; Sat, 20 Apr 2013 23:09:06 +0200 (CEST) Date: Sat, 20 Apr 2013 23:15:05 +0200 From: Pawel Jakub Dawidek To: Steven Hartland Subject: Re: ZFS ashift tuning revisited Message-ID: <20130420211504.GC1391@garage.freebsd.pl> References: <516FE82A.8060508@FreeBSD.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="B4IIlcmfBL/1gGOG" Content-Disposition: inline In-Reply-To: X-OS: FreeBSD 10.0-CURRENT amd64 User-Agent: Mutt/1.5.21 (2010-09-15) Cc: zfs-devel@FreeBSD.org, Alexander Motin X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 20 Apr 2013 21:12:50 -0000 --B4IIlcmfBL/1gGOG Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sat, Apr 20, 2013 at 06:20:58PM +0100, Steven Hartland wrote: > From: "Alexander Motin" > > I've never really investigated the question, but I guess that increasin= g=20 > > system-wide minimal desired ashift to 4K may cause additional disk spac= e=20 > > waste, possibly even more if compression is used. Though point that mor= e=20 > > and more disks are using 4K blocks is valid. From other side FusioIO=20 > > periodically mention that they can effectively handle records smaller= =20 > > then 512b, so who knows what we seen in 10 years. >=20 > It does indeed increase waste a bit but currently I'd say thats a small > downside compared to benefits of increased compatibility and performance > on more mainstream / popular HW. I saw presentation of one of the key ZFS devs, not sure which one (was it Matt Ahrens?) stating that 4kB sectors can waste a lot of space for reasonable small blocks and wide RAIDZ vdevs. For example if you use RAIDZ2 and 8kB recordsize you will be able to use only half of your pool size (effectively it becomes a mirror), because of how RAIDZ allocates the space. In my math half is a lot:) PS. I'm adding zfs-devel@ to the thread. --=20 Pawel Jakub Dawidek http://www.wheelsystems.com FreeBSD committer http://www.FreeBSD.org Am I Evil? Yes, I Am! http://mobter.com --B4IIlcmfBL/1gGOG Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (FreeBSD) iEYEARECAAYFAlFzBVgACgkQForvXbEpPzTa3ACfUqQgqBP77OH+1vp2u+SkZO6H 3a4AoKQOIRkBHwTy1HOsLHhHuDHq29Er =Mg33 -----END PGP SIGNATURE----- --B4IIlcmfBL/1gGOG-- From owner-zfs-devel@FreeBSD.ORG Sat Apr 20 22:35:03 2013 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id B43DAAF2; Sat, 20 Apr 2013 22:35:03 +0000 (UTC) (envelope-from prvs=18229b97a1=killing@multiplay.co.uk) Received: from mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) by mx1.freebsd.org (Postfix) with ESMTP id B37321115; Sat, 20 Apr 2013 22:35:02 +0000 (UTC) Received: from r2d2 ([46.65.172.4]) by mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) (MDaemon PRO v10.0.4) with ESMTP id md50003396862.msg; Sat, 20 Apr 2013 23:35:00 +0100 X-Spam-Processed: mail1.multiplay.co.uk, Sat, 20 Apr 2013 23:35:00 +0100 (not processed: message from valid local sender) X-MDDKIM-Result: neutral (mail1.multiplay.co.uk) X-MDRemoteIP: 46.65.172.4 X-Return-Path: prvs=18229b97a1=killing@multiplay.co.uk X-Envelope-From: killing@multiplay.co.uk Message-ID: From: "Steven Hartland" To: "Pawel Jakub Dawidek" References: <516FE82A.8060508@FreeBSD.org> <20130420211504.GC1391@garage.freebsd.pl> Subject: Re: ZFS ashift tuning revisited Date: Sat, 20 Apr 2013 23:35:23 +0100 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="iso-8859-1"; reply-type=original Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.5931 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.6157 Cc: zfs-devel@FreeBSD.org, Alexander Motin X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 20 Apr 2013 22:35:03 -0000 ----- Original Message ----- From: "Pawel Jakub Dawidek" On Sat, Apr 20, 2013 at 06:20:58PM +0100, Steven Hartland wrote: > > From: "Alexander Motin" > > > I've never really investigated the question, but I guess that increasing > > > system-wide minimal desired ashift to 4K may cause additional disk space > > > waste, possibly even more if compression is used. Though point that more > > > and more disks are using 4K blocks is valid. From other side FusioIO > > > periodically mention that they can effectively handle records smaller > > > then 512b, so who knows what we seen in 10 years. > > > > It does indeed increase waste a bit but currently I'd say thats a small > > downside compared to benefits of increased compatibility and performance > > on more mainstream / popular HW. > > I saw presentation of one of the key ZFS devs, not sure which one (was > it Matt Ahrens?) stating that 4kB sectors can waste a lot of space for > reasonable small blocks and wide RAIDZ vdevs. > > For example if you use RAIDZ2 and 8kB recordsize you will be able to use > only half of your pool size (effectively it becomes a mirror), because > of how RAIDZ allocates the space. > > In my math half is a lot:) > > PS. I'm adding zfs-devel@ to the thread. That is indeed a lot. I've got some machines here that are not yet in production I can test that and report back. Regards Steve ================================================ This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmaster@multiplay.co.uk. From owner-zfs-devel@FreeBSD.ORG Mon Apr 22 20:51:21 2013 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 0B0A57A1 for ; Mon, 22 Apr 2013 20:51:21 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: from mail-lb0-f170.google.com (mail-lb0-f170.google.com [209.85.217.170]) by mx1.freebsd.org (Postfix) with ESMTP id 8435D148D for ; Mon, 22 Apr 2013 20:51:20 +0000 (UTC) Received: by mail-lb0-f170.google.com with SMTP id 13so14059lba.1 for ; Mon, 22 Apr 2013 13:51:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=delphix.com; s=google; h=mime-version:x-received:in-reply-to:references:date:message-id :subject:from:to:cc:content-type; bh=WkV0jEQ3yDVbCbCbFeILwtq7H+Z6iafq7j1Q5QvQliM=; b=QPtA07cx+Dis7os4fFXmDCf+kBzczLTwuhAUijDx+DgFWIrTOYx914k7dz3tCPeC0Q HYb8sxb99uFFKfXpm/PIi3b6FA5u89bFU2f1XqkldDzVuLBVFbe8TGcMYV7sxZMkmm9f q7fR8DaUBIy9JQB9TPDq0827UWHl5J3FW3iE0= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:x-received:in-reply-to:references:date:message-id :subject:from:to:cc:content-type:x-gm-message-state; bh=WkV0jEQ3yDVbCbCbFeILwtq7H+Z6iafq7j1Q5QvQliM=; b=CfbzNkgEX/mbtk8DWgo/INax/B8D3RxDa7rdQWZqVB76eMI5BRZj0uO19xd47Ziq1B Nmlb2pRQMPAjR0dKGURUpkKRwQjhNITiLv2OvsbbcXgnT1ZwU7vliQULC3x4sYWpUp8p Opj7sC/ExvM3OMFhdYYoMVvtelAUzv9HZdVTJmuOTOa6PrJ5G2zzIAJ6xProVXnmRuKw CDZV27bSaV1w3mmlY0C/x9nQ4CFHrGSIN2e3yzD9ztvH7I1Qvu65yGlIGR2Wc2bTqV24 85tV608T/VEEdIGvqJlD+msSrunXAOd3tT4kwS5AMicB13DMg5tiHbrYNcV6ftFelpJv rS5A== MIME-Version: 1.0 X-Received: by 10.152.6.10 with SMTP id w10mr14256155law.30.1366663879403; Mon, 22 Apr 2013 13:51:19 -0700 (PDT) Received: by 10.114.22.4 with HTTP; Mon, 22 Apr 2013 13:51:19 -0700 (PDT) In-Reply-To: References: <516FE82A.8060508@FreeBSD.org> <20130420211504.GC1391@garage.freebsd.pl> Date: Mon, 22 Apr 2013 13:51:19 -0700 Message-ID: Subject: Re: ZFS ashift tuning revisited From: Matthew Ahrens To: Steven Hartland X-Gm-Message-State: ALoCoQmHpcmCdxlEgqZo37VGbGcceWumN2ppMgpJ+iv41IoV+6qt1thfqMhqUH4r7NcT3r1N+I/h Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: zfs-devel@freebsd.org, Alexander Motin X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Apr 2013 20:51:21 -0000 On Sat, Apr 20, 2013 at 3:35 PM, Steven Hartland wrote: > ----- Original Message ----- From: "Pawel Jakub Dawidek" > > On Sat, Apr 20, 2013 at 06:20:58PM +0100, Steven Hartland wrote: > >> > From: "Alexander Motin" >> > > I've never really investigated the question, but I guess that >> increasing > > system-wide minimal desired ashift to 4K may cause >> additional disk space > > waste, possibly even more if compression is used. >> Though point that more > > and more disks are using 4K blocks is valid. >> From other side FusioIO > > periodically mention that they can effectively >> handle records smaller > > then 512b, so who knows what we seen in 10 years. >> > > It does indeed increase waste a bit but currently I'd say thats a >> small >> > downside compared to benefits of increased compatibility and performance >> > on more mainstream / popular HW. >> >> I saw presentation of one of the key ZFS devs, not sure which one (was >> it Matt Ahrens?) stating that 4kB sectors can waste a lot of space for >> reasonable small blocks and wide RAIDZ vdevs. >> > That sounds right to me. RAIDZ-N always has to allocate at least N sectors for parity, and possibly a few additional sectors for alignment issues. So your example below is correct. I didn't see the beginning of this thread, but it sounds like you're considering changing the default ashift to 4k? I guess it depends on what kind of bug reports / complaints you want to field :) "my hardware is great but ZFS uses twice as much space" vs. "my hardware lies to ZFS, which therefore uses a slower way of accessing the hardware". --matt > >> For example if you use RAIDZ2 and 8kB recordsize you will be able to use >> only half of your pool size (effectively it becomes a mirror), because >> of how RAIDZ allocates the space. >> >> In my math half is a lot:) >> >> PS. I'm adding zfs-devel@ to the thread. >> > > That is indeed a lot. I've got some machines here that are not yet in > production I can test that and report back. > > > Regards > Steve > > ==============================**================== > This e.mail is private and confidential between Multiplay (UK) Ltd. and > the person or entity to whom it is addressed. In the event of misdirection, > the recipient is prohibited from using, copying, printing or otherwise > disseminating it or any information contained in it. > In the event of misdirection, illegible or incomplete transmission please > telephone +44 845 868 1337 > or return the E.mail to postmaster@multiplay.co.uk. > > ______________________________**_________________ > zfs-devel@freebsd.org mailing list > http://lists.freebsd.org/**mailman/listinfo/zfs-devel > To unsubscribe, send any mail to "zfs-devel-unsubscribe@**freebsd.org > " > From owner-zfs-devel@FreeBSD.ORG Mon Apr 29 17:56:16 2013 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id D6F35387; Mon, 29 Apr 2013 17:56:16 +0000 (UTC) (envelope-from davide.italiano@gmail.com) Received: from mail-ve0-x22d.google.com (mail-ve0-x22d.google.com [IPv6:2607:f8b0:400c:c01::22d]) by mx1.freebsd.org (Postfix) with ESMTP id 77C891453; Mon, 29 Apr 2013 17:56:16 +0000 (UTC) Received: by mail-ve0-f173.google.com with SMTP id ox1so3292224veb.32 for ; Mon, 29 Apr 2013 10:56:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; bh=QNUVWkwe/Oc/Z/Stu3ZF2OoiZqYqcOVt7Y2gP62TynI=; b=pLlvx8MX8AUcrK+457863xeDzBlTInIiz66SloPnnfMtUW3t43psE0GccDsXpQtDNl jk1VJeP8Mld9iOgfIsRO2u3BnekWGAEHingFKbyUQ8SNmJ0gsO/x9FcLEYKSk4EaL4ad 5EP74AxFkKezfb/Z7b8hohL0P7KZSzxvnhe2Yl7pvRcmSHR/i0hfGJ3r15igD5Hdq0OR p3oylTpEwSdR0SfSJmtDrDDlzzNmmxThFOx7NRBIjoAjIjIAj4LAXB80iLLOvcFtSa/j C3mY1wMbxL/KeoaGpjBF4CahiWUA/nmlaYSCAXSy8Uu+5ErdDWP+TtucnPoYELew4Mvd TlKw== MIME-Version: 1.0 X-Received: by 10.58.132.232 with SMTP id ox8mr33655928veb.45.1367258175174; Mon, 29 Apr 2013 10:56:15 -0700 (PDT) Sender: davide.italiano@gmail.com Received: by 10.220.203.199 with HTTP; Mon, 29 Apr 2013 10:56:15 -0700 (PDT) In-Reply-To: <20130420210712.GB1391@garage.freebsd.pl> References: <20130420210712.GB1391@garage.freebsd.pl> Date: Mon, 29 Apr 2013 19:56:15 +0200 X-Google-Sender-Auth: w5CXazL5_yUxqyLgQVA9cW3JdHQ Message-ID: Subject: Re: zio unused UMA zones From: Davide Italiano To: Pawel Jakub Dawidek Content-Type: text/plain; charset=ISO-8859-1 Cc: alc@freebsd.org, zfs-devel@freebsd.org X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 29 Apr 2013 17:56:16 -0000 On Sat, Apr 20, 2013 at 11:07 PM, Pawel Jakub Dawidek wrote: > On Fri, Apr 19, 2013 at 01:22:57PM +0200, Davide Italiano wrote: >> Hi. >> Looking at vmstat -z I saw zones zio_buf and zio_data_buf are created >> even if vfs.zfs.zio.use_uma is set to zero. >> If UMA is not used for allocating ZFS I/O buffers such zones remain >> unused. Is there any reason to create them I'm missing or is this just >> an oversight? >> In case it's the second choice, do you mind if I modify the code so >> that zones are created only if needed? > > The only reason I can think of to leave those zone around is ability to > switch between UMA-allocated buffers and malloc-allocated buffers at > runtime, but currently I don't think this is possible (we would be able > to tell how the given buffer was allocated), so most likely I first > implemented this by using UMA, but when problems with KVA fragmentation > appeared I added a switch to use malloc and haven't made UMA zones > creations to depend on this switch. > > I'd be happy to review the patch. > > -- > Pawel Jakub Dawidek http://www.wheelsystems.com > FreeBSD committer http://www.FreeBSD.org > Am I Evil? Yes, I Am! http://mobter.com This should suffice, I guess http://people.freebsd.org/~davide/review/zfs_noumacreate.diff Thanks, -- Davide "There are no solved problems; there are only problems that are more or less solved" -- Henri Poincare From owner-zfs-devel@FreeBSD.ORG Mon Apr 29 18:27:29 2013 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 7921FFB2; Mon, 29 Apr 2013 18:27:29 +0000 (UTC) (envelope-from pawel@dawidek.net) Received: from mail.dawidek.net (garage.dawidek.net [91.121.88.72]) by mx1.freebsd.org (Postfix) with ESMTP id 4284A16C0; Mon, 29 Apr 2013 18:27:28 +0000 (UTC) Received: from localhost (89-73-195-149.dynamic.chello.pl [89.73.195.149]) by mail.dawidek.net (Postfix) with ESMTPSA id BF328D6; Mon, 29 Apr 2013 20:23:37 +0200 (CEST) Date: Mon, 29 Apr 2013 20:29:50 +0200 From: Pawel Jakub Dawidek To: Davide Italiano Subject: Re: zio unused UMA zones Message-ID: <20130429182950.GB1417@garage.freebsd.pl> References: <20130420210712.GB1391@garage.freebsd.pl> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="MW5yreqqjyrRcusr" Content-Disposition: inline In-Reply-To: X-OS: FreeBSD 10.0-CURRENT amd64 User-Agent: Mutt/1.5.21 (2010-09-15) Cc: alc@freebsd.org, zfs-devel@freebsd.org X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 29 Apr 2013 18:27:29 -0000 --MW5yreqqjyrRcusr Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Mon, Apr 29, 2013 at 07:56:15PM +0200, Davide Italiano wrote: > On Sat, Apr 20, 2013 at 11:07 PM, Pawel Jakub Dawidek w= rote: > > On Fri, Apr 19, 2013 at 01:22:57PM +0200, Davide Italiano wrote: > >> Hi. > >> Looking at vmstat -z I saw zones zio_buf and zio_data_buf are created > >> even if vfs.zfs.zio.use_uma is set to zero. > >> If UMA is not used for allocating ZFS I/O buffers such zones remain > >> unused. Is there any reason to create them I'm missing or is this just > >> an oversight? > >> In case it's the second choice, do you mind if I modify the code so > >> that zones are created only if needed? > > > > The only reason I can think of to leave those zone around is ability to > > switch between UMA-allocated buffers and malloc-allocated buffers at > > runtime, but currently I don't think this is possible (we would be able > > to tell how the given buffer was allocated), so most likely I first > > implemented this by using UMA, but when problems with KVA fragmentation > > appeared I added a switch to use malloc and haven't made UMA zones > > creations to depend on this switch. > > > > I'd be happy to review the patch. > > > > -- > > Pawel Jakub Dawidek http://www.wheelsystems.com > > FreeBSD committer http://www.FreeBSD.org > > Am I Evil? Yes, I Am! http://mobter.com >=20 > This should suffice, I guess >=20 > http://people.freebsd.org/~davide/review/zfs_noumacreate.diff I wouldn't skip zfs_mg_alloc_failures initialization, so if you move the 'out' label above that, I'm fine with the patch. --=20 Pawel Jakub Dawidek http://www.wheelsystems.com FreeBSD committer http://www.FreeBSD.org Am I Evil? Yes, I Am! http://mobter.com --MW5yreqqjyrRcusr Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (FreeBSD) iEYEARECAAYFAlF+vB4ACgkQForvXbEpPzThqQCg4pGQU6PB4rAIjk8fDK8XzJMe lJ0Ani5vcK3TwWTxHlLbg7ijq2kd4HrA =C219 -----END PGP SIGNATURE----- --MW5yreqqjyrRcusr-- From owner-zfs-devel@FreeBSD.ORG Wed May 8 20:02:21 2013 Return-Path: Delivered-To: zfs-devel@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id BC5B8EB9; Wed, 8 May 2013 20:02:21 +0000 (UTC) (envelope-from delphij@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) by mx1.freebsd.org (Postfix) with ESMTP id 954499C5; Wed, 8 May 2013 20:02:21 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.6/8.14.6) with ESMTP id r48K2L6R074423; Wed, 8 May 2013 20:02:21 GMT (envelope-from delphij@freefall.freebsd.org) Received: (from delphij@localhost) by freefall.freebsd.org (8.14.6/8.14.6/Submit) id r48K2LIl074422; Wed, 8 May 2013 20:02:21 GMT (envelope-from delphij) Date: Wed, 8 May 2013 20:02:21 GMT Message-Id: <201305082002.r48K2LIl074422@freefall.freebsd.org> To: delphij@FreeBSD.org, freebsd-bugs@FreeBSD.org, zfs-devel@FreeBSD.org From: delphij@FreeBSD.org Subject: Re: kern/178387: [zfs] [patch] sparse files performance improvements X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 08 May 2013 20:02:21 -0000 Synopsis: [zfs] [patch] sparse files performance improvements Responsible-Changed-From-To: freebsd-bugs->zfs-devel Responsible-Changed-By: delphij Responsible-Changed-When: Wed May 8 20:01:52 UTC 2013 Responsible-Changed-Why: Over to zfs-devel@ for review. http://www.freebsd.org/cgi/query-pr.cgi?pr=178387 From owner-zfs-devel@FreeBSD.ORG Fri May 10 06:16:17 2013 Return-Path: Delivered-To: zfs-devel@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id F3BBE125; Fri, 10 May 2013 06:16:16 +0000 (UTC) (envelope-from delphij@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) by mx1.freebsd.org (Postfix) with ESMTP id D0A25F9B; Fri, 10 May 2013 06:16:16 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.7/8.14.7) with ESMTP id r4A6GG0o046241; Fri, 10 May 2013 06:16:16 GMT (envelope-from delphij@freefall.freebsd.org) Received: (from delphij@localhost) by freefall.freebsd.org (8.14.7/8.14.7/Submit) id r4A6GG1U046240; Fri, 10 May 2013 06:16:16 GMT (envelope-from delphij) Date: Fri, 10 May 2013 06:16:16 GMT Message-Id: <201305100616.r4A6GG1U046240@freefall.freebsd.org> To: delphij@FreeBSD.org, freebsd-fs@FreeBSD.org, zfs-devel@FreeBSD.org From: delphij@FreeBSD.org Subject: Re: kern/178467: [request] Optimized Checksum Code for ZFS X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 10 May 2013 06:16:17 -0000 Synopsis: [request] Optimized Checksum Code for ZFS Responsible-Changed-From-To: freebsd-fs->zfs-devel Responsible-Changed-By: delphij Responsible-Changed-When: Fri May 10 06:16:03 UTC 2013 Responsible-Changed-Why: Assign to zfs-devel@ http://www.freebsd.org/cgi/query-pr.cgi?pr=178467 From owner-zfs-devel@FreeBSD.ORG Fri May 10 08:50:01 2013 Return-Path: Delivered-To: zfs-devel@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 5D320924 for ; Fri, 10 May 2013 08:50:01 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) by mx1.freebsd.org (Postfix) with ESMTP id 4EC89ADA for ; Fri, 10 May 2013 08:50:01 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.7/8.14.7) with ESMTP id r4A8o1ai076692 for ; Fri, 10 May 2013 08:50:01 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.7/8.14.7/Submit) id r4A8o1C2076677; Fri, 10 May 2013 08:50:01 GMT (envelope-from gnats) Date: Fri, 10 May 2013 08:50:01 GMT Message-Id: <201305100850.r4A8o1C2076677@freefall.freebsd.org> To: zfs-devel@FreeBSD.org Cc: From: "Steven Hartland" Subject: Re: kern/178467: [request] Optimized Checksum Code for ZFS X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list Reply-To: Steven Hartland List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 10 May 2013 08:50:01 -0000 The following reply was made to PR kern/178467; it has been noted by GNATS. From: "Steven Hartland" To: , Cc: Subject: Re: kern/178467: [request] Optimized Checksum Code for ZFS Date: Fri, 10 May 2013 09:47:59 +0100 We would need something more to go no than "looks like" I'm afraid. Also Fletcher4 is the default checksum which achieves ~4GB/s per core in hashing performance, where as SHA-256 even with hand written assembly manages less than 1/10th that performance, so if your looking for performance for checksums use the default Fletcher4 instead of the SHA-256. That said new processors do have HW support which could be used to accelerate SHA-256 support, details of this can be found here:- http://download.intel.com/embedded/processor/whitepaper/327457.pdf These sorts of core feature enhancements should be discussed and implemented upstream at illumos. Regards Steve From owner-zfs-devel@FreeBSD.ORG Fri May 10 13:30:02 2013 Return-Path: Delivered-To: zfs-devel@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 0ED3CD39 for ; Fri, 10 May 2013 13:30:02 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) by mx1.freebsd.org (Postfix) with ESMTP id DBF649C3 for ; Fri, 10 May 2013 13:30:01 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.7/8.14.7) with ESMTP id r4ADU1K6031233 for ; Fri, 10 May 2013 13:30:01 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.7/8.14.7/Submit) id r4ADU1hS031232; Fri, 10 May 2013 13:30:01 GMT (envelope-from gnats) Date: Fri, 10 May 2013 13:30:01 GMT Message-Id: <201305101330.r4ADU1hS031232@freefall.freebsd.org> To: zfs-devel@FreeBSD.org Cc: From: Jason Keller Subject: RE: kern/178467: [request] Optimized Checksum Code for ZFS X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list Reply-To: Jason Keller List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 10 May 2013 13:30:02 -0000 The following reply was made to PR kern/178467; it has been noted by GNATS. From: Jason Keller To: Steven Hartland , "bug-followup@freebsd.org" Cc: Subject: RE: kern/178467: [request] Optimized Checksum Code for ZFS Date: Fri, 10 May 2013 13:21:26 +0000 Ok, thank you Steven - I'll gather up more detailed information when I have= my test environment fully fleshed out so I have absolute apples to apples = numbers and can fully constrain my testing to one hardware platform (the pl= atforms were slightly different, same processors and memory though). I'll = file that as an RFE with Illumos if that's what you think is best. I just = wanted to put that out there, since I certainly noticed the difference in m= y many weeks of testing different platforms here (OmniOS, Solaris 11.0, Sol= aris 11.1, FreeBSD 9.1, Nexenta 4 CE). Didn't really know where I should f= ile that particular RFE, so I figured I'd start with the kernel team. I di= dn't think that the SHA256 implementation in FreeBSD was taken exactly from= Illumos. JASON KELLER, CCNA | IT MANAGER | BBI INTERNATIONAL B: +1 701 738 4910 | :: +1 701 426 6240 | :: jkeller@bbiinternational.com j: Mon-Thu 0730-1130 & 1230-1700 UTC-5 and Fri 0730-1000 UTC-5 Like the job I did? Recommend me on LinkedIn > =20 ETHANOL PRODUCER MAGAZINE | BIODIESEL MAGAZINE | BIOMASS MAGAZINE | THE BAK= KEN MAGAZINE -----Original Message----- From: Steven Hartland [mailto:smh@freebsd.org]=20 Sent: Friday, May 10, 2013 3:48 AM To: bug-followup@freebsd.org; Jason Keller Subject: Re: kern/178467: [request] Optimized Checksum Code for ZFS We would need something more to go no than "looks like" I'm afraid. Also Fletcher4 is the default checksum which achieves ~4GB/s per core in ha= shing performance, where as SHA-256 even with hand written assembly manages= less than 1/10th that performance, so if your looking for performance for = checksums use the default Fletcher4 instead of the SHA-256. That said new processors do have HW support which could be used to accelera= te SHA-256 support, details of this can be found here:- http://download.int= el.com/embedded/processor/whitepaper/327457.pdf These sorts of core feature enhancements should be discussed and implemente= d upstream at illumos. Regards Steve From owner-zfs-devel@FreeBSD.ORG Fri May 10 14:30:01 2013 Return-Path: Delivered-To: zfs-devel@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 99AD4EEE for ; Fri, 10 May 2013 14:30:01 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) by mx1.freebsd.org (Postfix) with ESMTP id 72EB4D81 for ; Fri, 10 May 2013 14:30:01 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.7/8.14.7) with ESMTP id r4AEU1fQ043600 for ; Fri, 10 May 2013 14:30:01 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.7/8.14.7/Submit) id r4AEU1AI043599; Fri, 10 May 2013 14:30:01 GMT (envelope-from gnats) Date: Fri, 10 May 2013 14:30:01 GMT Message-Id: <201305101430.r4AEU1AI043599@freefall.freebsd.org> To: zfs-devel@FreeBSD.org Cc: From: "Steven Hartland" Subject: Re: kern/178467: [request] Optimized Checksum Code for ZFS X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list Reply-To: Steven Hartland List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 10 May 2013 14:30:01 -0000 The following reply was made to PR kern/178467; it has been noted by GNATS. From: "Steven Hartland" To: , Cc: Subject: Re: kern/178467: [request] Optimized Checksum Code for ZFS Date: Fri, 10 May 2013 15:29:11 +0100 Actually after double checking it looks like FreeBSD doesn't use the same SHA-256 implementation in ZFS as illumos so there may well be something to look at there. Would be good to know the difference in performance between FreeBSD and Openindiana (illumos distribution). Regards Steve From owner-zfs-devel@FreeBSD.ORG Fri May 10 14:40:04 2013 Return-Path: Delivered-To: zfs-devel@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id A1962AAF for ; Fri, 10 May 2013 14:40:04 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) by mx1.freebsd.org (Postfix) with ESMTP id 93FCBE35 for ; Fri, 10 May 2013 14:40:04 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.7/8.14.7) with ESMTP id r4AEe18i046077 for ; Fri, 10 May 2013 14:40:01 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.7/8.14.7/Submit) id r4AEe1bK046076; Fri, 10 May 2013 14:40:01 GMT (envelope-from gnats) Date: Fri, 10 May 2013 14:40:01 GMT Message-Id: <201305101440.r4AEe1bK046076@freefall.freebsd.org> To: zfs-devel@FreeBSD.org Cc: From: Jason Keller Subject: RE: kern/178467: [request] Optimized Checksum Code for ZFS X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list Reply-To: Jason Keller List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 10 May 2013 14:40:04 -0000 The following reply was made to PR kern/178467; it has been noted by GNATS. From: Jason Keller To: Steven Hartland , "bug-followup@freebsd.org" Cc: Subject: RE: kern/178467: [request] Optimized Checksum Code for ZFS Date: Fri, 10 May 2013 14:35:10 +0000 I'll gather some numbers together comparing OmniOS (Illumos) vs FreeBSD and= get back some numbers for you. I can use my Xeon E3-1240 at home for the = benchmarking, it'll just take me some time to gather everything together to= do it. Are there any specific tools you'd like me to run, or just basic z= pool iostat and mpstat / top -P ? JASON KELLER, CCNA | IT MANAGER | BBI INTERNATIONAL B: +1 701 738 4910 | :: +1 701 426 6240 | :: jkeller@bbiinternational.com j: Mon-Thu 0730-1130 & 1230-1700 UTC-5 and Fri 0730-1000 UTC-5 Like the job I did? Recommend me on LinkedIn > =20 ETHANOL PRODUCER MAGAZINE | BIODIESEL MAGAZINE | BIOMASS MAGAZINE | THE BAK= KEN MAGAZINE -----Original Message----- From: Steven Hartland [mailto:smh@freebsd.org]=20 Sent: Friday, May 10, 2013 9:29 AM To: bug-followup@freebsd.org; Jason Keller Subject: Re: kern/178467: [request] Optimized Checksum Code for ZFS Actually after double checking it looks like FreeBSD doesn't use the same S= HA-256 implementation in ZFS as illumos so there may well be something to l= ook at there. Would be good to know the difference in performance between FreeBSD and Ope= nindiana (illumos distribution). Regards Steve From owner-zfs-devel@FreeBSD.ORG Fri May 10 16:30:04 2013 Return-Path: Delivered-To: zfs-devel@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id D783A331 for ; Fri, 10 May 2013 16:30:04 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) by mx1.freebsd.org (Postfix) with ESMTP id CA3FE77E for ; Fri, 10 May 2013 16:30:04 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.7/8.14.7) with ESMTP id r4AGU1LU069211 for ; Fri, 10 May 2013 16:30:01 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.7/8.14.7/Submit) id r4AGU1oH069210; Fri, 10 May 2013 16:30:01 GMT (envelope-from gnats) Date: Fri, 10 May 2013 16:30:01 GMT Message-Id: <201305101630.r4AGU1oH069210@freefall.freebsd.org> To: zfs-devel@FreeBSD.org Cc: From: Jason Keller Subject: RE: kern/178467: [request] Optimized Checksum Code for ZFS X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list Reply-To: Jason Keller List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 10 May 2013 16:30:04 -0000 The following reply was made to PR kern/178467; it has been noted by GNATS. From: Jason Keller To: Steven Hartland , "bug-followup@freebsd.org" Cc: Subject: RE: kern/178467: [request] Optimized Checksum Code for ZFS Date: Fri, 10 May 2013 16:26:33 +0000 Steven, It also looks as if kern/125738 is related to hardware acceleration of SHA2= 56 in ZFS where it's available - PJD took this one, but doesn't look like h= e had time to work on it. So they are similar, though this request is a bi= t more broad. Also, is there any way to scrub my mobile number off there in the ticket de= tails? Totally spaced out that it's in my default signature here at work. --Jason From owner-zfs-devel@FreeBSD.ORG Fri May 10 17:40:48 2013 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 310A15AA; Fri, 10 May 2013 17:40:48 +0000 (UTC) (envelope-from delphij@delphij.net) Received: from anubis.delphij.net (anubis.delphij.net [IPv6:2001:470:1:117::25]) by mx1.freebsd.org (Postfix) with ESMTP id 16F38D2D; Fri, 10 May 2013 17:40:48 +0000 (UTC) Received: from zeta.ixsystems.com (drawbridge.ixsystems.com [206.40.55.65]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by anubis.delphij.net (Postfix) with ESMTPSA id 4DA5C9F5B; Fri, 10 May 2013 10:40:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=delphij.net; s=anubis; t=1368207647; bh=aqlbPnVfpnDrGMTYkQn8NTxC8vkoOaaMyCD8pk3Io/w=; h=Date:From:Reply-To:To:CC:Subject:References:In-Reply-To; b=n8CQKSHNHLSlnRptbbEjRcjBZOh06qbDGNnrD7aoQ7+w4Jy60Ao2N3cwPeIMLH7Wt nbmXkCeUIrTFXy3TwIp1SM1idn+6IEuUcfjSHgiwkHuRMME+GlrVmOL706mBk+m6ct owotBqYi3alOY6WCEGoG508YNYw67KKzLMXPIMRg= Message-ID: <518D311E.8030501@delphij.net> Date: Fri, 10 May 2013 10:40:46 -0700 From: Xin Li Organization: The FreeBSD Project MIME-Version: 1.0 To: Steven Hartland Subject: Re: kern/178467: [request] Optimized Checksum Code for ZFS References: <201305100850.r4A8o1C2076677@freefall.freebsd.org> In-Reply-To: <201305100850.r4A8o1C2076677@freefall.freebsd.org> X-Enigmail-Version: 1.5.1 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: zfs-devel@FreeBSD.org, "secteam@FreeBSD.org" X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list Reply-To: d@delphij.net List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 10 May 2013 17:40:48 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 On 05/10/13 01:50, Steven Hartland wrote: > The following reply was made to PR kern/178467; it has been noted > by GNATS. > > From: "Steven Hartland" To: > , Cc: > Subject: Re: kern/178467: [request] Optimized Checksum Code for > ZFS Date: Fri, 10 May 2013 09:47:59 +0100 > > We would need something more to go no than "looks like" I'm > afraid. > > Also Fletcher4 is the default checksum which achieves ~4GB/s per > core in hashing performance, where as SHA-256 even with hand > written assembly manages less than 1/10th that performance, so if > your looking for performance for checksums use the default > Fletcher4 instead of the SHA-256. > > That said new processors do have HW support which could be used to > accelerate SHA-256 support, details of this can be found here:- > http://download.intel.com/embedded/processor/whitepaper/327457.pdf > > These sorts of core feature enhancements should be discussed and > implemented upstream at illumos. Speaking for this -- I'm thinking that maybe these could leverage some sys/crypto facilities? These would be also helpful for some other kernel services like GELI, etc. Cheers, - -- Xin LI https://www.delphij.net/ FreeBSD - The Power to Serve! Live free or die -----BEGIN PGP SIGNATURE----- iQEcBAEBCgAGBQJRjTEdAAoJEG80Jeu8UPuz9asH/2kA95SBrc/pMiGSrT6soMDH Ao3f3+rFsAMsCSdzFxrEC0UoxI21+cMzGKsJjUvesf9yJa9A3bjL+sFqAIJ4exoE avE5mvYQhW/21po6C0iWSzAKi7JsjGpHJIMRw2PuPerRXtqrPoY1WtVTTiLPcFIp c/spGV0sh5ohaMstnSsb+gqvmDTsS426S8EJGxNdZcpGboFHMpFzeidmydTp7I86 vKZPI2esafoyEBtOtgCxRWGlocPnXtz/tIH9oyVcr6KveipFpyxAy5HfD+mYZXqY hZLKVUQ00gMefzlhdsgFRMHWfJuXee43XEe5mt3g2OXwIH6lv44d2JXqoFwlaP8= =iRct -----END PGP SIGNATURE----- From owner-zfs-devel@FreeBSD.ORG Mon May 13 11:08:21 2013 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id DA6C2E79 for ; Mon, 13 May 2013 11:08:21 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) by mx1.freebsd.org (Postfix) with ESMTP id B4153989 for ; Mon, 13 May 2013 11:08:21 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.7/8.14.7) with ESMTP id r4DB8LF7077914 for ; Mon, 13 May 2013 11:08:21 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.7/8.14.7/Submit) id r4DB8Lq8077912 for zfs-devel@FreeBSD.org; Mon, 13 May 2013 11:08:21 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 13 May 2013 11:08:21 GMT Message-Id: <201305131108.r4DB8Lq8077912@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: zfs-devel@FreeBSD.org Subject: Current problem reports assigned to zfs-devel@FreeBSD.org X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 13 May 2013 11:08:21 -0000 Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- s kern/178467 zfs-devel [request] Optimized Checksum Code for ZFS o kern/178387 zfs-devel [zfs] [patch] sparse files performance improvements 2 problems total. From owner-zfs-devel@FreeBSD.ORG Mon May 20 11:08:28 2013 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 51DBEBC7 for ; Mon, 20 May 2013 11:08:28 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) by mx1.freebsd.org (Postfix) with ESMTP id 2B85AEF5 for ; Mon, 20 May 2013 11:08:28 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.7/8.14.7) with ESMTP id r4KB8SLj084990 for ; Mon, 20 May 2013 11:08:28 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.7/8.14.7/Submit) id r4KB8RCX084988 for zfs-devel@FreeBSD.org; Mon, 20 May 2013 11:08:27 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 20 May 2013 11:08:27 GMT Message-Id: <201305201108.r4KB8RCX084988@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: zfs-devel@FreeBSD.org Subject: Current problem reports assigned to zfs-devel@FreeBSD.org X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 20 May 2013 11:08:28 -0000 Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- s kern/178467 zfs-devel [request] Optimized Checksum Code for ZFS o kern/178387 zfs-devel [zfs] [patch] sparse files performance improvements 2 problems total. From owner-zfs-devel@FreeBSD.ORG Mon May 27 11:08:28 2013 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 11426926 for ; Mon, 27 May 2013 11:08:28 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) by mx1.freebsd.org (Postfix) with ESMTP id DE2C5871 for ; Mon, 27 May 2013 11:08:27 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.7/8.14.7) with ESMTP id r4RB8RqX018089 for ; Mon, 27 May 2013 11:08:27 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.7/8.14.7/Submit) id r4RB8R3g018087 for zfs-devel@FreeBSD.org; Mon, 27 May 2013 11:08:27 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 27 May 2013 11:08:27 GMT Message-Id: <201305271108.r4RB8R3g018087@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: zfs-devel@FreeBSD.org Subject: Current problem reports assigned to zfs-devel@FreeBSD.org X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 27 May 2013 11:08:28 -0000 Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- s kern/178467 zfs-devel [request] Optimized Checksum Code for ZFS o kern/178387 zfs-devel [zfs] [patch] sparse files performance improvements 2 problems total. From owner-zfs-devel@FreeBSD.ORG Mon Jun 3 11:08:23 2013 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 6E3F8A6E for ; Mon, 3 Jun 2013 11:08:23 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) by mx1.freebsd.org (Postfix) with ESMTP id 4616F16D5 for ; Mon, 3 Jun 2013 11:08:23 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.7/8.14.7) with ESMTP id r53B8NGW017089 for ; Mon, 3 Jun 2013 11:08:23 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.7/8.14.7/Submit) id r53B8MJM017087 for zfs-devel@FreeBSD.org; Mon, 3 Jun 2013 11:08:22 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 3 Jun 2013 11:08:22 GMT Message-Id: <201306031108.r53B8MJM017087@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: zfs-devel@FreeBSD.org Subject: Current problem reports assigned to zfs-devel@FreeBSD.org X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Jun 2013 11:08:23 -0000 Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- s kern/178467 zfs-devel [request] Optimized Checksum Code for ZFS o kern/178387 zfs-devel [zfs] [patch] sparse files performance improvements 2 problems total. From owner-zfs-devel@FreeBSD.ORG Sat Jun 8 18:44:51 2013 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 8F3E4482; Sat, 8 Jun 2013 18:44:51 +0000 (UTC) (envelope-from smh@freebsd.org) Received: from smtp1.multiplay.co.uk (smtp1.multiplay.co.uk [85.236.96.35]) by mx1.freebsd.org (Postfix) with ESMTP id 42C321FDA; Sat, 8 Jun 2013 18:44:48 +0000 (UTC) Received: by smtp1.multiplay.co.uk (Postfix, from userid 65534) id B53A120E7088A; Sat, 8 Jun 2013 18:44:46 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.multiplay.co.uk X-Spam-Level: X-Spam-Status: No, score=-2.5 required=8.0 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham version=3.3.1 Received: from r2d2 (82-69-141-170.dsl.in-addr.zen.co.uk [82.69.141.170]) by smtp1.multiplay.co.uk (Postfix) with ESMTPA id 1010C20E70847; Sat, 8 Jun 2013 18:44:44 +0000 (UTC) Message-ID: From: "Steven Hartland" To: Subject: Feeback on patch to add ZFS support to mdconfig rc.d scripts Date: Sat, 8 Jun 2013 19:44:45 +0100 MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="----=_NextPart_000_02DE_01CE6480.A0B2FFC0" X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.5931 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.6157 Cc: zfs-devel@FreeBSD.org X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 08 Jun 2013 18:44:51 -0000 This is a multi-part message in MIME format. ------=_NextPart_000_02DE_01CE6480.A0B2FFC0 Content-Type: text/plain; format=flowed; charset="iso-8859-1"; reply-type=original Content-Transfer-Encoding: 7bit Attached is a patch which adds ZFS support to the mdconfig rc.d scritps along with some other little fixes while I was there. Our use case for this here is to create a ZFS pool on a swap backed node for temporary data which supports all the cool features of ZFS, for us thats compression. Possible enhancements, suggested by pjd in IRC include default flags such as:-o cachefile=none Current summary:- ======= Add ZFS pool creation support to mdconfig rc.d script Fix failure error redirection in mount check Fix invalid configuration detection enabling -t vnode to be skipped if -f is specified as supported by mdconfig. Fix fsck not working if mount point is not present in fstab. TODO: update rc.conf man page ======= I've tested on a number of configs including:- mdconfig_md0="-t swap -s 12g" mdconfig_md0_zfs_pool="tmpfs" mdconfig_md0_zfs_flags="-m /mnt -o cachefile=none -O compression=on -O atime=off -O exec=off -O primarycache=none -O sync=disabled" mdconfig_md1="-f /data/test.vnode" mdconfig_md1_zfs_pool="vnodefs" mdconfig_md1_zfs_flags="-m /root/vnode -o cachefile=none" mdconfig_md2="-f /data/test2.vnode" mdconfig_md3="-s 12g" # Should error, didn't before. What do people think? Regards steve ------=_NextPart_000_02DE_01CE6480.A0B2FFC0 Content-Type: application/octet-stream; name="mdconfig-zfs.patch" Content-Transfer-Encoding: quoted-printable Content-Disposition: attachment; filename="mdconfig-zfs.patch" Add ZFS pool creation support to mdconfig rc.d script=0A= =0A= Fix failure error redirection in mount check=0A= =0A= Fix invalid configuration detection enabling -t vnode=0A= to be skipped if -f is specified as supported=0A= by mdconfig.=0A= =0A= Fix fsck not working if mount point is not present in fstab.=0A= =0A= TODO: update rc.conf man page=0A= --- /etc/rc.d/mdconfig.orig 2013-06-08 00:55:48.599761883 +0000=0A= +++ /etc/rc.d/mdconfig 2013-06-08 16:24:06.092215311 +0000=0A= @@ -61,6 +61,29 @@=0A= fi=0A= }=0A= =0A= +get_opt()=0A= +{=0A= + local _flag _param _opt=0A= +=0A= + _flag=3D$1=0A= + _param=3D$2=0A= +=0A= + _opt=3D${_config##*-${_flag}\ }=0A= + if [ "${_opt}" =3D "${_config}" ]; then=0A= + _opt=3D""=0A= + else=0A= + _opt=3D${_opt%%\ *}=0A= + fi=0A= + eval _${_param}=3D${_opt}=0A= + debug "${_md} ${_param}=3D${_opt}"=0A= +=0A= + if [ -z "${_opt}" ]; then=0A= + return 0=0A= + fi=0A= +=0A= + return 1=0A= +}=0A= +=0A= init_variables()=0A= {=0A= local _i=0A= @@ -70,16 +93,20 @@=0A= _dev=3D"/dev/${_md}"=0A= eval _config=3D\$mdconfig_${_md}=0A= eval _newfs=3D\$mdconfig_${_md}_newfs=0A= + eval _zpool=3D\$mdconfig_${_md}_zfs_pool=0A= + eval _zflags=3D\$mdconfig_${_md}_zfs_flags=0A= =0A= - _type=3D${_config##*-t\ }=0A= - _type=3D${_type%%\ *}=0A= + get_opt "t" "type"=0A= + get_opt "f" "file"=0A= if [ -z "${_type}" ]; then=0A= - err 1 "You need to specify \"-t \" in mdconfig_${_md}"=0A= + if [ -n "${_file}" ]; then=0A= + _type=3D"vnode"=0A= + else=0A= + err 1 "You need to specify \"-t \" in mdconfig_${_md}"=0A= + fi=0A= fi=0A= =0A= if [ "${_type}" =3D "vnode" ]; then=0A= - _file=3D${_config##*-f\ }=0A= - _file=3D${_file%%\ *}=0A= if [ -z "${_file}" ]; then=0A= err 2 "You need to specify \"-f \" in mdconfig_${_md} for = vnode devices"=0A= fi=0A= @@ -96,11 +123,13 @@=0A= debug "${_md} file: ${_file}"=0A= debug "${_md} fs: ${_fs}"=0A= debug "${_md} newfs flags: ${_newfs}"=0A= + debug "${_md} ZFS pool: ${_zpool}"=0A= + debug "${_md} ZFS flags: ${_zflags}"=0A= }=0A= =0A= mdconfig_start()=0A= {=0A= - local _md _mp _config _type _dev _file _fs _newfs _fsck_cmd=0A= + local _md _mp _config _type _dev _file _fs _newfs _fsck_cmd _zpool = _zflags=0A= =0A= for _md in ${_mdconfig_list}; do=0A= init_variables ${_md}=0A= @@ -126,6 +155,18 @@=0A= echo "Creating ${_md} device failed, moving on."=0A= continue=0A= fi=0A= +=0A= + if [ -n "${_zpool}" ]; then=0A= + if [ "${_type}" =3D "vnode" ]; then=0A= + echo "Importing ZFS pool ${_zpool}."=0A= + zpool import ${_zpool}=0A= + else=0A= + echo "Creating ZFS pool ${_zpool}."=0A= + zpool create ${_zflags} ${_zpool} ${_md}=0A= + fi=0A= + continue=0A= + fi=0A= +=0A= # Skip fsck for uzip devices.=0A= if [ "${_type}" =3D "vnode" ]; then=0A= if [ "${_file}" !=3D "${_file%.uzip}" ]; then=0A= @@ -143,7 +184,8 @@=0A= else=0A= newfs ${_newfs} ${_dev} >/dev/null=0A= fi=0A= - if mount -d ${_dev} 2>&1 >/dev/null; then=0A= +=0A= + if mount -d ${_dev} >/dev/null 2>&1; then=0A= echo "Mounting ${_dev}."=0A= mount ${_dev}=0A= fi=0A= @@ -159,7 +201,12 @@=0A= init_variables ${_md}=0A= if [ "${_type}" !=3D "vnode" -o "${_fs}" =3D "/" ]; then=0A= for _i in `df ${_dev} 2>/dev/null`; do _mp=3D${_i}; done=0A= - if [ -z "${_mp}" -o "${_mp}" !=3D "${_mp%%%}" ]; then=0A= + if [ -n "${_zpool}" ]; then=0A= + if zpool list ${_zpool} >/dev/null 2>&1; then=0A= + echo "Exporting ZFS pool ${_zpool}."=0A= + zpool export ${_zpool}=0A= + fi=0A= + elif [ -z "${_mp}" -o "${_mp}" !=3D "${_mp%%%}" ]; then=0A= echo "Device ${_dev} isn't mounted."=0A= else=0A= echo "Umounting ${_dev}."=0A= --- /etc/rc.d/mdconfig2.orig 2013-06-08 16:33:49.583875813 +0000=0A= +++ /etc/rc.d/mdconfig2 2013-06-08 17:28:02.079420674 +0000=0A= @@ -62,6 +62,29 @@=0A= fi=0A= }=0A= =0A= +get_opt()=0A= +{=0A= + local _flag _param _opt=0A= +=0A= + _flag=3D$1=0A= + _param=3D$2=0A= +=0A= + _opt=3D${_config##*-${_flag}\ }=0A= + if [ "${_opt}" =3D "${_config}" ]; then=0A= + _opt=3D""=0A= + else=0A= + _opt=3D${_opt%%\ *}=0A= + fi=0A= + eval _${_param}=3D${_opt}=0A= + debug "${_md} ${_param}=3D${_opt}"=0A= +=0A= + if [ -z "${_opt}" ]; then=0A= + return 0=0A= + fi=0A= +=0A= + return 1=0A= +}=0A= +=0A= init_variables()=0A= {=0A= local _i=0A= @@ -75,16 +98,19 @@=0A= eval _perms=3D\$mdconfig_${_md}_perms=0A= eval _files=3D\$mdconfig_${_md}_files=0A= eval _populate=3D\$mdconfig_${_md}_cmd=0A= + eval _zpool=3D\$mdconfig_${_md}_zfs_pool=0A= =0A= - _type=3D${_config##*-t\ }=0A= - _type=3D${_type%%\ *}=0A= + get_opt "t" "type"=0A= + get_opt "f" "file"=0A= if [ -z "${_type}" ]; then=0A= - err 1 "You need to specify \"-t \" in mdconfig_${_md}"=0A= + if [ -n "${_file}" ]; then=0A= + _type=3D"vnode"=0A= + else=0A= + err 1 "You need to specify \"-t \" in mdconfig_${_md}"=0A= + fi=0A= fi=0A= =0A= if [ "${_type}" =3D "vnode" ]; then=0A= - _file=3D${_config##*-f\ }=0A= - _file=3D${_file%%\ *}=0A= if [ -z "${_file}" ]; then=0A= err 2 "You need to specify \"-f \" in mdconfig_${_md} for = vnode devices"=0A= fi=0A= @@ -136,26 +162,42 @@=0A= echo "Creating ${_md} device failed, moving on."=0A= continue=0A= fi=0A= - # Skip fsck for uzip devices.=0A= - if [ "${_file}" !=3D "${_file%.uzip}" ]; then=0A= - _fsck_cmd=3D":"=0A= - elif checkyesno background_fsck; then=0A= - _fsck_cmd=3D"fsck -F"=0A= +=0A= + if [ -n "${_zpool}" ]; then=0A= + if [ "${_type}" =3D "vnode" ]; then=0A= + echo "Importing ZFS pool ${_zpool}."=0A= + zpool import ${_zpool}=0A= + else=0A= + echo "Creating ZFS pool ${_zpool}."=0A= + zpool create ${_zflags} ${_zpool} ${_md}=0A= + fi=0A= else=0A= - _fsck_cmd=3D"fsck"=0A= - fi=0A= - if ! eval ${_fsck_cmd} -p ${_dev} >/dev/null; then=0A= - echo "Fsck failed on ${_dev}, not mounting the filesystem."=0A= - continue=0A= - fi=0A= - if mount -d ${_dev} >/dev/null 2>&1; then=0A= - echo "Mounting ${_dev}."=0A= - mount ${_dev}=0A= + # Skip fsck for uzip devices.=0A= + if [ "${_file}" !=3D "${_file%.uzip}" ]; then=0A= + _fsck_cmd=3D":"=0A= + elif checkyesno background_fsck; then=0A= + _fsck_cmd=3D"fsck -t ufs -F"=0A= + else=0A= + _fsck_cmd=3D"fsck -t ufs"=0A= + fi=0A= + if ! eval ${_fsck_cmd} -p ${_dev} >/dev/null; then=0A= + echo "Fsck failed on ${_dev}, not mounting the filesystem."=0A= + continue=0A= + fi=0A= + if mount -d ${_dev} >/dev/null 2>&1; then=0A= + echo "Mounting ${_dev}."=0A= + mount ${_dev}=0A= + fi=0A= fi=0A= fi=0A= =0A= - for _i in `df ${_dev} 2>/dev/null`; do _mp=3D${_i}; done=0A= - if [ ! -z "${_mp}" -a "${_mp}" =3D "${_mp%%%}" ]; then=0A= + if [ -n "${_zpool}" ]; then=0A= + _mp=3D`zfs list -H -o mountpoint ${_zpool} 2>/dev/null`=0A= + else=0A= + for _i in `df ${_dev} 2>/dev/null`; do _mp=3D${_i}; done=0A= + fi=0A= +=0A= + if [ -n "${_mp}" -a "${_mp}" =3D "${_mp%%%}" ]; then=0A= _mounted=3D"yes"=0A= fi=0A= =0A= ------=_NextPart_000_02DE_01CE6480.A0B2FFC0-- From owner-zfs-devel@FreeBSD.ORG Sat Jun 8 19:20:43 2013 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id BAB56D9E; Sat, 8 Jun 2013 19:20:43 +0000 (UTC) (envelope-from hrs@FreeBSD.org) Received: from mail.allbsd.org (gatekeeper.allbsd.org [IPv6:2001:2f0:104:e001::32]) by mx1.freebsd.org (Postfix) with ESMTP id 0BA6A10E6; Sat, 8 Jun 2013 19:20:39 +0000 (UTC) Received: from alph.d.allbsd.org (p2175-ipbf701funabasi.chiba.ocn.ne.jp [122.25.209.175]) (authenticated bits=128) by mail.allbsd.org (8.14.5/8.14.5) with ESMTP id r58JKMPt042630 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sun, 9 Jun 2013 04:20:32 +0900 (JST) (envelope-from hrs@FreeBSD.org) Received: from localhost (localhost [127.0.0.1]) (authenticated bits=0) by alph.d.allbsd.org (8.14.5/8.14.5) with ESMTP id r58JKK9F061043; Sun, 9 Jun 2013 04:20:22 +0900 (JST) (envelope-from hrs@FreeBSD.org) Date: Sun, 09 Jun 2013 04:19:03 +0900 (JST) Message-Id: <20130609.041903.541782050134731313.hrs@allbsd.org> To: smh@FreeBSD.org Subject: Re: Feeback on patch to add ZFS support to mdconfig rc.d scripts From: Hiroki Sato In-Reply-To: References: X-PGPkey-fingerprint: BDB3 443F A5DD B3D0 A530 FFD7 4F2C D3D8 2793 CF2D X-Mailer: Mew version 6.5 on Emacs 24.3 / Mule 6.0 (HANACHIRUSATO) Mime-Version: 1.0 Content-Type: Multipart/Signed; protocol="application/pgp-signature"; micalg=pgp-sha1; boundary="--Security_Multipart(Sun_Jun__9_04_19_03_2013_534)--" Content-Transfer-Encoding: 7bit X-Virus-Scanned: clamav-milter 0.97.4 at gatekeeper.allbsd.org X-Virus-Status: Clean X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (mail.allbsd.org [133.31.130.32]); Sun, 09 Jun 2013 04:20:32 +0900 (JST) X-Spam-Status: No, score=-94.5 required=13.0 tests=CONTENT_TYPE_PRESENT, ONLY1HOPDIRECT,RCVD_IN_PBL,SAMEHELOBY2HOP,USER_IN_WHITELIST autolearn=no version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on gatekeeper.allbsd.org Cc: rc@FreeBSD.org, zfs-devel@FreeBSD.org X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 08 Jun 2013 19:20:43 -0000 ----Security_Multipart(Sun_Jun__9_04_19_03_2013_534)-- Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit "Steven Hartland" wrote in : sm> Attached is a patch which adds ZFS support to the sm> mdconfig rc.d scritps along with some other little sm> fixes while I was there. sm> sm> Our use case for this here is to create a ZFS pool sm> on a swap backed node for temporary data which supports sm> all the cool features of ZFS, for us thats compression. sm> sm> Possible enhancements, suggested by pjd in IRC include sm> default flags such as:-o cachefile=none sm> sm> Current summary:- sm> ======= sm> Add ZFS pool creation support to mdconfig rc.d script sm> sm> Fix failure error redirection in mount check sm> sm> Fix invalid configuration detection enabling -t vnode sm> to be skipped if -f is specified as supported sm> by mdconfig. sm> sm> Fix fsck not working if mount point is not present in fstab. sm> sm> TODO: update rc.conf man page sm> ======= I think this should be separated into adding non-fs md support into rc.d/mdconfig and on-the-fly zpool creation/import into rc.d/zfs since the rc.d/mdconfig script does not (and should not) depend on zfs.ko module. rc.d/mdconfig is located before mountcritlocal, so probably rc.d/zfs_md or something is needed just after rc.d/mdconfig2. -- Hiroki ----Security_Multipart(Sun_Jun__9_04_19_03_2013_534)-- Content-Type: application/pgp-signature Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.13 (FreeBSD) iEYEABECAAYFAlGzg6cACgkQTyzT2CeTzy3ABgCZARMSpN4m7I251NYJHcxJbQNi mhIAn2AGQsDRK99iL9uo13uU0uFcNJlV =yMgS -----END PGP SIGNATURE----- ----Security_Multipart(Sun_Jun__9_04_19_03_2013_534)---- From owner-zfs-devel@FreeBSD.ORG Sun Jun 9 18:13:36 2013 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id EB925710; Sun, 9 Jun 2013 18:13:36 +0000 (UTC) (envelope-from prvs=18721298a7=killing@multiplay.co.uk) Received: from mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) by mx1.freebsd.org (Postfix) with ESMTP id 317E51819; Sun, 9 Jun 2013 18:13:35 +0000 (UTC) Received: from r2d2 ([82.69.141.170]) by mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) (MDaemon PRO v10.0.4) with ESMTP id md50004232324.msg; Sun, 09 Jun 2013 19:13:35 +0100 X-Spam-Processed: mail1.multiplay.co.uk, Sun, 09 Jun 2013 19:13:35 +0100 (not processed: message from valid local sender) X-MDDKIM-Result: neutral (mail1.multiplay.co.uk) X-MDRemoteIP: 82.69.141.170 X-Return-Path: prvs=18721298a7=killing@multiplay.co.uk X-Envelope-From: killing@multiplay.co.uk Message-ID: <5E2843FF4724492FBF2A29E43B698385@multiplay.co.uk> From: "Steven Hartland" To: "Hiroki Sato" References: <20130609.041903.541782050134731313.hrs@allbsd.org> Subject: Re: Feeback on patch to add ZFS support to mdconfig rc.d scripts Date: Sun, 9 Jun 2013 19:13:31 +0100 MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="----=_NextPart_000_0570_01CE6545.6E1C99C0" X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.5931 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.6157 Cc: rc@FreeBSD.org, zfs-devel@FreeBSD.org X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 09 Jun 2013 18:13:37 -0000 This is a multi-part message in MIME format. ------=_NextPart_000_0570_01CE6545.6E1C99C0 Content-Type: text/plain; format=flowed; charset="iso-8859-1"; reply-type=original Content-Transfer-Encoding: 7bit ----- Original Message ----- From: "Hiroki Sato" > I think this should be separated into adding non-fs md support into > rc.d/mdconfig and on-the-fly zpool creation/import into rc.d/zfs > since the rc.d/mdconfig script does not (and should not) depend on > zfs.ko module. > > rc.d/mdconfig is located before mountcritlocal, so probably > rc.d/zfs_md or something is needed just after rc.d/mdconfig2. Thanks for that Hiroki. If I understand correctly that your only concern is the zfs module dependency? If so then its easier to make that a dynamic dependency using a prestart. Updated patch attached, which does just that. Does this look better? Regards Steve ================================================ This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmaster@multiplay.co.uk. ------=_NextPart_000_0570_01CE6545.6E1C99C0 Content-Type: application/octet-stream; name="mdconfig-zfs.patch" Content-Transfer-Encoding: quoted-printable Content-Disposition: attachment; filename="mdconfig-zfs.patch" Add ZFS pool creation support to mdconfig rc.d script=0A= =0A= Fix failure error redirection in mount check=0A= =0A= Fix invalid configuration detection enabling -t vnode=0A= to be skipped if -f is specified as supported=0A= by mdconfig.=0A= =0A= Fix fsck not working if mount point is not present in fstab.=0A= =0A= TODO: update rc.conf man page=0A= --- /etc/rc.d/mdconfig.orig 2013-06-08 00:55:48.599761883 +0000=0A= +++ /etc/rc.d/mdconfig 2013-06-09 18:02:08.037511687 +0000=0A= @@ -35,9 +35,28 @@=0A= name=3D"mdconfig"=0A= stop_cmd=3D"mdconfig_stop"=0A= start_cmd=3D"mdconfig_start"=0A= -start_precmd=3D'[ -n "${_mdconfig_list}" ]'=0A= +start_precmd=3D"mdconfig_prestart"=0A= required_modules=3D"geom_md:g_md"=0A= =0A= +mdconfig_prestart()=0A= +{=0A= + local _mp=0A= +=0A= + if [ -z "${_mdconfig_list}" ]; then=0A= + return 1=0A= + fi=0A= +=0A= + for _md in ${_mdconfig_list}; do=0A= + eval _zpool=3D\$mdconfig_${_md}_zfs_pool=0A= + if [ -n "${_zpool}" ]; then=0A= + required_modules=3D"$required_modules zfs"=0A= + break=0A= + fi=0A= + done=0A= +=0A= + return 0=0A= +}=0A= +=0A= is_readonly()=0A= {=0A= local _mp _ret=0A= @@ -61,6 +80,29 @@=0A= fi=0A= }=0A= =0A= +get_opt()=0A= +{=0A= + local _flag _param _opt=0A= +=0A= + _flag=3D$1=0A= + _param=3D$2=0A= +=0A= + _opt=3D${_config##*-${_flag}\ }=0A= + if [ "${_opt}" =3D "${_config}" ]; then=0A= + _opt=3D""=0A= + else=0A= + _opt=3D${_opt%%\ *}=0A= + fi=0A= + eval _${_param}=3D${_opt}=0A= + debug "${_md} ${_param}=3D${_opt}"=0A= +=0A= + if [ -z "${_opt}" ]; then=0A= + return 0=0A= + fi=0A= +=0A= + return 1=0A= +}=0A= +=0A= init_variables()=0A= {=0A= local _i=0A= @@ -70,16 +112,20 @@=0A= _dev=3D"/dev/${_md}"=0A= eval _config=3D\$mdconfig_${_md}=0A= eval _newfs=3D\$mdconfig_${_md}_newfs=0A= + eval _zpool=3D\$mdconfig_${_md}_zfs_pool=0A= + eval _zflags=3D\$mdconfig_${_md}_zfs_flags=0A= =0A= - _type=3D${_config##*-t\ }=0A= - _type=3D${_type%%\ *}=0A= + get_opt "t" "type"=0A= + get_opt "f" "file"=0A= if [ -z "${_type}" ]; then=0A= - err 1 "You need to specify \"-t \" in mdconfig_${_md}"=0A= + if [ -n "${_file}" ]; then=0A= + _type=3D"vnode"=0A= + else=0A= + err 1 "You need to specify \"-t \" in mdconfig_${_md}"=0A= + fi=0A= fi=0A= =0A= if [ "${_type}" =3D "vnode" ]; then=0A= - _file=3D${_config##*-f\ }=0A= - _file=3D${_file%%\ *}=0A= if [ -z "${_file}" ]; then=0A= err 2 "You need to specify \"-f \" in mdconfig_${_md} for = vnode devices"=0A= fi=0A= @@ -96,11 +142,13 @@=0A= debug "${_md} file: ${_file}"=0A= debug "${_md} fs: ${_fs}"=0A= debug "${_md} newfs flags: ${_newfs}"=0A= + debug "${_md} ZFS pool: ${_zpool}"=0A= + debug "${_md} ZFS flags: ${_zflags}"=0A= }=0A= =0A= mdconfig_start()=0A= {=0A= - local _md _mp _config _type _dev _file _fs _newfs _fsck_cmd=0A= + local _md _mp _config _type _dev _file _fs _newfs _fsck_cmd _zpool = _zflags=0A= =0A= for _md in ${_mdconfig_list}; do=0A= init_variables ${_md}=0A= @@ -126,14 +174,26 @@=0A= echo "Creating ${_md} device failed, moving on."=0A= continue=0A= fi=0A= +=0A= + if [ -n "${_zpool}" ]; then=0A= + if [ "${_type}" =3D "vnode" ]; then=0A= + echo "Importing ZFS pool ${_zpool}."=0A= + zpool import ${_zpool}=0A= + else=0A= + echo "Creating ZFS pool ${_zpool}."=0A= + zpool create ${_zflags} ${_zpool} ${_md}=0A= + fi=0A= + continue=0A= + fi=0A= +=0A= # Skip fsck for uzip devices.=0A= if [ "${_type}" =3D "vnode" ]; then=0A= if [ "${_file}" !=3D "${_file%.uzip}" ]; then=0A= _fsck_cmd=3D":"=0A= elif checkyesno background_fsck; then=0A= - _fsck_cmd=3D"fsck -F"=0A= + _fsck_cmd=3D"fsck -t ufs -F"=0A= else=0A= - _fsck_cmd=3D"fsck"=0A= + _fsck_cmd=3D"fsck -t ufs"=0A= fi=0A= if ! eval ${_fsck_cmd} -p ${_dev} >/dev/null; then=0A= echo "Fsck failed on ${_dev}, not mounting the filesystem."=0A= @@ -143,7 +203,8 @@=0A= else=0A= newfs ${_newfs} ${_dev} >/dev/null=0A= fi=0A= - if mount -d ${_dev} 2>&1 >/dev/null; then=0A= +=0A= + if mount -d ${_dev} >/dev/null 2>&1; then=0A= echo "Mounting ${_dev}."=0A= mount ${_dev}=0A= fi=0A= @@ -159,7 +220,12 @@=0A= init_variables ${_md}=0A= if [ "${_type}" !=3D "vnode" -o "${_fs}" =3D "/" ]; then=0A= for _i in `df ${_dev} 2>/dev/null`; do _mp=3D${_i}; done=0A= - if [ -z "${_mp}" -o "${_mp}" !=3D "${_mp%%%}" ]; then=0A= + if [ -n "${_zpool}" ]; then=0A= + if zpool list ${_zpool} >/dev/null 2>&1; then=0A= + echo "Exporting ZFS pool ${_zpool}."=0A= + zpool export ${_zpool}=0A= + fi=0A= + elif [ -z "${_mp}" -o "${_mp}" !=3D "${_mp%%%}" ]; then=0A= echo "Device ${_dev} isn't mounted."=0A= else=0A= echo "Umounting ${_dev}."=0A= --- /etc/rc.d/mdconfig2.orig 2013-06-08 16:33:49.583875813 +0000=0A= +++ /etc/rc.d/mdconfig2 2013-06-09 18:03:39.055114045 +0000=0A= @@ -36,9 +36,28 @@=0A= name=3D"mdconfig2"=0A= stop_cmd=3D"mdconfig2_stop"=0A= start_cmd=3D"mdconfig2_start"=0A= -start_precmd=3D'[ -n "${_mdconfig2_list}" ]'=0A= +start_precmd=3D"mdconfig2_prestart"=0A= required_modules=3D"geom_md:g_md"=0A= =0A= +mdconfig2_prestart()=0A= +{=0A= + local _mp=0A= +=0A= + if [ -z "${_mdconfig2_list}" ]; then=0A= + return 1=0A= + fi=0A= +=0A= + for _md in ${_mdconfig2_list}; do=0A= + eval _zpool=3D\$mdconfig_${_md}_zfs_pool=0A= + if [ -n "${_zpool}" ]; then=0A= + required_modules=3D"$required_modules zfs"=0A= + break=0A= + fi=0A= + done=0A= +=0A= + return 0=0A= +}=0A= +=0A= is_readonly()=0A= {=0A= local _mp _ret=0A= @@ -62,6 +81,29 @@=0A= fi=0A= }=0A= =0A= +get_opt()=0A= +{=0A= + local _flag _param _opt=0A= +=0A= + _flag=3D$1=0A= + _param=3D$2=0A= +=0A= + _opt=3D${_config##*-${_flag}\ }=0A= + if [ "${_opt}" =3D "${_config}" ]; then=0A= + _opt=3D""=0A= + else=0A= + _opt=3D${_opt%%\ *}=0A= + fi=0A= + eval _${_param}=3D${_opt}=0A= + debug "${_md} ${_param}=3D${_opt}"=0A= +=0A= + if [ -z "${_opt}" ]; then=0A= + return 0=0A= + fi=0A= +=0A= + return 1=0A= +}=0A= +=0A= init_variables()=0A= {=0A= local _i=0A= @@ -75,16 +117,19 @@=0A= eval _perms=3D\$mdconfig_${_md}_perms=0A= eval _files=3D\$mdconfig_${_md}_files=0A= eval _populate=3D\$mdconfig_${_md}_cmd=0A= + eval _zpool=3D\$mdconfig_${_md}_zfs_pool=0A= =0A= - _type=3D${_config##*-t\ }=0A= - _type=3D${_type%%\ *}=0A= + get_opt "t" "type"=0A= + get_opt "f" "file"=0A= if [ -z "${_type}" ]; then=0A= - err 1 "You need to specify \"-t \" in mdconfig_${_md}"=0A= + if [ -n "${_file}" ]; then=0A= + _type=3D"vnode"=0A= + else=0A= + err 1 "You need to specify \"-t \" in mdconfig_${_md}"=0A= + fi=0A= fi=0A= =0A= if [ "${_type}" =3D "vnode" ]; then=0A= - _file=3D${_config##*-f\ }=0A= - _file=3D${_file%%\ *}=0A= if [ -z "${_file}" ]; then=0A= err 2 "You need to specify \"-f \" in mdconfig_${_md} for = vnode devices"=0A= fi=0A= @@ -136,26 +181,42 @@=0A= echo "Creating ${_md} device failed, moving on."=0A= continue=0A= fi=0A= - # Skip fsck for uzip devices.=0A= - if [ "${_file}" !=3D "${_file%.uzip}" ]; then=0A= - _fsck_cmd=3D":"=0A= - elif checkyesno background_fsck; then=0A= - _fsck_cmd=3D"fsck -F"=0A= +=0A= + if [ -n "${_zpool}" ]; then=0A= + if [ "${_type}" =3D "vnode" ]; then=0A= + echo "Importing ZFS pool ${_zpool}."=0A= + zpool import ${_zpool}=0A= + else=0A= + echo "Creating ZFS pool ${_zpool}."=0A= + zpool create ${_zflags} ${_zpool} ${_md}=0A= + fi=0A= else=0A= - _fsck_cmd=3D"fsck"=0A= - fi=0A= - if ! eval ${_fsck_cmd} -p ${_dev} >/dev/null; then=0A= - echo "Fsck failed on ${_dev}, not mounting the filesystem."=0A= - continue=0A= - fi=0A= - if mount -d ${_dev} >/dev/null 2>&1; then=0A= - echo "Mounting ${_dev}."=0A= - mount ${_dev}=0A= + # Skip fsck for uzip devices.=0A= + if [ "${_file}" !=3D "${_file%.uzip}" ]; then=0A= + _fsck_cmd=3D":"=0A= + elif checkyesno background_fsck; then=0A= + _fsck_cmd=3D"fsck -t ufs -F"=0A= + else=0A= + _fsck_cmd=3D"fsck -t ufs"=0A= + fi=0A= + if ! eval ${_fsck_cmd} -p ${_dev} >/dev/null; then=0A= + echo "Fsck failed on ${_dev}, not mounting the filesystem."=0A= + continue=0A= + fi=0A= + if mount -d ${_dev} >/dev/null 2>&1; then=0A= + echo "Mounting ${_dev}."=0A= + mount ${_dev}=0A= + fi=0A= fi=0A= fi=0A= =0A= - for _i in `df ${_dev} 2>/dev/null`; do _mp=3D${_i}; done=0A= - if [ ! -z "${_mp}" -a "${_mp}" =3D "${_mp%%%}" ]; then=0A= + if [ -n "${_zpool}" ]; then=0A= + _mp=3D`zfs list -H -o mountpoint ${_zpool} 2>/dev/null`=0A= + else=0A= + for _i in `df ${_dev} 2>/dev/null`; do _mp=3D${_i}; done=0A= + fi=0A= +=0A= + if [ -n "${_mp}" -a "${_mp}" =3D "${_mp%%%}" ]; then=0A= _mounted=3D"yes"=0A= fi=0A= =0A= ------=_NextPart_000_0570_01CE6545.6E1C99C0-- From owner-zfs-devel@FreeBSD.ORG Mon Jun 10 09:26:19 2013 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 0C2E4958; Mon, 10 Jun 2013 09:26:19 +0000 (UTC) (envelope-from hrs@FreeBSD.org) Received: from mail.allbsd.org (gatekeeper.allbsd.org [IPv6:2001:2f0:104:e001::32]) by mx1.freebsd.org (Postfix) with ESMTP id 4020116CC; Mon, 10 Jun 2013 09:26:18 +0000 (UTC) Received: from alph.d.allbsd.org (p2175-ipbf701funabasi.chiba.ocn.ne.jp [122.25.209.175]) (authenticated bits=128) by mail.allbsd.org (8.14.5/8.14.5) with ESMTP id r5A9Q0kw021751 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 10 Jun 2013 18:26:10 +0900 (JST) (envelope-from hrs@FreeBSD.org) Received: from localhost (localhost [127.0.0.1]) (authenticated bits=0) by alph.d.allbsd.org (8.14.5/8.14.5) with ESMTP id r5A9PveY087034; Mon, 10 Jun 2013 18:25:59 +0900 (JST) (envelope-from hrs@FreeBSD.org) Date: Mon, 10 Jun 2013 18:25:18 +0900 (JST) Message-Id: <20130610.182518.138879193135244843.hrs@allbsd.org> To: killing@multiplay.co.uk Subject: Re: Feeback on patch to add ZFS support to mdconfig rc.d scripts From: Hiroki Sato In-Reply-To: <5E2843FF4724492FBF2A29E43B698385@multiplay.co.uk> References: <20130609.041903.541782050134731313.hrs@allbsd.org> <5E2843FF4724492FBF2A29E43B698385@multiplay.co.uk> X-PGPkey-fingerprint: BDB3 443F A5DD B3D0 A530 FFD7 4F2C D3D8 2793 CF2D X-Mailer: Mew version 6.5 on Emacs 24.3 / Mule 6.0 (HANACHIRUSATO) Mime-Version: 1.0 Content-Type: Multipart/Signed; protocol="application/pgp-signature"; micalg=pgp-sha1; boundary="--Security_Multipart0(Mon_Jun_10_18_25_18_2013_749)--" Content-Transfer-Encoding: 7bit X-Virus-Scanned: clamav-milter 0.97.4 at gatekeeper.allbsd.org X-Virus-Status: Clean X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (mail.allbsd.org [133.31.130.32]); Mon, 10 Jun 2013 18:26:11 +0900 (JST) X-Spam-Status: No, score=-93.8 required=13.0 tests=CONTENT_TYPE_PRESENT, FAKEDWORD_BACKQUOTE,ONLY1HOPDIRECT,QENCPTR1,RCVD_IN_PBL,SAMEHELOBY2HOP, USER_IN_WHITELIST autolearn=no version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on gatekeeper.allbsd.org Cc: rc@FreeBSD.org, zfs-devel@FreeBSD.org X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 10 Jun 2013 09:26:19 -0000 ----Security_Multipart0(Mon_Jun_10_18_25_18_2013_749)-- Content-Type: Multipart/Mixed; boundary="--Next_Part(Mon_Jun_10_18_25_18_2013_246)--" Content-Transfer-Encoding: 7bit ----Next_Part(Mon_Jun_10_18_25_18_2013_246)-- Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit "Steven Hartland" wrote in <5E2843FF4724492FBF2A29E43B698385@multiplay.co.uk>: ki> ----- Original Message ----- ki> From: "Hiroki Sato" ki> > I think this should be separated into adding non-fs md support into ki> > rc.d/mdconfig and on-the-fly zpool creation/import into rc.d/zfs ki> > since the rc.d/mdconfig script does not (and should not) depend on ki> > zfs.ko module. ki> > ki> > rc.d/mdconfig is located before mountcritlocal, so probably ki> > rc.d/zfs_md or something is needed just after rc.d/mdconfig2. ki> ki> Thanks for that Hiroki. If I understand correctly that your only ki> concern is the zfs module dependency? If so then its easier to make ki> that a dynamic dependency using a prestart. Not only that. What I am concerned about is adding ZFS support into rc.d/mdconfig makes mount sequence in rc.d script more complicated. Currently, /etc/rc.d/mdconfig runs, /etc/fstab is parsed, and then "zfs mount -a" is done in rc.d/zfs. The rc.d scripts is not good at handling dependency, so mounting/unmounting ZFS datasets in multiple scripts make difficult to use these scripts separetely at run time while probably they work fine at boot time. If we support md-backed zpool, "rc.d/zfs stop" should also handle unmounting all of ZFS datasets and destroy the md devices, for example, because it is responsible for stopping the ZFS and its related functionality. This is difficult to realize by using only rc.d scripts, I created and attached a prototype to add ZFS support directly to mount_mfs(8). Does this satisfy your needs? Just adding /etc/fstab entry like the following, md-backed zpools are automatically created and mounted via rc.d/mountcritlocal. If it is too early, "late" flag can be used as other file systems. Unmounting them is handled only by rc.d/zfs in this case. The mount options in an fstab entry look a bit odd because most of letters are already used in mount_mfs(8), but they are flexible to support zpool's command-line arguments. ---- # /etc/fstab examples: # # create swap-backed md and zpool "poola" on it, and then mount it on /a. md /a mfs rw,-s1g,-zpoola,-kcompression:on,-kexec:off 0 0 # # create vnode-backed md and zpool "poolb" on it, and then mount it on /b. md /b mfs rw,-s1g,-zpoolb,-F/var/tmp/zero 0 0 # # import (-P flag) a zpool "zpoolc" on vnode-backed md "/var/tmp/zero". # Mountpoint is automatically determined (/c is ignored). md /c mfs rw,-s1g,-zpoolc,-F/var/tmp/zero,-P 0 0 # # import a zpool "zpoold" on vnode-backed md at rc.d/mountlate stage. # Mountpoint is automatically determined (/d is ignored). md /d mfs rw,-s1g,-zpoold,-F/var/tmp/zero,-P,late 0 0 ---- The following command line should work with the above example: # mount /a # sh /etc/rc.d/zfs stop -- Hiroki ----Next_Part(Mon_Jun_10_18_25_18_2013_246)-- Content-Type: Text/X-Patch; charset=us-ascii Content-Transfer-Encoding: 7bit Content-Disposition: inline; filename="mdmfs_zfs_support.20130610-1.diff" Index: etc/rc.d/zfs =================================================================== --- etc/rc.d/zfs (revision 251176) +++ etc/rc.d/zfs (working copy) @@ -48,6 +48,26 @@ zfs_stop_main() { + # Export md-backed zpools and destroy the md devices if any + while read dev mt fstype opt; do + case $dev:$opt in + \#*) # skip comment line + ;; + md:*,-F/*,-z*|md:*,-z*,-F/*) + opt=${opt##*,-z} + poolname=${opt%%,*} + # Check if $poolname exists. + df -t zfs $poolname > /dev/null 2>&1 || continue + set -- `zpool list -Hv $poolname` + if zpool export $poolname && mdconfig -d -u $9; then + : + else + warn "Cannot unmount zpool $poolname." + fi + ;; + esac + done < /etc/fstab + zfs unshare -a zfs unmount -a } Index: sbin/mdmfs/mdmfs.8 =================================================================== --- sbin/mdmfs/mdmfs.8 (revision 251176) +++ sbin/mdmfs/mdmfs.8 (working copy) @@ -25,7 +25,7 @@ .\" .\" $FreeBSD$ .\" -.Dd September 4, 2011 +.Dd June 10, 2013 .Dt MDMFS 8 .Os .Sh NAME @@ -46,6 +46,7 @@ .Op Fl F Ar file .Op Fl f Ar frag-size .Op Fl i Ar bytes +.Op Fl k Ar zfs-filesystem-property:value .Op Fl m Ar percent-free .Op Fl n Ar rotational-positions .Op Fl O Ar optimization @@ -54,6 +55,8 @@ .Op Fl s Ar size .Op Fl v Ar version .Op Fl w Ar user : Ns Ar group +.Op Fl z Ar zfs-zpoolname +.Op Fl Z Ar zfs-property:value .Ar md-device .Ar mount-point .Sh DESCRIPTION @@ -263,6 +266,36 @@ .It Fl X Print what command will be run before running it, and other assorted debugging information. +.It Fl z Ar zfs-zpoolname +Specify to use ZFS and the pool name as +.Ar zfs-zpoolname . +When this is specified, +parameters for +.Xr newfs 8 +are ignored except for a +.Fl P +flag. +.It Fl Z Ar zfs-property:value +Specify zpool property. +This will be passed to +.Xr zpool 8 +utility as +.Fl o Ar zfs-property=value . +The default value is +.Fl Z Ar cechefile:none . +.It Fl k Ar zfs-filesystem-property:value +Specify zfs property. +This will be passed to +.Xr zpool 8 +utility as +.Fl O Ar zfs-filesystem-property=value . +The default values are +.Fl k Ar atime:off , +.Fl k Ar primarycache:none , +and +.Fl k Ar sync:disabled +for swap-backed disk, +and none for vnode-backed disk. .El .Pp The @@ -289,6 +322,13 @@ .Xr newfs 8 as .Fl o . +The values in +.Fl z , +.Fl Z , +and +.Fl k +options are passed to +.Xr zpool 8 . The .Fl o option is passed to Index: sbin/mdmfs/mdmfs.c =================================================================== --- sbin/mdmfs/mdmfs.c (revision 251176) +++ sbin/mdmfs/mdmfs.c (working copy) @@ -81,6 +81,8 @@ static void do_mount(const char *, const char *); static void do_mtptsetup(const char *, struct mtpt_info *); static void do_newfs(const char *); +static void do_zpool(const char *, const char *, const char *, + const enum md_types, int); static void extract_ugid(const char *, struct mtpt_info *); static int run(int *, const char *, ...) __printflike(2, 3); static void usage(void); @@ -90,11 +92,11 @@ { struct mtpt_info mi; /* Mountpoint info. */ char *mdconfig_arg, *newfs_arg, /* Args to helper programs. */ - *mount_arg; + *mount_arg, *zpool_arg; enum md_types mdtype; /* The type of our memory disk. */ bool have_mdtype; - bool detach, softdep, autounit, newfs; - char *mtpoint, *unitstr; + bool detach, softdep, autounit, newfs, zfs; + char *mtpoint, *unitstr, *poolname; char *p; int ch; void *set; @@ -106,6 +108,7 @@ softdep = true; autounit = false; newfs = true; + zfs = false; have_mdtype = false; mdtype = MD_SWAP; mdname = MD_NAME; @@ -118,6 +121,7 @@ */ mdconfig_arg = strdup(""); newfs_arg = strdup(""); + zpool_arg = strdup(""); mount_arg = strdup(""); /* If we were started as mount_mfs or mfs, imply -C. */ @@ -129,7 +133,7 @@ } while ((ch = getopt(argc, argv, - "a:b:Cc:Dd:E:e:F:f:hi:LlMm:NnO:o:Pp:Ss:tUv:w:X")) != -1) + "a:B:b:Cc:Dd:E:e:F:f:hi:k:LlMm:NnO:o:Pp:Ss:tUv:w:Xz:Z:")) != -1) switch (ch) { case 'a': argappend(&newfs_arg, "-a %s", optarg); @@ -171,6 +175,14 @@ case 'i': argappend(&newfs_arg, "-i %s", optarg); break; + case 'k': + if (zfs != true) + errx(1, "-z poolname required first"); + p = optarg; + while ((p = strchr(p, ':')) != NULL) + p[0] = (char)'='; + argappend(&zpool_arg, "-O %s", optarg); + break; case 'L': loudsubs = true; break; @@ -231,6 +243,18 @@ case 'X': debug = true; break; + case 'z': + poolname = strdup(optarg); + zfs = true; + break; + case 'Z': + if (zfs != true) + errx(1, "-z poolname required first"); + p = optarg; + while ((p = strchr(p, ':')) != NULL) + p[0] = (char)'='; + argappend(&zpool_arg, "-o %s", optarg); + break; default: usage(); } @@ -272,9 +296,13 @@ do_mdconfig_attach_au(mdconfig_arg, mdtype); else do_mdconfig_attach(mdconfig_arg, mdtype); - if (newfs) - do_newfs(newfs_arg); - do_mount(mount_arg, mtpoint); + if (zfs) + do_zpool(zpool_arg, poolname, mtpoint, mdtype, newfs); + else { + if (newfs) + do_newfs(newfs_arg); + do_mount(mount_arg, mtpoint); + } do_mtptsetup(mtpoint, &mi); return (0); @@ -514,6 +542,63 @@ } /* + * Put a zpool on the memory disk. When vnode is specified, try to import + * zpool. + */ +static void +do_zpool(const char *args, const char *poolname, const char *mtpoint, + const enum md_types mdtype, int newfs) +{ + const char *cmd; + const char *addargs; + char mtarg[PATH_MAX + 10]; + char mdarg[PATH_MAX + 10]; + int rv; + + switch (mdtype) { + case MD_SWAP: + case MD_MALLOC: + cmd = "create"; + /* A set of reasonable default parameters. */ + addargs = " -o cachefile=none " + "-O primarycache=none " + "-O sync=disabled "; + snprintf(mtarg, sizeof(mtarg), " -m %s" , mtpoint); + mtarg[sizeof(mtarg) - 1] = '\0'; + snprintf(mdarg, sizeof(mdarg), " /dev/%s%d" , mdname, unit); + mdarg[sizeof(mdarg) - 1] = '\0'; + break; + case MD_VNODE: + /* Ignore return code. */ + run(NULL, "%s export %s", _PATH_ZPOOL, poolname); + + if (newfs) { + cmd = "create"; + /* Override the existing pool. */ + addargs = " -f -o cachefile=none"; + snprintf(mtarg, sizeof(mtarg), " -m %s" , mtpoint); + mtarg[sizeof(mtarg) - 1] = '\0'; + snprintf(mdarg, sizeof(mdarg), " /dev/%s%d" , mdname, + unit); + mdarg[sizeof(mdarg) - 1] = '\0'; + } else { + cmd = "import"; + addargs = " -o cachefile=none"; + mtarg[0] = '\0'; + mdarg[0] = '\0'; + } + break; + default: + abort(); + } + + rv = run(NULL, "%s %s%s%s%s %s%s", + _PATH_ZPOOL, cmd, addargs, args, mtarg, poolname, mdarg); + if (rv) + errx(1, "zpool exited with error code %d", rv); +} + +/* * 'str' should be a user and group name similar to the last argument * to chown(1); i.e., a user, followed by a colon, followed by a * group. The user and group in 'str' may be either a [ug]id or a Index: include/paths.h =================================================================== --- include/paths.h (revision 251176) +++ include/paths.h (working copy) @@ -88,6 +88,7 @@ #define _PATH_UFSSUSPEND "/dev/ufssuspend" #define _PATH_VI "/usr/bin/vi" #define _PATH_WALL "/usr/bin/wall" +#define _PATH_ZPOOL "/sbin/zpool" /* Provide trailing slash, since mostly used for building pathnames. */ #define _PATH_DEV "/dev/" ----Next_Part(Mon_Jun_10_18_25_18_2013_246)---- ----Security_Multipart0(Mon_Jun_10_18_25_18_2013_749)-- Content-Type: application/pgp-signature Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.13 (FreeBSD) iEYEABECAAYFAlG1m34ACgkQTyzT2CeTzy0OKgCfRLFGzjAuf/MPKmBSBzZILRd1 GQQAn1vs/zlIGHU5zZLVqzB+HO3WaRBx =26qm -----END PGP SIGNATURE----- ----Security_Multipart0(Mon_Jun_10_18_25_18_2013_749)---- From owner-zfs-devel@FreeBSD.ORG Mon Jun 10 11:08:28 2013 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 99CC955D for ; Mon, 10 Jun 2013 11:08:28 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) by mx1.freebsd.org (Postfix) with ESMTP id 722C31DD1 for ; Mon, 10 Jun 2013 11:08:28 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.7/8.14.7) with ESMTP id r5AB8SiG099027 for ; Mon, 10 Jun 2013 11:08:28 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.7/8.14.7/Submit) id r5AB8S5d099025 for zfs-devel@FreeBSD.org; Mon, 10 Jun 2013 11:08:28 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 10 Jun 2013 11:08:28 GMT Message-Id: <201306101108.r5AB8S5d099025@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: zfs-devel@FreeBSD.org Subject: Current problem reports assigned to zfs-devel@FreeBSD.org X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 10 Jun 2013 11:08:28 -0000 Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- s kern/178467 zfs-devel [zfs] [request] Optimized Checksum Code for ZFS o kern/178387 zfs-devel [zfs] [patch] sparse files performance improvements 2 problems total. From owner-zfs-devel@FreeBSD.ORG Mon Jun 17 11:08:22 2013 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 422DA6A7 for ; Mon, 17 Jun 2013 11:08:22 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) by mx1.freebsd.org (Postfix) with ESMTP id 1A3951D44 for ; Mon, 17 Jun 2013 11:08:22 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.7/8.14.7) with ESMTP id r5HB8LED014796 for ; Mon, 17 Jun 2013 11:08:21 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.7/8.14.7/Submit) id r5HB8LHW014794 for zfs-devel@FreeBSD.org; Mon, 17 Jun 2013 11:08:21 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 17 Jun 2013 11:08:21 GMT Message-Id: <201306171108.r5HB8LHW014794@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: zfs-devel@FreeBSD.org Subject: Current problem reports assigned to zfs-devel@FreeBSD.org X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Jun 2013 11:08:22 -0000 Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- s kern/178467 zfs-devel [zfs] [request] Optimized Checksum Code for ZFS o kern/178387 zfs-devel [zfs] [patch] sparse files performance improvements 2 problems total. From owner-zfs-devel@FreeBSD.ORG Tue Jun 18 11:16:32 2013 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 929368F7 for ; Tue, 18 Jun 2013 11:16:32 +0000 (UTC) (envelope-from prvs=188190cd5e=killing@multiplay.co.uk) Received: from mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) by mx1.freebsd.org (Postfix) with ESMTP id 394D61028 for ; Tue, 18 Jun 2013 11:16:32 +0000 (UTC) Received: from r2d2 ([82.69.141.170]) by mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) (MDaemon PRO v10.0.4) with ESMTP id md50004407150.msg for ; Tue, 18 Jun 2013 12:16:25 +0100 X-Spam-Processed: mail1.multiplay.co.uk, Tue, 18 Jun 2013 12:16:25 +0100 (not processed: message from valid local sender) X-MDDKIM-Result: neutral (mail1.multiplay.co.uk) X-MDRemoteIP: 82.69.141.170 X-Return-Path: prvs=188190cd5e=killing@multiplay.co.uk X-Envelope-From: killing@multiplay.co.uk X-MDaemon-Deliver-To: zfs-devel@FreeBSD.org Message-ID: <2434B71D392A46698EB970AA67699A6C@multiplay.co.uk> From: "Steven Hartland" To: , Subject: ZFS registering ENOSPC as pool write errors Date: Tue, 18 Jun 2013 12:16:32 +0100 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="iso-8859-1"; reply-type=original Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.5931 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.6157 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Jun 2013 11:16:32 -0000 We've been testing using a swap backed md for temporary storage via a zfs pool. This was working well until an application error caused the pool to fill. This shouldn't have been an issue but it appears that it caused write errors to be registered against the pool apparently for error 28 (ENOSPC). Whats unusual is that the only backing device md0 shows 0 errors. zpool status -x output: pool: ramdisk state: ONLINE status: One or more devices are faulted in response to IO failures. action: Make sure the affected devices are connected, then run 'zpool clear'. see: http://www.sun.com/msg/ZFS-8000-HC scrub: none requested config: NAME STATE READ WRITE CKSUM ramdisk ONLINE 0 11.1K 0 md0 ONLINE 0 0 0 errors: 1393 data errors, use '-v' for a list /var/log/messages showed:- Jun 12 10:46:48 node9 root: ZFS: zpool I/O failure, zpool=ramdisk error=28 Jun 12 10:46:49 node9 last message repeated 999 times Jun 12 10:46:49 node9 root: ZFS: vdev I/O failure, zpool=ramdisk path= offset= size= error= These are logged via devd triggering the following:- notify 10 { match "system" "ZFS"; match "type" "data"; action "logger -p kern.warn 'ZFS: zpool I/O failure, zpool=$pool error=$zio_err'"; }; notify 10 { match "system" "ZFS"; match "type" "io"; action "logger -p kern.warn 'ZFS: vdev I/O failure, zpool=$pool path=$vdev_path offset=$zio_offset size=$zio_size error=$zio_err'"; }; The first of these (type=data) corrisponds to FM_EREPORT_ZFS_DATA and the second (type=io) corrisponds to FM_EREPORT_ZFS_IO both of which are reported via zio_done. I've tried to reproduce this by filling an identically configured pool, deleting some and filling again but have had no joy so far, so it seems like its a fairly rare edge case. So the questions:- 1. I'm assuming not, but should ENOSPC ever get registered as a pool write error? 2. Has anyone ever seen similar behavour before? Unfortunately I was away when this happened and as it was a production machine the pool was destroyed before I could do any further diagnosis. The machine in question is running FreeBSD 8.3 so a little old in terms of ZFS code base but wanted to get general feedback from illumos and FreeBSD given it seems like an edge case so likely not that common in practice. Regards Steve ================================================ This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmaster@multiplay.co.uk. From owner-zfs-devel@FreeBSD.ORG Tue Jun 18 16:41:38 2013 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id D588F6EF for ; Tue, 18 Jun 2013 16:41:38 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-qa0-x22b.google.com (mail-qa0-x22b.google.com [IPv6:2607:f8b0:400d:c00::22b]) by mx1.freebsd.org (Postfix) with ESMTP id 9BFA11713 for ; Tue, 18 Jun 2013 16:41:38 +0000 (UTC) Received: by mail-qa0-f43.google.com with SMTP id d13so2339108qak.9 for ; Tue, 18 Jun 2013 09:41:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:date:x-google-sender-auth:message-id:subject :from:to:content-type; bh=BedRbEtcxN690xZD+tDPoPKXDmoUaRIfGz5BrKJ+ViM=; b=NPTZe96KGSO0bm+yd46hqktSNWAfSdFp7v3BXQ2KuIXaSg5yV4Uzw+Bt28q83/dP33 +p2yfLqcGU+cWX3MLQxiLTcL7sI4LwOqcpA/RdPJ4APDy+7svpuNiq0LGkTTY8c6vwh5 C5GId8c70UjkqOOeFLZ2t1jim8E1opepYaXtQI9x0rC3buap5LZPBqZJM9hV8YHRXUZc stsVOkxiLS1DNPef0NU5JzTHM6k85nQ8g04YTAV4d/J6/5mRj+u3VSA027VqySP99Oc1 QKMY75lTU8tf0yueS+JJjYF+tectgpVVCxbc1ntT+/jQiRDAQmKFs/Fl3s2Y0mSkr41j THYA== MIME-Version: 1.0 X-Received: by 10.224.57.82 with SMTP id b18mr24266687qah.36.1371573698141; Tue, 18 Jun 2013 09:41:38 -0700 (PDT) Sender: asomers@gmail.com Received: by 10.49.37.226 with HTTP; Tue, 18 Jun 2013 09:41:38 -0700 (PDT) Date: Tue, 18 Jun 2013 10:41:38 -0600 X-Google-Sender-Auth: tm2_kJbV1Ixn8UVpV9dYnRfgU4A Message-ID: Subject: STF ZFS test suite, part 2 From: Alan Somers To: zfs-devel@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Jun 2013 16:41:38 -0000 A long time ago, HighCloud security ported OpenSolaris's ZFS Test suite to FreeBSD. Spectra Logic improved it, and last September gibbs posted a snapshot of our work to this list. Here is an updated snapshot that contains an additional 9 months' worth of work. The most significant update is that the tests have been adapted to run under the ATF tools instead of the STF tools. It also contains a few contributions from araujo. The README file contains instructions on how to build and run the tests, but let me summarize the first two rules of the ZFS test suite: 1) You do not run it on a production system. 2) You do not run it on a production system. http://people.freebsd.org/~asomers/stf_zfs/ -Alan From owner-zfs-devel@FreeBSD.ORG Tue Jun 18 16:56:22 2013 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 2D00624A; Tue, 18 Jun 2013 16:56:22 +0000 (UTC) (envelope-from mattjeet@gmail.com) Received: from mail-wi0-x236.google.com (mail-wi0-x236.google.com [IPv6:2a00:1450:400c:c05::236]) by mx1.freebsd.org (Postfix) with ESMTP id 8BDBB17CE; Tue, 18 Jun 2013 16:56:21 +0000 (UTC) Received: by mail-wi0-f182.google.com with SMTP id m6so3445076wiv.3 for ; Tue, 18 Jun 2013 09:56:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:reply-to:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; bh=tOTnJHSA6+BPNQRM57M4N+u20GmrpaXUeIzhyITMi1s=; b=iWNXInGdq6OEwb1/PlvwYu500/to+Kiepwcr7GoGA01qYro1CWOW1WwPF5JWOHNnmJ l91D1lxMcf6+onM0Y6w9I8KgLKuLtlRzO5kWETb4Mb5k01yiG8FKQnAu+aYVo3AhQvac 41GvgAHXySDzmS0z50Q9V2zmseqKzDhCGVosLO5gPgHfCyAQhvndiIo0Qu3WNNyy5ZSB hcoShAyHO5pDfYE+boIaWKXxmje9KZn0EAXqM7SoDJNZ/FnyiuduARGcR1bFC2SGUq/B BLjFLoL7WTzog9znGNZLKwkMZPvpThgEmT4dVHy2Io+Q1QutuId0QQqI6q2ZztURKzMQ VkCg== MIME-Version: 1.0 X-Received: by 10.194.11.72 with SMTP id o8mr11687578wjb.0.1371574580682; Tue, 18 Jun 2013 09:56:20 -0700 (PDT) Sender: mattjeet@gmail.com Received: by 10.194.45.132 with HTTP; Tue, 18 Jun 2013 09:56:20 -0700 (PDT) In-Reply-To: References: Date: Tue, 18 Jun 2013 09:56:20 -0700 X-Google-Sender-Auth: zEw2Ar_IaYxVhV7bZbpKP5RCwqk Message-ID: Subject: Re: STF ZFS test suite, part 2 From: Matt Olander To: Alan Somers Content-Type: text/plain; charset=ISO-8859-1 Cc: "zfs-devel@freebsd.org" , Alfred Perlstein X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list Reply-To: matt@ixsystems.com List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Jun 2013 16:56:22 -0000 Hi Alan, On Tue, Jun 18, 2013 at 9:41 AM, Alan Somers wrote: > A long time ago, HighCloud security ported OpenSolaris's ZFS Test > suite to FreeBSD. Spectra Logic improved it, and last September gibbs > posted a snapshot of our work to this list. Here is an updated > snapshot that contains an additional 9 months' worth of work. The > most significant update is that the tests have been adapted to run > under the ATF tools instead of the STF tools. It also contains a few > contributions from araujo. Thanks, this is fabulous. We will give it a try! > 1) You do not run it on a production system. > 2) You do not run it on a production system. So, we should definitely try it on a production system :P Thanks to both HighCloud and Spectra Logic for working on this! Cheers, -matt From owner-zfs-devel@FreeBSD.ORG Wed Jun 19 00:46:49 2013 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 1578F1AA; Wed, 19 Jun 2013 00:46:49 +0000 (UTC) (envelope-from prvs=1882457d64=killing@multiplay.co.uk) Received: from mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) by mx1.freebsd.org (Postfix) with ESMTP id 89F961ECB; Wed, 19 Jun 2013 00:46:48 +0000 (UTC) Received: from r2d2 ([82.69.141.170]) by mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) (MDaemon PRO v10.0.4) with ESMTP id md50004417614.msg; Wed, 19 Jun 2013 01:46:46 +0100 X-Spam-Processed: mail1.multiplay.co.uk, Wed, 19 Jun 2013 01:46:46 +0100 (not processed: message from valid local sender) X-MDDKIM-Result: neutral (mail1.multiplay.co.uk) X-MDRemoteIP: 82.69.141.170 X-Return-Path: prvs=1882457d64=killing@multiplay.co.uk X-Envelope-From: killing@multiplay.co.uk Message-ID: <67F372464AA94883B383D2BDDCC4C9B2@multiplay.co.uk> From: "Steven Hartland" To: "Alan Somers" , References: Subject: Re: STF ZFS test suite, part 2 Date: Wed, 19 Jun 2013 01:46:55 +0100 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="iso-8859-1"; reply-type=original Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.5931 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.6157 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Jun 2013 00:46:49 -0000 ----- Original Message ----- From: "Alan Somers" >A long time ago, HighCloud security ported OpenSolaris's ZFS Test > suite to FreeBSD. Spectra Logic improved it, and last September gibbs > posted a snapshot of our work to this list. Here is an updated > snapshot that contains an additional 9 months' worth of work. The > most significant update is that the tests have been adapted to run > under the ATF tools instead of the STF tools. It also contains a few > contributions from araujo. > > The README file contains instructions on how to build and run the > tests, but let me summarize the first two rules of the ZFS test suite: > > 1) You do not run it on a production system. > 2) You do not run it on a production system. > > http://people.freebsd.org/~asomers/stf_zfs/ Thanks for this! I've just given this a wirl on a recent 8/stable and here's some feedback:- 1. Be good to mention devel/atf port dependency in README ;-) 2. Initial tests all skip with following, is this expected? Skipped: Required configuration variable 'X-zfs_acl' not defined 3. Is there a list of the configuration variables and what they do? 4. bootfs tests seem to hang at: bootfs_005_neg: ... which is running: /sbin/zpool import -d //tmp v1-pool procstat -k -k 30723 PID TID COMM TDNAME KSTACK 30723 100857 zpool initial thread mi_switch+0x186 sleepq_catch_signals+0x31c sleepq_wait_sig+0x16 _sleep+0x29d fifo_open+0x53b VOP_OPEN_APV+0x62 vn_open_cred+0x2e5 kern_openat+0x16a amd64_syscall+0x1f4 Xfast_syscall+0xfc Top shows: 30723 root 1 76 0 19056K 1948K fifoor 15 0:00 0.00% zpool After 300 seconds it times out, is this expected? 5. Pretty much all the cache & clean_mirror tests fail, again expected? 6. system deadlocks on test: zfs_rename_008_pos: :( Be good to get an idea on what the expected results are for standard FreeBSD installs i.e. stable/8, stable/9 and current? Regards Steve ================================================ This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmaster@multiplay.co.uk. From owner-zfs-devel@FreeBSD.ORG Wed Jun 19 02:54:42 2013 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 49E1E957; Wed, 19 Jun 2013 02:54:42 +0000 (UTC) (envelope-from araujobsdport@gmail.com) Received: from mail-wi0-x22f.google.com (mail-wi0-x22f.google.com [IPv6:2a00:1450:400c:c05::22f]) by mx1.freebsd.org (Postfix) with ESMTP id AC6E912FC; Wed, 19 Jun 2013 02:54:41 +0000 (UTC) Received: by mail-wi0-f175.google.com with SMTP id m6so177595wiv.2 for ; Tue, 18 Jun 2013 19:54:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:reply-to:in-reply-to:references:date:message-id :subject:from:to:cc:content-type; bh=FcXEZJgNjRXX1OuAZkrKwI6ij7udvOCg6ingq2Fmfkw=; b=TRPYr86TLOwflMaFA81MSQ65yRSHPGI/iKLVkW2Ro6zLHIb64cPuv4tlP8fgeb3R4n MJ0FHdixveJ52yF84DgORr+WWSbfPfgXVyX8S0fbmzZjXAEfmphPL4ql7aPPVYKP4cg8 SJFSeggoomFof1kd7549LdCN98DcFvtJbrn1DsaHiT8QKTGOpWIVmzsIHiHsIXpOpn/u ftBgHfJy1cGfvYh9AeqEFBrl9gi+hyOLOymKWf2FF1iylv0I1+ie5AtGf1/A7sZpQcBX bHfNWJYbNOMZGhWr7jyPNi9vL5nWgEyXdL1ZY36yMl4VCeHb3Cp4hGb63FHm3nbhA8P/ 7wNQ== MIME-Version: 1.0 X-Received: by 10.194.83.195 with SMTP id s3mr445942wjy.82.1371610480885; Tue, 18 Jun 2013 19:54:40 -0700 (PDT) Received: by 10.180.73.180 with HTTP; Tue, 18 Jun 2013 19:54:40 -0700 (PDT) In-Reply-To: References: Date: Wed, 19 Jun 2013 10:54:40 +0800 Message-ID: Subject: Re: STF ZFS test suite, part 2 From: Marcelo Araujo To: Alan Somers Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: zfs-devel@freebsd.org X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list Reply-To: araujo@FreeBSD.org List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Jun 2013 02:54:42 -0000 2013/6/19 Alan Somers > A long time ago, HighCloud security ported OpenSolaris's ZFS Test > suite to FreeBSD. Spectra Logic improved it, and last September gibbs > posted a snapshot of our work to this list. Here is an updated > snapshot that contains an additional 9 months' worth of work. The > most significant update is that the tests have been adapted to run > under the ATF tools instead of the STF tools. It also contains a few > contributions from araujo. > > The README file contains instructions on how to build and run the > tests, but let me summarize the first two rules of the ZFS test suite: > > 1) You do not run it on a production system. > 2) You do not run it on a production system. > > http://people.freebsd.org/~asomers/stf_zfs/ > > -Alan > _______________________________________________ > Dear Alan, Thank you so much to share it with us as well as mention about myself with my few contributions. Unfortunately I'm pretty much busy at work and I could not contribute anymore. Wow, these guys have done a good work. Amazing!!! Best Regards, -- Marcelo Araujo araujo@FreeBSD.org From owner-zfs-devel@FreeBSD.ORG Wed Jun 19 16:38:24 2013 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id CB7B178C; Wed, 19 Jun 2013 16:38:24 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-qe0-f44.google.com (mail-qe0-f44.google.com [209.85.128.44]) by mx1.freebsd.org (Postfix) with ESMTP id 7CCD61E40; Wed, 19 Jun 2013 16:38:24 +0000 (UTC) Received: by mail-qe0-f44.google.com with SMTP id 5so3419495qeb.31 for ; Wed, 19 Jun 2013 09:38:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=2wdjeyYyvb3OxfgjCzcOdgwLe7IUZcA5wEFGwL1VUwI=; b=VHZ20oBf0BZ+hFp7qnq/F68bBIBN8GKqvHaVH+him9GUx+5IrYZEsbWNaHeFnhKDJv IB1iOq3oWHBihonuAJ0lnL/0m5YU2jadmAUCMeDaV8opQtxIsexJvbpCZ5kJJ4xLbUAG EYsfEa9Bh9DnwCrMPfVAp39uHMYLtMVGkwwjoyHtmcPHQC+wf08AKnfSRyycrwnx5ltd 4h5/TJchuCwt8rTMPhyOsQzTRKp9A5AvBRRHuiAuXJw/duXwv6XC+u9UVBbPV60RCRAP Y6K8QwJPNHJI16wEyWcxB7DlmbF9qNjWYLDzL0TeqVNNzhFnp/G5P6ILsDPCbfny4jCh hA0w== MIME-Version: 1.0 X-Received: by 10.229.107.14 with SMTP id z14mr1457294qco.43.1371659897933; Wed, 19 Jun 2013 09:38:17 -0700 (PDT) Received: by 10.49.37.226 with HTTP; Wed, 19 Jun 2013 09:38:17 -0700 (PDT) In-Reply-To: <67F372464AA94883B383D2BDDCC4C9B2@multiplay.co.uk> References: <67F372464AA94883B383D2BDDCC4C9B2@multiplay.co.uk> Date: Wed, 19 Jun 2013 10:38:17 -0600 Message-ID: Subject: Re: STF ZFS test suite, part 2 From: asomers@gmail.com To: Steven Hartland Content-Type: text/plain; charset=ISO-8859-1 Cc: zfs-devel@freebsd.org X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Jun 2013 16:38:24 -0000 I've updated the README to address Steven's concerns. Details inline below: On Tue, Jun 18, 2013 at 6:46 PM, Steven Hartland wrote: > > ----- Original Message ----- From: "Alan Somers" > > > >> A long time ago, HighCloud security ported OpenSolaris's ZFS Test >> suite to FreeBSD. Spectra Logic improved it, and last September gibbs >> posted a snapshot of our work to this list. Here is an updated >> snapshot that contains an additional 9 months' worth of work. The >> most significant update is that the tests have been adapted to run >> under the ATF tools instead of the STF tools. It also contains a few >> contributions from araujo. >> >> The README file contains instructions on how to build and run the >> tests, but let me summarize the first two rules of the ZFS test suite: >> >> 1) You do not run it on a production system. >> 2) You do not run it on a production system. >> >> http://people.freebsd.org/~asomers/stf_zfs/ > > > Thanks for this! > > I've just given this a wirl on a recent 8/stable and here's > some feedback:- > 1. Be good to mention devel/atf port dependency in README ;-) Done. BTW, ATF is included in the base in FreeBSD-10 > > 2. Initial tests all skip with following, is this expected? > Skipped: Required configuration variable 'X-zfs_acl' not defined Yes, it's expected. That variable enables tests that depend on Solaris's syntax for ACLs. > > 3. Is there a list of the configuration variables and what they do? Only what's in the README. > > 4. bootfs tests seem to hang at: bootfs_005_neg: ... which is running: > /sbin/zpool import -d //tmp v1-pool > > procstat -k -k 30723 > PID TID COMM TDNAME KSTACK > 30723 100857 zpool initial thread mi_switch+0x186 > sleepq_catch_signals+0x31c sleepq_wait_sig+0x16 _sleep+0x29d fifo_open+0x53b > VOP_OPEN_APV+0x62 vn_open_cred+0x2e5 kern_openat+0x16a amd64_syscall+0x1f4 > Xfast_syscall+0xfc > > Top shows: > 30723 root 1 76 0 19056K 1948K fifoor 15 0:00 0.00% zpool > > After 300 seconds it times out, is this expected? Nope. That's a bug in FreeBSD 8, apparently. > > 5. Pretty much all the cache & clean_mirror tests fail, again expected? Not expected. I think that the cache tests have a bug that makes them fail if you're only running with one disk. With more disks, they should pass. Over here, they all pass except for cache_009_pos. > > 6. system deadlocks on test: zfs_rename_008_pos: :( Well, don't run that one ;). > > Be good to get an idea on what the expected results are for standard > FreeBSD installs i.e. stable/8, stable/9 and current? Spectra Logic's custom FreeBSD fork has a lot of bug fixes in ZFS. Will Andrews is slowly pushing those upstream. As they go in, I would expect FreeBSD-CURRENT to improve. stable/8 and stable/9, however, are going to do very badly. About 40 of the tests are marked as expected failures and will show up that way in the ATF results. Those are the tests that fail in SpectraBSD, and almost all of them should also fail in FreeBSD. I think that only inuse_005_pos,hotspare_add_004_neg, zfs_mount_007_pos, and zfs_get_003_pos will pass in FreeBSD but not in SpectraBSD. > > Regards > Steve > > ================================================ > This e.mail is private and confidential between Multiplay (UK) Ltd. and the > person or entity to whom it is addressed. In the event of misdirection, the > recipient is prohibited from using, copying, printing or otherwise > disseminating it or any information contained in it. > In the event of misdirection, illegible or incomplete transmission please > telephone +44 845 868 1337 > or return the E.mail to postmaster@multiplay.co.uk. > From owner-zfs-devel@FreeBSD.ORG Wed Jun 19 17:24:16 2013 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 99786BDA; Wed, 19 Jun 2013 17:24:16 +0000 (UTC) (envelope-from prvs=1882457d64=killing@multiplay.co.uk) Received: from mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) by mx1.freebsd.org (Postfix) with ESMTP id 039B311E7; Wed, 19 Jun 2013 17:24:15 +0000 (UTC) Received: from r2d2 ([82.69.141.170]) by mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) (MDaemon PRO v10.0.4) with ESMTP id md50004425869.msg; Wed, 19 Jun 2013 18:24:11 +0100 X-Spam-Processed: mail1.multiplay.co.uk, Wed, 19 Jun 2013 18:24:11 +0100 (not processed: message from valid local sender) X-MDDKIM-Result: neutral (mail1.multiplay.co.uk) X-MDRemoteIP: 82.69.141.170 X-Return-Path: prvs=1882457d64=killing@multiplay.co.uk X-Envelope-From: killing@multiplay.co.uk Message-ID: From: "Steven Hartland" To: References: <67F372464AA94883B383D2BDDCC4C9B2@multiplay.co.uk> Subject: Re: STF ZFS test suite, part 2 Date: Wed, 19 Jun 2013 18:24:19 +0100 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="iso-8859-1"; reply-type=original Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.5931 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.6157 Cc: zfs-devel@freebsd.org X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Jun 2013 17:24:16 -0000 ----- Original Message ----- From: >> I've just given this a wirl on a recent 8/stable and here's >> some feedback:- >> 1. Be good to mention devel/atf port dependency in README ;-) > > Done. BTW, ATF is included in the base in FreeBSD-10 Good to know :) >> 2. Initial tests all skip with following, is this expected? >> Skipped: Required configuration variable 'X-zfs_acl' not defined > > Yes, it's expected. That variable enables tests that depend on > Solaris's syntax for ACLs. > >> >> 3. Is there a list of the configuration variables and what they do? > > Only what's in the README. > >> >> 4. bootfs tests seem to hang at: bootfs_005_neg: ... which is running: >> /sbin/zpool import -d //tmp v1-pool >> >> procstat -k -k 30723 >> PID TID COMM TDNAME KSTACK >> 30723 100857 zpool initial thread mi_switch+0x186 >> sleepq_catch_signals+0x31c sleepq_wait_sig+0x16 _sleep+0x29d fifo_open+0x53b >> VOP_OPEN_APV+0x62 vn_open_cred+0x2e5 kern_openat+0x16a amd64_syscall+0x1f4 >> Xfast_syscall+0xfc >> >> Top shows: >> 30723 root 1 76 0 19056K 1948K fifoor 15 0:00 0.00% zpool >> >> After 300 seconds it times out, is this expected? > > Nope. That's a bug in FreeBSD 8, apparently. This appears to be an issue with using /tmp as the target dir, using another directory and I can run the import without issue it seems. Other tests also hang with the same issue:- zpool_upgrade_002_pos zpool_upgrade_003_pos zpool_upgrade_007_pos zpool_upgrade_008_pos zpool_upgrade_009_neg Would it be an issue to change the directory? >> 5. Pretty much all the cache & clean_mirror tests fail, again expected? > > Not expected. I think that the cache tests have a bug that makes them > fail if you're only running with one disk. With more disks, they > should pass. Over here, they all pass except for cache_009_pos. > >> >> 6. system deadlocks on test: zfs_rename_008_pos: :( > > Well, don't run that one ;). This is related to the following PR: http://www.freebsd.org/cgi/query-pr.cgi?pr=161968 Doesn't seem to effect current or stable/9 I've only managed to reproduce on only stable/8, so already getting benefit from this excellent work :) Whats the best way to disable a specific test Alan? >> Be good to get an idea on what the expected results are for standard >> FreeBSD installs i.e. stable/8, stable/9 and current? > > Spectra Logic's custom FreeBSD fork has a lot of bug fixes in ZFS. > Will Andrews is slowly pushing those upstream. As they go in, I would > expect FreeBSD-CURRENT to improve. stable/8 and stable/9, however, > are going to do very badly. About 40 of the tests are marked as > expected failures and will show up that way in the ATF results. Those > are the tests that fail in SpectraBSD, and almost all of them should > also fail in FreeBSD. I think that only > inuse_005_pos,hotspare_add_004_neg, zfs_mount_007_pos, and > zfs_get_003_pos will pass in FreeBSD but not in SpectraBSD. Thanks Alan, good to know. Just completed Regards Steve ================================================ This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmaster@multiplay.co.uk. From owner-zfs-devel@FreeBSD.ORG Wed Jun 19 17:37:01 2013 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 32C08F91; Wed, 19 Jun 2013 17:37:01 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-qa0-x230.google.com (mail-qa0-x230.google.com [IPv6:2607:f8b0:400d:c00::230]) by mx1.freebsd.org (Postfix) with ESMTP id D75AA128B; Wed, 19 Jun 2013 17:37:00 +0000 (UTC) Received: by mail-qa0-f48.google.com with SMTP id cm16so566802qab.7 for ; Wed, 19 Jun 2013 10:37:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=NoD+IuzrlUHzn/9V80/k1h/I37/EtpII8x/iVNxQPzg=; b=UOI4qt5vXFgmLnFSRrrJjV+QsrkqOC7ZsLREeGPfHMcVGKPCKd17V43tyyh5g7UI0N gKzSyVzseFk3+H8v9OiCMiZPZKpF0KxjlbNgV8wz8UPV2azJIhFLpDUQ9ZDKwr7jsOJB e/1bOBnDQ0l9guRJkdcG7iCWbG2hih4SBbkcIF44ryGZo46w7UXyt4xflbO8M65Gi+XJ gL2W0qx2exwJkWpcwMAYCzFNxNbCp3YPmGlwKYILlIyPKSoEdGalF145bbZDiBFghWxD vkG272V4v6kKGpQUMTG6gdIjY0diOe31fzxI9q63RzgkdKn0bmAu/+ZdmsqJgfYGkPTO ttWw== MIME-Version: 1.0 X-Received: by 10.229.78.141 with SMTP id l13mr1435943qck.58.1371663420299; Wed, 19 Jun 2013 10:37:00 -0700 (PDT) Received: by 10.49.37.226 with HTTP; Wed, 19 Jun 2013 10:37:00 -0700 (PDT) In-Reply-To: References: <67F372464AA94883B383D2BDDCC4C9B2@multiplay.co.uk> Date: Wed, 19 Jun 2013 11:37:00 -0600 Message-ID: Subject: Re: STF ZFS test suite, part 2 From: asomers@gmail.com To: Steven Hartland Content-Type: text/plain; charset=ISO-8859-1 Cc: zfs-devel@freebsd.org X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Jun 2013 17:37:01 -0000 On Wed, Jun 19, 2013 at 11:24 AM, Steven Hartland wrote: > ----- Original Message ----- From: > >>> I've just given this a wirl on a recent 8/stable and here's >>> some feedback:- >>> 1. Be good to mention devel/atf port dependency in README ;-) >> >> >> Done. BTW, ATF is included in the base in FreeBSD-10 > > > Good to know :) > > >>> 2. Initial tests all skip with following, is this expected? >>> Skipped: Required configuration variable 'X-zfs_acl' not defined >> >> >> Yes, it's expected. That variable enables tests that depend on >> Solaris's syntax for ACLs. >> >>> >>> 3. Is there a list of the configuration variables and what they do? >> >> >> Only what's in the README. >> >>> >>> 4. bootfs tests seem to hang at: bootfs_005_neg: ... which is running: >>> /sbin/zpool import -d //tmp v1-pool >>> >>> procstat -k -k 30723 >>> PID TID COMM TDNAME KSTACK >>> 30723 100857 zpool initial thread mi_switch+0x186 >>> sleepq_catch_signals+0x31c sleepq_wait_sig+0x16 _sleep+0x29d >>> fifo_open+0x53b >>> VOP_OPEN_APV+0x62 vn_open_cred+0x2e5 kern_openat+0x16a >>> amd64_syscall+0x1f4 >>> Xfast_syscall+0xfc >>> >>> Top shows: >>> 30723 root 1 76 0 19056K 1948K fifoor 15 0:00 0.00% >>> zpool >>> >>> After 300 seconds it times out, is this expected? >> >> >> Nope. That's a bug in FreeBSD 8, apparently. > > > This appears to be an issue with using /tmp as the target dir, using another > directory and I can run the import without issue it seems. Other tests also > hang with the same issue:- > zpool_upgrade_002_pos > zpool_upgrade_003_pos > zpool_upgrade_007_pos > zpool_upgrade_008_pos > zpool_upgrade_009_neg > > Would it be an issue to change the directory? You should be able to use any output directory at all. Also, I usually put TMPDIR on its own UFS filesystem because it makes the hotspare tests go faster. > > >>> 5. Pretty much all the cache & clean_mirror tests fail, again expected? >> >> >> Not expected. I think that the cache tests have a bug that makes them >> fail if you're only running with one disk. With more disks, they >> should pass. Over here, they all pass except for cache_009_pos. >> >>> >>> 6. system deadlocks on test: zfs_rename_008_pos: :( >> >> >> Well, don't run that one ;). > > > This is related to the following PR: > http://www.freebsd.org/cgi/query-pr.cgi?pr=161968 > > Doesn't seem to effect current or stable/9 I've only managed to reproduce > on only stable/8, so already getting benefit from this excellent work :) > > Whats the best way to disable a specific test Alan? The best way to disable a test is to edit the ATF test program file and add an "atf_skip" statement to the body of the test. For example, see zfs_003_neg_body in tests/functional/cli_root/zfs/zfs. > > >>> Be good to get an idea on what the expected results are for standard >>> FreeBSD installs i.e. stable/8, stable/9 and current? >> >> >> Spectra Logic's custom FreeBSD fork has a lot of bug fixes in ZFS. >> Will Andrews is slowly pushing those upstream. As they go in, I would >> expect FreeBSD-CURRENT to improve. stable/8 and stable/9, however, >> are going to do very badly. About 40 of the tests are marked as >> expected failures and will show up that way in the ATF results. Those >> are the tests that fail in SpectraBSD, and almost all of them should >> also fail in FreeBSD. I think that only >> inuse_005_pos,hotspare_add_004_neg, zfs_mount_007_pos, and >> zfs_get_003_pos will pass in FreeBSD but not in SpectraBSD. > > > Thanks Alan, good to know. > > Just completed > Regards > Steve > > ================================================ > This e.mail is private and confidential between Multiplay (UK) Ltd. and the > person or entity to whom it is addressed. In the event of misdirection, the > recipient is prohibited from using, copying, printing or otherwise > disseminating it or any information contained in it. > In the event of misdirection, illegible or incomplete transmission please > telephone +44 845 868 1337 > or return the E.mail to postmaster@multiplay.co.uk. > From owner-zfs-devel@FreeBSD.ORG Thu Jun 20 11:15:36 2013 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 4120AB03 for ; Thu, 20 Jun 2013 11:15:36 +0000 (UTC) (envelope-from prvs=1883808c8e=killing@multiplay.co.uk) Received: from mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) by mx1.freebsd.org (Postfix) with ESMTP id D90351DA1 for ; Thu, 20 Jun 2013 11:15:35 +0000 (UTC) Received: from r2d2 ([82.69.141.170]) by mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) (MDaemon PRO v10.0.4) with ESMTP id md50004431005.msg for ; Thu, 20 Jun 2013 12:15:34 +0100 X-Spam-Processed: mail1.multiplay.co.uk, Thu, 20 Jun 2013 12:15:34 +0100 (not processed: message from valid local sender) X-MDDKIM-Result: neutral (mail1.multiplay.co.uk) X-MDRemoteIP: 82.69.141.170 X-Return-Path: prvs=1883808c8e=killing@multiplay.co.uk X-Envelope-From: killing@multiplay.co.uk X-MDaemon-Deliver-To: zfs-devel@freebsd.org Message-ID: From: "Steven Hartland" To: References: <67F372464AA94883B383D2BDDCC4C9B2@multiplay.co.uk> Subject: Re: STF ZFS test suite, part 2 Date: Thu, 20 Jun 2013 12:15:41 +0100 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="iso-8859-1"; reply-type=original Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.5931 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.6157 Cc: zfs-devel@freebsd.org X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Jun 2013 11:15:36 -0000 ----- Original Message ----- From: >> This appears to be an issue with using /tmp as the target dir, using another >> directory and I can run the import without issue it seems. Other tests also >> hang with the same issue:- >> zpool_upgrade_002_pos >> zpool_upgrade_003_pos >> zpool_upgrade_007_pos >> zpool_upgrade_008_pos >> zpool_upgrade_009_neg >> >> Would it be an issue to change the directory? > > You should be able to use any output directory at all. Also, I > usually put TMPDIR on its own UFS filesystem because it makes the > hotspare tests go faster. I've confirmed changing the default TMPDIR fixes ths issue. I believe the problem is in /tmp there's some fifo's, so possibly some bad interaction with those, that and I'm sure its not a good idea to mount a pool over the main system tmp directory ;-) >>>> 5. Pretty much all the cache & clean_mirror tests fail, again expected? >>> >>> >>> Not expected. I think that the cache tests have a bug that makes them >>> fail if you're only running with one disk. With more disks, they >>> should pass. Over here, they all pass except for cache_009_pos. >>> The problem with cache tests is they use the shell function is_physical_device which is using a hard coded list of device names it considers a physical devices, this doesn't match our devices. Possibly use [[-c $1]] to look for character special device instead? The following patch to the lib makes the tests pass here: --- include/libtest.kshlib.orig 2013-06-20 00:00:37.923125499 +0000 +++ include/libtest.kshlib 2013-06-20 00:51:14.302027497 +0000 @@ -2682,9 +2682,8 @@ # function is_physical_device #device { - typeset device=${1#/dev/} + [[ -c $1 ]] - $ECHO $device | $EGREP "^(ada|da|mlxd|myld|aacd|ided|twed)[0-9]+(s[0-9]+)?$" > /dev/null 2>&1 return $? } Regards Steve ================================================ This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmaster@multiplay.co.uk. From owner-zfs-devel@FreeBSD.ORG Thu Jun 20 15:44:59 2013 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 30A09F3A for ; Thu, 20 Jun 2013 15:44:59 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-qc0-x234.google.com (mail-qc0-x234.google.com [IPv6:2607:f8b0:400d:c01::234]) by mx1.freebsd.org (Postfix) with ESMTP id EA79811A7 for ; Thu, 20 Jun 2013 15:44:58 +0000 (UTC) Received: by mail-qc0-f180.google.com with SMTP id a1so3833970qcx.11 for ; Thu, 20 Jun 2013 08:44:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; bh=QL4LI+TaTiPTlKnBeNqurZS/2QrfuWwocgcbGYFm/9U=; b=0s/9LgsXYmLV/kJFspwVafB/KuzXh0E/qOa19L+wkJcGar4OTbTYVa7T+AAs5wlHd+ op6u0yhXl/emRfNf/x9LPTA3neD+Mz0DZTkkKlG2Swjol3cNow1SF7ryuCtRmB6JhJCp KoyBEULHW7OOwFjE38JjwMVEgUDg+FoiG3zPTdLaLraYRbptw8bqryLvESgl/Q4wWq0A yTUe9l+eJ0ZsasoDW+UiMBXN7yObQ6y9vb0X4p7AyDUy3WEWXRbNdexceg6Fw7LlkUCA O8SjxZAYwn96a5f8e4zulyzQPIbaAbMgV4+enIioyFpn4xXXexAQJBvuE3Mt0mzHvt0X fhKQ== MIME-Version: 1.0 X-Received: by 10.49.6.6 with SMTP id w6mr9912507qew.44.1371743098454; Thu, 20 Jun 2013 08:44:58 -0700 (PDT) Sender: asomers@gmail.com Received: by 10.49.37.226 with HTTP; Thu, 20 Jun 2013 08:44:58 -0700 (PDT) In-Reply-To: References: <67F372464AA94883B383D2BDDCC4C9B2@multiplay.co.uk> Date: Thu, 20 Jun 2013 09:44:58 -0600 X-Google-Sender-Auth: uUybhGpZD8_6vwwKA-4vDu0oqZs Message-ID: Subject: Re: STF ZFS test suite, part 2 From: Alan Somers To: Steven Hartland Content-Type: text/plain; charset=ISO-8859-1 Cc: zfs-devel@freebsd.org X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Jun 2013 15:44:59 -0000 On Thu, Jun 20, 2013 at 5:15 AM, Steven Hartland wrote: > ----- Original Message ----- From: > >>> This appears to be an issue with using /tmp as the target dir, using >>> another >>> directory and I can run the import without issue it seems. Other tests >>> also >>> hang with the same issue:- >>> zpool_upgrade_002_pos >>> zpool_upgrade_003_pos >>> zpool_upgrade_007_pos >>> zpool_upgrade_008_pos >>> zpool_upgrade_009_neg >>> >>> Would it be an issue to change the directory? >> >> >> You should be able to use any output directory at all. Also, I >> usually put TMPDIR on its own UFS filesystem because it makes the >> hotspare tests go faster. > > > I've confirmed changing the default TMPDIR fixes ths issue. I believe > the problem is in /tmp there's some fifo's, so possibly some bad > interaction with those, that and I'm sure its not a good idea to mount > a pool over the main system tmp directory ;-) The tests shouldn't be mounting a pool on TMPDIR. If they are, then that's a bug. It's probably something like "mount foo ${TMPDIR}/${BAR}", where BAR is undefined. Can you tell which test is doing it? > > >>>>> 5. Pretty much all the cache & clean_mirror tests fail, again expected? >>>> >>>> >>>> >>>> Not expected. I think that the cache tests have a bug that makes them >>>> fail if you're only running with one disk. With more disks, they >>>> should pass. Over here, they all pass except for cache_009_pos. >>>> > > The problem with cache tests is they use the shell function > is_physical_device which is using a hard coded list of device names > it considers a physical devices, this doesn't match our devices. > > Possibly use [[-c $1]] to look for character special device instead? > > The following patch to the lib makes the tests pass here: > --- include/libtest.kshlib.orig 2013-06-20 00:00:37.923125499 +0000 > +++ include/libtest.kshlib 2013-06-20 00:51:14.302027497 +0000 > @@ -2682,9 +2682,8 @@ > # > function is_physical_device #device > { > - typeset device=${1#/dev/} > + [[ -c $1 ]] > > - $ECHO $device | $EGREP > "^(ada|da|mlxd|myld|aacd|ided|twed)[0-9]+(s[0-9]+)?$" > /dev/null 2>&1 > return $? > > } Thanks for the patch. > > Regards > Steve > > ================================================ > This e.mail is private and confidential between Multiplay (UK) Ltd. and the > person or entity to whom it is addressed. In the event of misdirection, the > recipient is prohibited from using, copying, printing or otherwise > disseminating it or any information contained in it. > In the event of misdirection, illegible or incomplete transmission please > telephone +44 845 868 1337 > or return the E.mail to postmaster@multiplay.co.uk. > From owner-zfs-devel@FreeBSD.ORG Fri Jun 21 15:22:44 2013 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id E13AAA36; Fri, 21 Jun 2013 15:22:44 +0000 (UTC) (envelope-from prvs=1884904eea=killing@multiplay.co.uk) Received: from mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) by mx1.freebsd.org (Postfix) with ESMTP id 49DC41667; Fri, 21 Jun 2013 15:22:43 +0000 (UTC) Received: from r2d2 ([82.69.141.170]) by mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) (MDaemon PRO v10.0.4) with ESMTP id md50004456480.msg; Fri, 21 Jun 2013 16:22:36 +0100 X-Spam-Processed: mail1.multiplay.co.uk, Fri, 21 Jun 2013 16:22:36 +0100 (not processed: message from valid local sender) X-MDDKIM-Result: neutral (mail1.multiplay.co.uk) X-MDRemoteIP: 82.69.141.170 X-Return-Path: prvs=1884904eea=killing@multiplay.co.uk X-Envelope-From: killing@multiplay.co.uk Message-ID: <16E7A29DFC6445C49A2B363C23BA62ED@multiplay.co.uk> From: "Steven Hartland" To: "Alan Somers" References: <67F372464AA94883B383D2BDDCC4C9B2@multiplay.co.uk> Subject: Re: STF ZFS test suite, part 2 Date: Fri, 21 Jun 2013 16:22:42 +0100 MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="----=_NextPart_000_00D9_01CE6E9B.8E056EB0" X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.5931 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.6157 Cc: zfs-devel@FreeBSD.org X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Jun 2013 15:22:44 -0000 This is a multi-part message in MIME format. ------=_NextPart_000_00D9_01CE6E9B.8E056EB0 Content-Type: text/plain; format=flowed; charset="iso-8859-1"; reply-type=original Content-Transfer-Encoding: 7bit ----- Original Message ----- From: "Alan Somers" >>>> This appears to be an issue with using /tmp as the target dir, using >>>> another directory and I can run the import without issue it seems. >>>> Other tests also hang with the same issue:- >>>> zpool_upgrade_002_pos >>>> zpool_upgrade_003_pos >>>> zpool_upgrade_007_pos >>>> zpool_upgrade_008_pos >>>> zpool_upgrade_009_neg >>>> >>>> Would it be an issue to change the directory? >>> >>> You should be able to use any output directory at all. Also, I >>> usually put TMPDIR on its own UFS filesystem because it makes the >>> hotspare tests go faster. >> >> >> I've confirmed changing the default TMPDIR fixes ths issue. I believe >> the problem is in /tmp there's some fifo's, so possibly some bad >> interaction with those, that and I'm sure its not a good idea to mount >> a pool over the main system tmp directory ;-) > > The tests shouldn't be mounting a pool on TMPDIR. If they are, then > that's a bug. It's probably something like "mount foo > ${TMPDIR}/${BAR}", where BAR is undefined. > Can you tell which test is doing it? Pretty much all of the zpool_upgrade_* tests I believe the function at fault is create_old_pool which runs: log_must $ZPOOL import -d /$TMPDIR $POOL_NAME Some new issues:- In the xml log I'm seeing a number of errors, which I believe are setup problems which could well be causing issues with the tests e.g. gpart: arg0 'md0': Invalid argument The following fixes that:- --- ./include/libtest.kshlib.orig 2013-06-20 00:00:37.923125499 +0000 +++ ./include/libtest.kshlib 2013-06-20 19:02:31.396756337 +0000 @@ -635,7 +635,7 @@ function wipe_partition_table # /dev/null 2>&1 shift done } A number of tests fail as the ports mkfile is broken and exits 0 when the file creation fails including:- refquota_00*_pos, refreserv_001_pos, zpool_add_006_pos, zpool_create_004_pos The patch attached fixes this, which I've submitted to the port maintainer. Here are my current results from stable/9, initally I was seeing 36 unexpected test failures but with some patches I'm now down to 19 of which I believe only 4 are actual failures, details below. == Failure Key == ! = Broken test * = Test passed but expected failure - = Test was broken but now passes with local fixes ? = Possibly broken test # = Real failure == Failed test cases == !cli_root/zfs_snapshot/zfs_snapshot:zfs_snapshot_001_neg + broken test: multiple snaps can be created !cli_root/zpool_import/zpool_import:zpool_import_corrupt_001_pos + broken test: dd oseek cant be negative !cli_root/zpool_scrub/zpool_scrub:zpool_scrub_003_pos + broken test: resliver completes before check for in progress can complete !mmap/mmap_write/mmap_write:mmap_write_001_pos + broken test: runs out of space *cache/cache:cache_009_pos + passed: expected failure *cli_root/zfs_copies/zfs_copies:zfs_copies_003_pos + passed: expected failure *cli_root/zfs_create/zfs_create:zfs_create_013_pos + passed: expected failure *cli_root/zfs_get/zfs_get:zfs_get_009_pos + passed: expected failure *cli_root/zpool_destroy/zpool_destroy:zpool_destroy_001_pos + passed: expected failure *hotspare/hotspare:hotspare_add_004_neg + passed: expected failure *pool_names/pool_names:pool_names_001_pos + passed: expected failure -cli_root/zfs_rename/zfs_rename:zfs_rename_007_pos + failed: clone not creating zvol entry -cli_root/zfs_set/zfs_set:readonly_001_pos + failed: zvol clone bug -cli_root/zfs_set/zfs_set:ro_props_001_pos + failed: fixed by mkfile patch -cli_root/zpool_add/zpool_add:zpool_add_006_pos + failed: fixed by mkfile patch -cli_root/zpool_create/zpool_create:zpool_create_004_pos + failed: fixed by mkfile patch -cli_root/zpool_import/zpool_import:zpool_import_014_pos + failed: cant import deleted pools with -D -refquota/refquota:refquota_001_pos + failed: mkfile doesn't check for any errors on write -refquota/refquota:refquota_002_pos + failed: mkfile doesn't check for any errors on write -refquota/refquota:refquota_003_pos + failed: mkfile doesn't check for any errors on write -refquota/refquota:refquota_004_pos + failed: mkfile doesn't check for any errors on write -refquota/refquota:refquota_005_pos + failed: mkfile doesn't check for any errors on write -refreserv/refreserv:refreserv_001_pos + failed: mkfile doesn't check for any errors on write -snapshot/snapshot:snapshot_014_pos + failed: mkfile doesn't check for any errors on write -zil/zil:zil_001_pos + failed: broken freeze ioctl -zil/zil:zil_002_pos + failed: broken freeze ioctl -zvol/zvol_misc/zvol_misc:zvol_misc_007_pos + failed: recursive zvol rename fails -zvol/zvol_misc/zvol_misc:zvol_misc_008_pos + failed: zvol rename in promote -zvol/zvol_misc/zvol_misc:zvol_misc_009_pos + failed: recursive zvol rename fails ?cli_root/zpool_create/zpool_create:zpool_create_005_pos + failed: possibly broken test, everything reports SUCCESS? ?inuse/inuse:inuse_005_pos + passed: expected failure, could be a broken test as everything reports SUCCESS? ?zvol_thrash/zvol_thrash:zvol_thrash_001_pos + failed: uses camcontrol which fails on md's #history/history:history_008_pos + failed: missing history #history/history:history_009_pos + failed: missing history #history/history:history_010_pos + failed: missing history #snapshot/snapshot:snapshot_018_pos + failed: unknown sysctl abbreviated_snapdir == Before fixes === Summary for 102 test programs: 393 passed test cases. 36 failed test cases. 30 expected failed test cases. 173 skipped test cases. == After fixes == Summary for 102 test programs: 411 passed test cases. 18 failed test cases. 30 expected failed test cases. 173 skipped test cases. == Time taken for test suite == 1h38m32.62s real 7m6.98s user 36m3.05s sys I'll be committing the fixes I've found as soon as full head tests are complete. Regards Steve ================================================ This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmaster@multiplay.co.uk. ------=_NextPart_000_00D9_01CE6E9B.8E056EB0 Content-Type: text/plain; format=flowed; name="patch-mkfile.c"; reply-type=original Content-Transfer-Encoding: quoted-printable Content-Disposition: attachment; filename="patch-mkfile.c" --- mkfile.c.orig 2004-01-21 14:47:44.000000000 +0000=0A= +++ mkfile.c 2013-06-20 17:09:46.928028522 +0000=0A= @@ -36,6 +36,9 @@=0A= #include =0A= #include =0A= #include =0A= +#include =0A= +#include =0A= +#include =0A= =0A= #define MKFILE_WBUF (1048576) /* XXX - is 1M an reasonable value? */=0A= =0A= @@ -44,6 +47,9 @@=0A= */=0A= #define MKFILE_MODE (S_IRUSR | S_IWUSR | S_ISVTX)=0A= =0A= +=0A= +#define MIN(a, b) ((a) < (b) ? (a) : (b))=0A= +=0A= static char buf[MKFILE_WBUF];=0A= static int nofill =3D 0;=0A= static int verbose =3D 0;=0A= @@ -122,36 +128,65 @@=0A= bzero(buf, MKFILE_WBUF);=0A= }=0A= =0A= -static void=0A= +static int=0A= create_file(f, s)=0A= char *f;=0A= off_t s;=0A= {=0A= - int fd, i;=0A= - size_t bk, ix;=0A= + int fd, i, ret;=0A= +=0A= + ret =3D 0;=0A= + fd =3D -1;=0A= =0A= if (verbose) {=0A= - fprintf(stdout, "%s %qu bytes\n", f, s);=0A= + (void) fprintf(stdout, "%s %zd bytes\n", f, s);=0A= fflush(stdout);=0A= }=0A= if ((fd =3D open(f, O_WRONLY | O_CREAT | O_TRUNC, MKFILE_MODE)) < 0) {=0A= - perror(f);=0A= - } else {=0A= - lseek(fd, s - (off_t)1, SEEK_SET);=0A= - write(fd, buf, (size_t)1);=0A= - if (!nofill) {=0A= - lseek(fd, (off_t)0, SEEK_SET);=0A= - bk =3D (size_t)(s / (off_t)MKFILE_WBUF);=0A= - ix =3D (size_t)(s % (off_t)MKFILE_WBUF);=0A= - for (i =3D 0; i < bk; i++) {=0A= - write(fd, buf, (size_t)MKFILE_WBUF);=0A= - }=0A= - if (ix) {=0A= - write(fd, buf, (size_t)ix);=0A= + ret =3D errno;=0A= + warn("Could not open %s", f);=0A= + goto done;=0A= + }=0A= + if (lseek(fd, s - (off_t)1, SEEK_SET) < 0) {=0A= + ret =3D errno;=0A= + warn("Could not seek to offset %zd in %s", s - (off_t)1, f);=0A= + goto done;=0A= + }=0A= +=0A= + if (write(fd, buf, (size_t)1) !=3D 1) {=0A= + ret =3D errno;=0A= + warn("Could not set length of %s:", f);=0A= + goto done;=0A= + }=0A= +=0A= + if (!nofill) {=0A= + off_t written;=0A= +=0A= + written =3D 0;=0A= + if (lseek(fd, (off_t)0, SEEK_SET) < 0) {=0A= + ret =3D errno;=0A= + warn("Could not seek to beginning of %s", f);=0A= + goto done;=0A= + }=0A= + while (written < s) {=0A= + ssize_t result;=0A= + size_t bytes;=0A= +=0A= + bytes =3D (size_t)MIN(MKFILE_WBUF, s - written);=0A= + if ((result =3D write(fd, buf, bytes)) =3D=3D -1) {=0A= + ret =3D errno;=0A= + warn("Initialized %s to %zu of %zu bytes",=0A= + f, written, s);=0A= + goto done;=0A= }=0A= + written +=3D result;=0A= }=0A= - close(fd);=0A= }=0A= +done:=0A= + if (fd !=3D -1)=0A= + close(fd);=0A= +=0A= + return ret;=0A= }=0A= =0A= int=0A= @@ -161,9 +196,13 @@=0A= {=0A= off_t fsize;=0A= char *p;=0A= + int ret;=0A= +=0A= + ret =3D 0;=0A= =0A= if (argc < 3) {=0A= usage();=0A= + return (1);=0A= }=0A= argv++;=0A= init_buf();=0A= @@ -185,10 +224,10 @@=0A= if (!fsize) {=0A= fsize =3D getsize(*argv);=0A= } else {=0A= - create_file(*argv, fsize);=0A= + ret +=3D create_file(*argv, fsize);=0A= }=0A= }=0A= }=0A= =0A= - return 0;=0A= + return (ret);=0A= }=0A= ------=_NextPart_000_00D9_01CE6E9B.8E056EB0-- From owner-zfs-devel@FreeBSD.ORG Fri Jun 21 15:32:12 2013 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 8B6E6F57 for ; Fri, 21 Jun 2013 15:32:12 +0000 (UTC) (envelope-from will@firepipe.net) Received: from mail-ve0-x236.google.com (mail-ve0-x236.google.com [IPv6:2607:f8b0:400c:c01::236]) by mx1.freebsd.org (Postfix) with ESMTP id 4C20116EC for ; Fri, 21 Jun 2013 15:32:12 +0000 (UTC) Received: by mail-ve0-f182.google.com with SMTP id ox1so6524614veb.27 for ; Fri, 21 Jun 2013 08:32:11 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:x-gm-message-state; bh=mqwoxXGhTYS8C6rxbDmGjY4JTRgY82dOfmpEGCyCo5s=; b=PoIu1uSwLyJQCngqYoKBVV8ZSG6bwzs4f030/ygE1xj/M8EyHlAMU7kx438wWfsNvs +GXbRVYfibfTDlpwja6RnqMFoiJWWbWAHSBx6y927kiQGzTjL85daOtTiO3UYW1EqRg0 AcHcBdIzNi9/oir/fCf7Alwiy/Wt3totZ6JP2sB62bevzLf7ydnR5+LRA2XhBneyUGpZ wn1SuhbUii8+U7zB3oTtPt/x9ziq9OXkIN7rpeRRBEzX4qXvdeFIl/o6pzKUyq1Zlybz 4ILym6zyT+cDXs9PZC2Aj6+zCmNibGMiQhAMs2WMF0GE8dzMwgAsnFtC8IjEwbD50LD3 Ehmg== MIME-Version: 1.0 X-Received: by 10.52.94.142 with SMTP id dc14mr5120288vdb.86.1371828731831; Fri, 21 Jun 2013 08:32:11 -0700 (PDT) Received: by 10.58.226.66 with HTTP; Fri, 21 Jun 2013 08:32:11 -0700 (PDT) In-Reply-To: <16E7A29DFC6445C49A2B363C23BA62ED@multiplay.co.uk> References: <67F372464AA94883B383D2BDDCC4C9B2@multiplay.co.uk> <16E7A29DFC6445C49A2B363C23BA62ED@multiplay.co.uk> Date: Fri, 21 Jun 2013 09:32:11 -0600 Message-ID: Subject: Re: STF ZFS test suite, part 2 From: Will Andrews To: Steven Hartland Content-Type: text/plain; charset=UTF-8 X-Gm-Message-State: ALoCoQko6ccABgz3UcXsEH2MovEgRRlFgmBJJtF+heqq7+jwgpmMSJB1d3F3KJ9cER5PPm0QV8W5 Cc: zfs-devel@freebsd.org X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Jun 2013 15:32:12 -0000 On Fri, Jun 21, 2013 at 9:22 AM, Steven Hartland wrote: > == Failed test cases == > -cli_root/zfs_rename/zfs_rename:zfs_rename_007_pos > + failed: clone not creating zvol entry > -cli_root/zfs_set/zfs_set:readonly_001_pos > + failed: zvol clone bug > -zil/zil:zil_001_pos > + failed: broken freeze ioctl > -zil/zil:zil_002_pos > + failed: broken freeze ioctl > -zvol/zvol_misc/zvol_misc:zvol_misc_007_pos > + failed: recursive zvol rename fails > -zvol/zvol_misc/zvol_misc:zvol_misc_008_pos > + failed: zvol rename in promote > -zvol/zvol_misc/zvol_misc:zvol_misc_009_pos > + failed: recursive zvol rename fails > ?zvol_thrash/zvol_thrash:zvol_thrash_001_pos > + failed: uses camcontrol which fails on md's > #history/history:history_008_pos + failed: missing history > #history/history:history_009_pos > + failed: missing history > #history/history:history_010_pos > + failed: missing history These are all fixed in SpectraBSD. I specifically wrote zvol_misc_00[789]_pos for that purpose. On Solaris, they use vnops which don't have the problem of having to populate a device tree at all times. But FreeBSD uses geoms which have other advantages. > #snapshot/snapshot:snapshot_018_pos > + failed: unknown sysctl abbreviated_snapdir This one is currently SpectraBSD-specific. abbreviated_snapdir is a sysctl Alan added which allows having snapshots be one level higher up, i.e. .snapshots instead of .zfs/snapshots. We should change it to be a per-zfs property and upstream it. Thanks, --Will. From owner-zfs-devel@FreeBSD.ORG Fri Jun 21 16:37:40 2013 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 60764A98 for ; Fri, 21 Jun 2013 16:37:40 +0000 (UTC) (envelope-from prvs=1884904eea=killing@multiplay.co.uk) Received: from mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) by mx1.freebsd.org (Postfix) with ESMTP id 02EB8193D for ; Fri, 21 Jun 2013 16:37:39 +0000 (UTC) Received: from r2d2 ([82.69.141.170]) by mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) (MDaemon PRO v10.0.4) with ESMTP id md50004457656.msg for ; Fri, 21 Jun 2013 17:37:37 +0100 X-Spam-Processed: mail1.multiplay.co.uk, Fri, 21 Jun 2013 17:37:37 +0100 (not processed: message from valid local sender) X-MDDKIM-Result: neutral (mail1.multiplay.co.uk) X-MDRemoteIP: 82.69.141.170 X-Return-Path: prvs=1884904eea=killing@multiplay.co.uk X-Envelope-From: killing@multiplay.co.uk X-MDaemon-Deliver-To: zfs-devel@freebsd.org Message-ID: From: "Steven Hartland" To: "Will Andrews" References: <67F372464AA94883B383D2BDDCC4C9B2@multiplay.co.uk> <16E7A29DFC6445C49A2B363C23BA62ED@multiplay.co.uk> Subject: Re: STF ZFS test suite, part 2 Date: Fri, 21 Jun 2013 17:37:46 +0100 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="iso-8859-1"; reply-type=original Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.5931 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.6157 Cc: zfs-devel@freebsd.org X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Jun 2013 16:37:40 -0000 ----- Original Message ----- From: "Will Andrews" To: "Steven Hartland" Cc: Sent: Friday, June 21, 2013 4:32 PM Subject: Re: STF ZFS test suite, part 2 > On Fri, Jun 21, 2013 at 9:22 AM, Steven Hartland >> == Failed test cases == >> -cli_root/zfs_rename/zfs_rename:zfs_rename_007_pos >> + failed: clone not creating zvol entry >> -cli_root/zfs_set/zfs_set:readonly_001_pos >> + failed: zvol clone bug >> -zil/zil:zil_001_pos >> + failed: broken freeze ioctl >> -zil/zil:zil_002_pos >> + failed: broken freeze ioctl >> -zvol/zvol_misc/zvol_misc:zvol_misc_007_pos >> + failed: recursive zvol rename fails >> -zvol/zvol_misc/zvol_misc:zvol_misc_008_pos >> + failed: zvol rename in promote >> -zvol/zvol_misc/zvol_misc:zvol_misc_009_pos >> + failed: recursive zvol rename fails >> ?zvol_thrash/zvol_thrash:zvol_thrash_001_pos >> + failed: uses camcontrol which fails on md's >> #history/history:history_008_pos >> + failed: missing history >> #history/history:history_009_pos >> + failed: missing history >> #history/history:history_010_pos >> + failed: missing history > > These are all fixed in SpectraBSD. I specifically wrote > zvol_misc_00[789]_pos for that purpose. On Solaris, they use vnops > which don't have the problem of having to populate a device tree at > all times. But FreeBSD uses geoms which have other advantages. I have the zvol_misc_00[789]_pos fixed here too, so be good to have a look at your patches for these, compare with what we have and get the fixes into the tree :) Did you also address the geom and zvol thread deadlock? Regards Steve ================================================ This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmaster@multiplay.co.uk. From owner-zfs-devel@FreeBSD.ORG Fri Jun 21 16:41:32 2013 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 722C5D85; Fri, 21 Jun 2013 16:41:32 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-qc0-x22f.google.com (mail-qc0-x22f.google.com [IPv6:2607:f8b0:400d:c01::22f]) by mx1.freebsd.org (Postfix) with ESMTP id 25AA11964; Fri, 21 Jun 2013 16:41:32 +0000 (UTC) Received: by mail-qc0-f175.google.com with SMTP id k14so4803153qcv.34 for ; Fri, 21 Jun 2013 09:41:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; bh=mfpIddF+V74q+9OYWxyFVb40Vg0HOZySakqvA0JIUyI=; b=QRGurN+53db5MJscILymC4NVhCLMwM1rdLjicP/JRlKLvlhwddeyo+JulhbLDzhqXz OMu5M6Bvmg8ROWsMKp0OZU6QVi2Rwi6f6Q77Ie7XH6wYIOqnKtdslOdMzHppwL0GOaFv HPpRUfhsAO6RR9Bjwdf+2qqUZEoP9vAvvmnTCynvJ+MUPsKTQkC7W5b2BG5XDQgMMh9M uur9ssk3Vv2AQDTTadZTSK/YN3QjprekZkkYp1Tv+/uBAFaXejLq6/f0Yle4KZ5OaW7g 1b2MV+tS2wZWxMmAiAyQNQHJc7zBbK18WrwIWMxqGCRleanW7Yp4fLikoZIDUqX12ler XBpg== MIME-Version: 1.0 X-Received: by 10.224.183.142 with SMTP id cg14mr15231045qab.27.1371832891690; Fri, 21 Jun 2013 09:41:31 -0700 (PDT) Sender: asomers@gmail.com Received: by 10.49.37.226 with HTTP; Fri, 21 Jun 2013 09:41:31 -0700 (PDT) In-Reply-To: <16E7A29DFC6445C49A2B363C23BA62ED@multiplay.co.uk> References: <67F372464AA94883B383D2BDDCC4C9B2@multiplay.co.uk> <16E7A29DFC6445C49A2B363C23BA62ED@multiplay.co.uk> Date: Fri, 21 Jun 2013 10:41:31 -0600 X-Google-Sender-Auth: NFbJNhxqkHr8gCJdEtWbqc9LonQ Message-ID: Subject: Re: STF ZFS test suite, part 2 From: Alan Somers To: Steven Hartland Content-Type: text/plain; charset=ISO-8859-1 Cc: zfs-devel@freebsd.org X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Jun 2013 16:41:32 -0000 On Fri, Jun 21, 2013 at 9:22 AM, Steven Hartland wrote: > ----- Original Message ----- From: "Alan Somers" > >>>>> This appears to be an issue with using /tmp as the target dir, using >>>>> another directory and I can run the import without issue it seems. >>>>> Other tests also hang with the same issue:- >>>>> zpool_upgrade_002_pos >>>>> zpool_upgrade_003_pos >>>>> zpool_upgrade_007_pos >>>>> zpool_upgrade_008_pos >>>>> zpool_upgrade_009_neg >>>>> >>>>> Would it be an issue to change the directory? >>>> >>>> >>>> You should be able to use any output directory at all. Also, I >>>> usually put TMPDIR on its own UFS filesystem because it makes the >>>> hotspare tests go faster. >>> >>> >>> >>> I've confirmed changing the default TMPDIR fixes ths issue. I believe >>> the problem is in /tmp there's some fifo's, so possibly some bad >>> interaction with those, that and I'm sure its not a good idea to mount >>> a pool over the main system tmp directory ;-) >> >> >> The tests shouldn't be mounting a pool on TMPDIR. If they are, then >> that's a bug. It's probably something like "mount foo >> ${TMPDIR}/${BAR}", where BAR is undefined. >> Can you tell which test is doing it? > > > Pretty much all of the zpool_upgrade_* tests I believe the function at > fault is create_old_pool which runs: > log_must $ZPOOL import -d /$TMPDIR $POOL_NAME That command shouldn't mount a pool over TMPDIR. It attempts to import a pool, looking in TMPIR for vdev files. I can run the zpool_upgrade tests just fine with TMPDIR=/tmp, though it takes 5 times longer than using a dedicated UFS filesystem for TMPIR. > > Some new issues:- > In the xml log I'm seeing a number of errors, which I believe are setup > problems which could well be causing issues with the tests e.g. > gpart: arg0 'md0': Invalid argument > > The following fixes that:- > --- ./include/libtest.kshlib.orig 2013-06-20 00:00:37.923125499 +0000 > +++ ./include/libtest.kshlib 2013-06-20 19:02:31.396756337 +0000 > @@ -635,7 +635,7 @@ function wipe_partition_table # { > while [[ -n $* ]]; do > typeset diskname=$1 > - $GPART destroy -F $diskname > + $GPART destroy -F $diskname > /dev/null 2>&1 > shift > done > } All that does is suppress the error message from gpart. It shouldn't affect the running of any test, unless the test evalutes wipe_partition_table's STDERR. That function is normally used during setup and cleanup, where it's unknown what partition table might be present on the disk. So gpart complains if it didn't find a partition table there when you try to destroy it. > > A number of tests fail as the ports mkfile is broken and exits 0 when > the file creation fails including:- > refquota_00*_pos, refreserv_001_pos, zpool_add_006_pos, > zpool_create_004_pos Oops. I forgot that we had already fixed that port, but not upstreamed the patch. We were pretty lazy about upstreaming in 2011 and 2012. But your patch looks better than ours. > > The patch attached fixes this, which I've submitted to the port maintainer. > > Here are my current results from stable/9, initally I was seeing 36 > unexpected > test failures but with some patches I'm now down to 19 of which I believe > only 4 are actual failures, details below. > > == Failure Key == > ! = Broken test > * = Test passed but expected failure > - = Test was broken but now passes with local fixes > ? = Possibly broken test > # = Real failure > > == Failed test cases == > !cli_root/zfs_snapshot/zfs_snapshot:zfs_snapshot_001_neg > + broken test: multiple snaps can be created > !cli_root/zpool_import/zpool_import:zpool_import_corrupt_001_pos > + broken test: dd oseek cant be negative These both work for us. Could you please show me your atf_results.xml file? > !cli_root/zpool_scrub/zpool_scrub:zpool_scrub_003_pos > + broken test: resliver completes before check for in progress can complete Yeah, this one is racy as you discovered. There are a few others like it. > !mmap/mmap_write/mmap_write:mmap_write_001_pos > + broken test: runs out of space Works for us. I'd have to see your atf_results.xml file > *cache/cache:cache_009_pos > + passed: expected failure A known bug in SpectraBSD > *cli_root/zfs_copies/zfs_copies:zfs_copies_003_pos > + passed: expected failure > *cli_root/zpool_destroy/zpool_destroy:zpool_destroy_001_pos > + passed: expected failure In order to fix some deadlocks, we disabled creating pools on zvols. Obviously, that's not an ideal fix that we would upstream. > *cli_root/zfs_create/zfs_create:zfs_create_013_pos > + passed: expected failure > *cli_root/zfs_get/zfs_get:zfs_get_009_pos > + passed: expected failure These failures are due to FreeBSD's mount path length limitation. If it works for you, then the failure might be the result of our zvol changes which slightly enlarge the name of the zvol device. > *hotspare/hotspare:hotspare_add_004_neg > + passed: expected failure > ?inuse/inuse:inuse_005_pos > + passed: expected failure, could be a broken test as everything reports > SUCCESS? The "zpool add" operation failed for you, but not for the reason that the test was designed to catch. I think it failed because ZFS opens geom providers in exclusive mode. But in SpectraBSD, we disabled that to fix a swath of other hotspare bugs. Hotspare support in ZFS is quite poorly designed. Oracle doesn't even enable it in their storage appliances. We've fixed many hotspare bugs, but it was necessary to open geoms nonexclusively. > *pool_names/pool_names:pool_names_001_pos > + passed: expected failure Surprising. I didn't think that SpectraBSD had regressed in this area. We'll have to look into it. > -cli_root/zfs_rename/zfs_rename:zfs_rename_007_pos > + failed: clone not creating zvol entry > -cli_root/zfs_set/zfs_set:readonly_001_pos > + failed: zvol clone bug > -zil/zil:zil_001_pos > + failed: broken freeze ioctl > -zil/zil:zil_002_pos > + failed: broken freeze ioctl > -zvol/zvol_misc/zvol_misc:zvol_misc_007_pos > + failed: recursive zvol rename fails > -zvol/zvol_misc/zvol_misc:zvol_misc_008_pos > + failed: zvol rename in promote > -zvol/zvol_misc/zvol_misc:zvol_misc_009_pos > + failed: recursive zvol rename fails > ?zvol_thrash/zvol_thrash:zvol_thrash_001_pos > + failed: uses camcontrol which fails on md's > #history/history:history_008_pos + failed: missing history > #history/history:history_009_pos > + failed: missing history > #history/history:history_010_pos > + failed: missing history > #snapshot/snapshot:snapshot_018_pos > + failed: unknown sysctl abbreviated_snapdir What Will wrote. > -cli_root/zfs_set/zfs_set:ro_props_001_pos > + failed: fixed by mkfile patch > -cli_root/zpool_add/zpool_add:zpool_add_006_pos > + failed: fixed by mkfile patch > -cli_root/zpool_create/zpool_create:zpool_create_004_pos > + failed: fixed by mkfile patch > -refquota/refquota:refquota_001_pos > + failed: mkfile doesn't check for any errors on write > -refquota/refquota:refquota_002_pos > + failed: mkfile doesn't check for any errors on write > -refquota/refquota:refquota_003_pos > + failed: mkfile doesn't check for any errors on write > -refquota/refquota:refquota_004_pos > + failed: mkfile doesn't check for any errors on write > -refquota/refquota:refquota_005_pos > + failed: mkfile doesn't check for any errors on write > -refreserv/refreserv:refreserv_001_pos > + failed: mkfile doesn't check for any errors on write > -snapshot/snapshot:snapshot_014_pos > + failed: mkfile doesn't check for any errors on write Sorry about not upstreaming that earlier > -cli_root/zpool_import/zpool_import:zpool_import_014_pos > + failed: cant import deleted pools with -D Again, we've already fixed that but failed to upstream the patch. I see that you fixed it yourself with a near-identical patch. > ?cli_root/zpool_create/zpool_create:zpool_create_005_pos + failed: possibly > broken test, everything reports SUCCESS? There are some annoying tests which don't print error messages for their failures. This one passes for us; I'd have to see your atf_results.xml to figure out why it fails for you. > > == Before fixes === > Summary for 102 test programs: > 393 passed test cases. > 36 failed test cases. > 30 expected failed test cases. > 173 skipped test cases. > > == After fixes == > Summary for 102 test programs: > 411 passed test cases. > 18 failed test cases. > 30 expected failed test cases. > 173 skipped test cases. > > == Time taken for test suite == > 1h38m32.62s real 7m6.98s user 36m3.05s sys > > I'll be committing the fixes I've found as soon as full head tests are > complete. > > > Regards > Steve > > ================================================ > This e.mail is private and confidential between Multiplay (UK) Ltd. and the > person or entity to whom it is addressed. In the event of misdirection, the > recipient is prohibited from using, copying, printing or otherwise > disseminating it or any information contained in it. > In the event of misdirection, illegible or incomplete transmission please > telephone +44 845 868 1337 > or return the E.mail to postmaster@multiplay.co.uk. From owner-zfs-devel@FreeBSD.ORG Fri Jun 21 16:50:36 2013 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id C1474E88 for ; Fri, 21 Jun 2013 16:50:36 +0000 (UTC) (envelope-from will@firepipe.net) Received: from mail-ve0-x22e.google.com (mail-ve0-x22e.google.com [IPv6:2607:f8b0:400c:c01::22e]) by mx1.freebsd.org (Postfix) with ESMTP id 7DF1919C9 for ; Fri, 21 Jun 2013 16:50:36 +0000 (UTC) Received: by mail-ve0-f174.google.com with SMTP id oz10so6685524veb.33 for ; Fri, 21 Jun 2013 09:50:36 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:x-gm-message-state; bh=+UDiVANOdYgVmbfG+odgu1Af4R0c/BQj4uHYgwua5zM=; b=hg1WZ++iaiOO2bMOniB/FSWp1vLIf2fbyjk+qtdJNJJhSd+e9yJY9gTdpQQaEwsppw NWvNQtkjGX6rOk9okxMRS0l7t7FJEMlSr0ErIYB7i9HcHsOEqBowPMvuNIHuvNDJjhcx RTCcKRV6QxWOmHpzy8G2vArIf3B7HSH2L7izET7EUyUyf+2LRRSx6+g4YP3JU9Py+pCc QS3aff6bI2WaeNsehtS3DTCe8FSXYdPfK6Hn7AMr/GciSWmBiBjVC6jZGTkXMtMOitll lJtVdHnnUAt1kcWSO4IlO1RfQtNf+AitRgbSzDtQhJgfyxEu/EWQzFkWVhxL/Mh5Mnos THSA== MIME-Version: 1.0 X-Received: by 10.52.70.211 with SMTP id o19mr5226124vdu.90.1371833436050; Fri, 21 Jun 2013 09:50:36 -0700 (PDT) Received: by 10.58.226.66 with HTTP; Fri, 21 Jun 2013 09:50:35 -0700 (PDT) In-Reply-To: References: <67F372464AA94883B383D2BDDCC4C9B2@multiplay.co.uk> <16E7A29DFC6445C49A2B363C23BA62ED@multiplay.co.uk> Date: Fri, 21 Jun 2013 10:50:35 -0600 Message-ID: Subject: Re: STF ZFS test suite, part 2 From: Will Andrews To: Steven Hartland Content-Type: text/plain; charset=UTF-8 X-Gm-Message-State: ALoCoQl9jSC2451udb1jo7aqb1ipvslFCsDcNpS1AaioRK5DN6lFy8vGsZTZIQh422VZSWXEzqMS Cc: zfs-devel@freebsd.org X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Jun 2013 16:50:36 -0000 On Fri, Jun 21, 2013 at 10:37 AM, Steven Hartland wrote: > I have the zvol_misc_00[789]_pos fixed here too, so be good to have a > look at your patches for these, compare with what we have and get the > fixes into the tree :) My changes for these issues are mixed in with many other changes to the zvol code and so are likely non-trivial to separate. I haven't gotten far enough in our upstreaming efforts to reach this part of the change stack. Review & help wanted pushing things to illumos. > Did you also address the geom and zvol thread deadlock? Could you elaborate on this deadlock? I did address several zvol deadlocks. --Will. From owner-zfs-devel@FreeBSD.ORG Fri Jun 21 17:13:28 2013 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 7EC981EC for ; Fri, 21 Jun 2013 17:13:28 +0000 (UTC) (envelope-from prvs=1884904eea=killing@multiplay.co.uk) Received: from mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) by mx1.freebsd.org (Postfix) with ESMTP id 201CF1ADD for ; Fri, 21 Jun 2013 17:13:27 +0000 (UTC) Received: from r2d2 ([82.69.141.170]) by mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) (MDaemon PRO v10.0.4) with ESMTP id md50004458101.msg for ; Fri, 21 Jun 2013 18:13:27 +0100 X-Spam-Processed: mail1.multiplay.co.uk, Fri, 21 Jun 2013 18:13:27 +0100 (not processed: message from valid local sender) X-MDDKIM-Result: neutral (mail1.multiplay.co.uk) X-MDRemoteIP: 82.69.141.170 X-Return-Path: prvs=1884904eea=killing@multiplay.co.uk X-Envelope-From: killing@multiplay.co.uk X-MDaemon-Deliver-To: zfs-devel@freebsd.org Message-ID: <7F8DCC08ED3B4A07B578C69C13A9493C@multiplay.co.uk> From: "Steven Hartland" To: "Will Andrews" References: <67F372464AA94883B383D2BDDCC4C9B2@multiplay.co.uk><16E7A29DFC6445C49A2B363C23BA62ED@multiplay.co.uk> Subject: Re: STF ZFS test suite, part 2 Date: Fri, 21 Jun 2013 18:13:39 +0100 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="UTF-8"; reply-type=original Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.5931 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.6157 Cc: zfs-devel@freebsd.org X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Jun 2013 17:13:28 -0000 ----- Original Message ----- From: "Will Andrews" > On Fri, Jun 21, 2013 at 10:37 AM, Steven Hartland >> I have the zvol_misc_00[789]_pos fixed here too, so be good to have a >> look at your patches for these, compare with what we have and get the >> fixes into the tree :) > > My changes for these issues are mixed in with many other changes to > the zvol code and so are likely non-trivial to separate. I haven't > gotten far enough in our upstreaming efforts to reach this part of the > change stack. Review & help wanted pushing things to illumos. LMK illumos issues / webrev's URL's, and I'll do what I can but I wouldn't let illumos stop you pushing known good changes to the FreeBSD tree ;-) >> Did you also address the geom and zvol thread deadlock? > > Could you elaborate on this deadlock? I did address several zvol deadlocks. http://www.freebsd.org/cgi/query-pr.cgi?pr=161968 I've addressed them here, but its quite a belt and braces approach so would be interested to see other approaches. Regards Steve ================================================ This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmaster@multiplay.co.uk. From owner-zfs-devel@FreeBSD.ORG Mon Jun 24 11:08:27 2013 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 84747614 for ; Mon, 24 Jun 2013 11:08:27 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) by mx1.freebsd.org (Postfix) with ESMTP id 5CED81F07 for ; Mon, 24 Jun 2013 11:08:27 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.7/8.14.7) with ESMTP id r5OB8R3g003061 for ; Mon, 24 Jun 2013 11:08:27 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.7/8.14.7/Submit) id r5OB8QiJ003059 for zfs-devel@FreeBSD.org; Mon, 24 Jun 2013 11:08:26 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 24 Jun 2013 11:08:26 GMT Message-Id: <201306241108.r5OB8QiJ003059@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: zfs-devel@FreeBSD.org Subject: Current problem reports assigned to zfs-devel@FreeBSD.org X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 Jun 2013 11:08:27 -0000 Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- s kern/178467 zfs-devel [zfs] [request] Optimized Checksum Code for ZFS o kern/178387 zfs-devel [zfs] [patch] sparse files performance improvements 2 problems total. From owner-zfs-devel@FreeBSD.ORG Mon Jun 24 15:30:01 2013 Return-Path: Delivered-To: zfs-devel@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 62CACC83 for ; Mon, 24 Jun 2013 15:30:01 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) by mx1.freebsd.org (Postfix) with ESMTP id 5529C1DCC for ; Mon, 24 Jun 2013 15:30:01 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.7/8.14.7) with ESMTP id r5OFU1hj054610 for ; Mon, 24 Jun 2013 15:30:01 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.7/8.14.7/Submit) id r5OFU1n9054609; Mon, 24 Jun 2013 15:30:01 GMT (envelope-from gnats) Date: Mon, 24 Jun 2013 15:30:01 GMT Message-Id: <201306241530.r5OFU1n9054609@freefall.freebsd.org> To: zfs-devel@FreeBSD.org Cc: From: Alan Somers Subject: Re: kern/178467: [zfs] [request] Optimized Checksum Code for ZFS X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list Reply-To: Alan Somers List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 Jun 2013 15:30:01 -0000 The following reply was made to PR kern/178467; it has been noted by GNATS. From: Alan Somers To: bug-followup@FreeBSD.org, jkeller@bbiinternational.com Cc: Subject: Re: kern/178467: [zfs] [request] Optimized Checksum Code for ZFS Date: Mon, 24 Jun 2013 09:28:08 -0600 FWIW, I spent a full day trying to accelerate Fletcher-4 using SIMD instructions (tested on Sandy Bridge and Nehalem). I was unable to improve on the current code; the Fletcher-4 hash is very fast and doesn't vectorize well. However, I believe that AVX-2 will probably be able to beat the non-vectorized version. I plan to try it out as soon as I can get my hands on a Haswell CPU. I've also spent several weeks analyzing the strength of Fletcher-4, and concluded that it's really quite good. Good enough for every non-cryptographic application, certainly. My recommendation is that all ZFS users should prefer Fletcher-4 over SHA-256. I haven't tried vectorizing SHA-256 and don't plan to. From owner-zfs-devel@FreeBSD.ORG Mon Jun 24 15:49:06 2013 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 2C5C025D for ; Mon, 24 Jun 2013 15:49:06 +0000 (UTC) (envelope-from will@firepipe.net) Received: from mail-ve0-x234.google.com (mail-ve0-x234.google.com [IPv6:2607:f8b0:400c:c01::234]) by mx1.freebsd.org (Postfix) with ESMTP id DF9D21EBC for ; Mon, 24 Jun 2013 15:49:05 +0000 (UTC) Received: by mail-ve0-f180.google.com with SMTP id pa12so8946038veb.11 for ; Mon, 24 Jun 2013 08:49:05 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:x-gm-message-state; bh=e+isS2o4PBlZWbUzqFsSKWgr99puuqRFYbq/oZMO/s4=; b=j7bZb+ZnhyYxzPLA0SSpwyBBoSqBlUvO0Lg4q3In30B8k9ZDjwJRXacL1n9C3qAfdS OTD2JBN6zOtXqnHpl760g19pw79TlPORGOg81If2V0e4zbzubTrFaKmDr1JlyXjR3n4f QtniOHqILZL+Rfxm0G2wcCsw3lMiQn/uovLodrKgT/dHBX8BW0SDrlVAsOaAP8gZOjX4 IQtgnBjtEk+7CPBnjRROk7M0Kph3QszOM5OxLxhPIljkbm2T9LtbAI/dhcOJbVIdXgNz MX7WbsN4pyAHFyJ+NLKPcQC0qXFVgi6KnJMFvgHUZEk1UfUAC6WnJGxUNcraoPoRazNF LbmQ== MIME-Version: 1.0 X-Received: by 10.52.65.10 with SMTP id t10mr2343494vds.90.1372088945447; Mon, 24 Jun 2013 08:49:05 -0700 (PDT) Received: by 10.58.226.66 with HTTP; Mon, 24 Jun 2013 08:49:05 -0700 (PDT) In-Reply-To: <7F8DCC08ED3B4A07B578C69C13A9493C@multiplay.co.uk> References: <67F372464AA94883B383D2BDDCC4C9B2@multiplay.co.uk> <16E7A29DFC6445C49A2B363C23BA62ED@multiplay.co.uk> <7F8DCC08ED3B4A07B578C69C13A9493C@multiplay.co.uk> Date: Mon, 24 Jun 2013 09:49:05 -0600 Message-ID: Subject: Re: STF ZFS test suite, part 2 From: Will Andrews To: Steven Hartland Content-Type: text/plain; charset=UTF-8 X-Gm-Message-State: ALoCoQm6NO1c5bKZF40jW5iqpgeXgEGzGD+1hN/FH/btYUFC4ACaiVPoOOtJ9T9dJWsdgpY3JiPk Cc: zfs-devel@freebsd.org X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 Jun 2013 15:49:06 -0000 On Fri, Jun 21, 2013 at 11:13 AM, Steven Hartland wrote: > LMK illumos issues / webrev's URL's, and I'll do what I can but I wouldn't > let illumos stop you pushing known good changes to the FreeBSD tree > ;-) OK, here's the diffstat, of the branch I created to illumos-gate, normalized to compare FreeBSD stable/9 to SpectraBSD: usr/src/cmd/zdb/zdb.c | 7 +- usr/src/cmd/zfs/zfs_main.c | 41 +- usr/src/cmd/zpool/zpool_main.c | 126 ++++- usr/src/cmd/ztest/ztest.c | 299 +++++++----- usr/src/common/ctf/ctf_create.c | 2 +- usr/src/common/zfs/zfs_fletcher.c | 11 + usr/src/lib/libnvpair/libnvpair.c | 2 +- usr/src/lib/libzfs/common/libzfs.h | 1 + usr/src/lib/libzfs/common/libzfs_dataset.c | 17 +- usr/src/lib/libzfs/common/libzfs_diff.c | 4 +- usr/src/lib/libzfs/common/libzfs_import.c | 8 + usr/src/lib/libzfs/common/libzfs_mount.c | 21 +- usr/src/lib/libzfs/common/libzfs_pool.c | 40 +- usr/src/lib/libzfs/common/libzfs_sendrecv.c | 4 +- usr/src/lib/libzfs/common/libzfs_util.c | 74 ++- usr/src/lib/libzpool/common/kernel.c | 13 +- usr/src/lib/libzpool/common/sys/zfs_context.h | 72 ++- usr/src/lib/libzpool/common/taskq.c | 24 +- usr/src/lib/libzpool/common/util.c | 2 +- usr/src/uts/common/Makefile.files | 3 +- usr/src/uts/common/dtrace/fasttrap.c | 62 +-- usr/src/uts/common/fs/zfs/arc.c | 94 +++- usr/src/uts/common/fs/zfs/bptree.c | 2 +- usr/src/uts/common/fs/zfs/dbuf.c | 3256 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++-------------------------------- usr/src/uts/common/fs/zfs/dmu.c | 1265 ++++++++++++++++++++++++++++++++---------------- usr/src/uts/common/fs/zfs/dmu_objset.c | 115 ++--- usr/src/uts/common/fs/zfs/dmu_tx.c | 51 +- usr/src/uts/common/fs/zfs/dmu_zfetch.c | 4 +- usr/src/uts/common/fs/zfs/dnode.c | 105 ++-- usr/src/uts/common/fs/zfs/dnode_sync.c | 49 +- usr/src/uts/common/fs/zfs/dsl_dataset.c | 156 +++--- usr/src/uts/common/fs/zfs/dsl_deleg.c | 4 +- usr/src/uts/common/fs/zfs/dsl_dir.c | 29 +- usr/src/uts/common/fs/zfs/dsl_pool.c | 13 + usr/src/uts/common/fs/zfs/dsl_prop.c | 16 +- usr/src/uts/common/fs/zfs/dsl_scan.c | 7 +- usr/src/uts/common/fs/zfs/metaslab.c | 31 +- usr/src/uts/common/fs/zfs/refcount.c | 28 +- usr/src/uts/common/fs/zfs/sa.c | 41 +- usr/src/uts/common/fs/zfs/spa.c | 277 ++++++++--- usr/src/uts/common/fs/zfs/spa_config.c | 53 +- usr/src/uts/common/fs/zfs/spa_errlog.c | 4 +- usr/src/uts/common/fs/zfs/spa_history.c | 8 +- usr/src/uts/common/fs/zfs/spa_misc.c | 104 +++- usr/src/uts/common/fs/zfs/space_map.c | 27 +- usr/src/uts/common/fs/zfs/sys/arc.h | 12 + usr/src/uts/common/fs/zfs/sys/dbuf.h | 352 ++++++++++++-- usr/src/uts/common/fs/zfs/sys/ddt.h | 13 +- usr/src/uts/common/fs/zfs/sys/dmu.h | 360 +++++++++++--- usr/src/uts/common/fs/zfs/sys/dmu_objset.h | 2 +- usr/src/uts/common/fs/zfs/sys/dmu_tx.h | 8 +- usr/src/uts/common/fs/zfs/sys/dnode.h | 19 +- usr/src/uts/common/fs/zfs/sys/dsl_dataset.h | 54 ++- usr/src/uts/common/fs/zfs/sys/dsl_dir.h | 17 +- usr/src/uts/common/fs/zfs/sys/dsl_pool.h | 1 + usr/src/uts/common/fs/zfs/sys/metaslab.h | 1 + usr/src/uts/common/fs/zfs/sys/metaslab_impl.h | 1 + usr/src/uts/common/fs/zfs/sys/refcount.h | 2 +- usr/src/uts/common/fs/zfs/sys/sa.h | 10 +- usr/src/uts/common/fs/zfs/sys/sa_impl.h | 42 +- usr/src/uts/common/fs/zfs/sys/spa.h | 9 + usr/src/uts/common/fs/zfs/sys/spa_impl.h | 13 +- usr/src/uts/common/fs/zfs/sys/space_map.h | 1 - usr/src/uts/common/fs/zfs/sys/unique.h | 2 +- usr/src/uts/common/fs/zfs/sys/vdev_impl.h | 33 +- usr/src/uts/common/fs/zfs/sys/zap.h | 42 +- usr/src/uts/common/fs/zfs/sys/zap_impl.h | 39 +- usr/src/uts/common/fs/zfs/sys/zap_leaf.h | 33 +- usr/src/uts/common/fs/zfs/sys/zfs_acl.h | 14 +- usr/src/uts/common/fs/zfs/sys/zfs_ctldir.h | 24 +- usr/src/uts/common/fs/zfs/sys/zfs_debug.h | 28 ++ usr/src/uts/common/fs/zfs/sys/zfs_fuid.h | 6 + usr/src/uts/common/fs/zfs/sys/zfs_ioctl.h | 37 +- usr/src/uts/common/fs/zfs/sys/zfs_rlock.h | 15 +- usr/src/uts/common/fs/zfs/sys/zfs_znode.h | 23 +- usr/src/uts/common/fs/zfs/sys/zil.h | 14 +- usr/src/uts/common/fs/zfs/sys/zio.h | 3 + usr/src/uts/common/fs/zfs/sys/zio_compress.h | 13 +- usr/src/uts/common/fs/zfs/sys/zvol.h | 16 +- usr/src/uts/common/fs/zfs/txg.c | 48 +- usr/src/uts/common/fs/zfs/vdev.c | 104 ++-- usr/src/uts/common/fs/zfs/vdev_file.c | 5 +- usr/src/uts/common/fs/zfs/vdev_label.c | 1 + usr/src/uts/common/fs/zfs/vdev_mirror.c | 60 ++- usr/src/uts/common/fs/zfs/vdev_missing.c | 5 +- usr/src/uts/common/fs/zfs/vdev_queue.c | 11 +- usr/src/uts/common/fs/zfs/vdev_raidz.c | 80 ++- usr/src/uts/common/fs/zfs/vdev_root.c | 5 +- usr/src/uts/common/fs/zfs/zap.c | 186 +++---- usr/src/uts/common/fs/zfs/zap_leaf.c | 4 +- usr/src/uts/common/fs/zfs/zap_micro.c | 41 +- usr/src/uts/common/fs/zfs/zfs_acl.c | 389 ++++++++++++++- usr/src/uts/common/fs/zfs/zfs_ctldir.c | 194 +++++++- usr/src/uts/common/fs/zfs/zfs_dir.c | 7 +- usr/src/uts/common/fs/zfs/zfs_fuid.c | 51 +- usr/src/uts/common/fs/zfs/zfs_ioctl.c | 160 +++--- usr/src/uts/common/fs/zfs/zfs_log.c | 75 ++- usr/src/uts/common/fs/zfs/zfs_onexit.c | 1 + usr/src/uts/common/fs/zfs/zfs_replay.c | 27 +- usr/src/uts/common/fs/zfs/zfs_rlock.c | 15 +- usr/src/uts/common/fs/zfs/zfs_sa.c | 2 +- usr/src/uts/common/fs/zfs/zfs_vfsops.c | 34 +- usr/src/uts/common/fs/zfs/zfs_vnops.c | 786 +++++++++++++++++++++++------- usr/src/uts/common/fs/zfs/zfs_znode.c | 119 +++-- usr/src/uts/common/fs/zfs/zil.c | 8 +- usr/src/uts/common/fs/zfs/zio.c | 38 +- usr/src/uts/common/fs/zfs/zio_compress.c | 68 ++- usr/src/uts/common/fs/zfs/zrlock.c | 45 +- usr/src/uts/common/fs/zfs/zvol.c | 2743 ++++++++++++++++++++++++++++++++++-------------------------------------------------------------------- usr/src/uts/common/sys/acl.h | 8 +- usr/src/uts/common/sys/fm/fs/zfs.h | 1 + usr/src/uts/common/sys/fs/zfs.h | 1 + usr/src/uts/common/sys/sysevent/dev.h | 1 + usr/src/uts/common/sys/taskq.h | 6 +- So, you're telling me that if I simply commit all that to FreeBSD/head, no one will rip my head off? --Will. From owner-zfs-devel@FreeBSD.ORG Mon Jun 24 16:30:01 2013 Return-Path: Delivered-To: zfs-devel@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 893C5D52 for ; Mon, 24 Jun 2013 16:30:01 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) by mx1.freebsd.org (Postfix) with ESMTP id 7B52110F4 for ; Mon, 24 Jun 2013 16:30:01 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.7/8.14.7) with ESMTP id r5OGU14l065908 for ; Mon, 24 Jun 2013 16:30:01 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.7/8.14.7/Submit) id r5OGU0mx065907; Mon, 24 Jun 2013 16:30:00 GMT (envelope-from gnats) Date: Mon, 24 Jun 2013 16:30:00 GMT Message-Id: <201306241630.r5OGU0mx065907@freefall.freebsd.org> To: zfs-devel@FreeBSD.org Cc: From: Jason Keller Subject: RE: kern/178467: [zfs] [request] Optimized Checksum Code for ZFS X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list Reply-To: Jason Keller List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 Jun 2013 16:30:01 -0000 The following reply was made to PR kern/178467; it has been noted by GNATS. From: Jason Keller To: Alan Somers , "bug-followup@FreeBSD.org" Cc: Subject: RE: kern/178467: [zfs] [request] Optimized Checksum Code for ZFS Date: Mon, 24 Jun 2013 16:20:35 +0000 Thank you very much for following up on this. Any further optimization of = checksums is gravy, for everyone that uses FreeBSD/ZFS. Sure Fletcher4 is = pretty light, but every little bit helps. Fewer CPU cycles used =3D less l= atency =3D more win. I think for crypto/dedup applications, perhaps effort= should be focused on an optimized implementation of Keccak (SHA3 winner) i= nstead of SHA256? From owner-zfs-devel@FreeBSD.ORG Sat Jun 29 23:47:11 2013 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 508F0A06 for ; Sat, 29 Jun 2013 23:47:11 +0000 (UTC) (envelope-from smh@freebsd.org) Received: from smtp1.multiplay.co.uk (smtp1.multiplay.co.uk [85.236.96.35]) by mx1.freebsd.org (Postfix) with ESMTP id 04CA11A06 for ; Sat, 29 Jun 2013 23:47:10 +0000 (UTC) Received: by smtp1.multiplay.co.uk (Postfix, from userid 65534) id 2602C20E7088A; Sat, 29 Jun 2013 23:47:03 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.multiplay.co.uk X-Spam-Level: X-Spam-Status: No, score=-2.4 required=8.0 tests=ALL_TRUSTED,AWL,BAYES_00, MIME_QP_LONG_LINE autolearn=ham version=3.3.1 Received: from r2d2 (82-69-141-170.dsl.in-addr.zen.co.uk [82.69.141.170]) by smtp1.multiplay.co.uk (Postfix) with ESMTPA id 9596920E70847 for ; Sat, 29 Jun 2013 23:47:01 +0000 (UTC) Message-ID: <5AD89226AC69454DA8297F8C90A4E7E2@multiplay.co.uk> From: "Steven Hartland" To: Subject: ZFS_DEBUG + enable ZFS ASSERTS with INVARIANTS any objections? Date: Sun, 30 Jun 2013 00:47:11 +0100 MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="----=_NextPart_000_04A5_01CE752B.5B0BDA30" X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.5931 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.6157 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 29 Jun 2013 23:47:11 -0000 This is a multi-part message in MIME format. ------=_NextPart_000_04A5_01CE752B.5B0BDA30 Content-Type: text/plain; format=flowed; charset="iso-8859-1"; reply-type=original Content-Transfer-Encoding: 7bit The attached patch adds a ZFS_DEBUG option as well as enabling ZFS ASSERTS by default when the kernel is compiled with INVARIANTS. This should enable us to get better test coverage from CURRENT users and has already enabled me to catch a number of issues when testing code + fix for invalid ASSERT removed by r252390. So the question is there any objections to this patch? Regards Steve ------=_NextPart_000_04A5_01CE752B.5B0BDA30 Content-Type: application/octet-stream; name="zfs-debug.patch" Content-Transfer-Encoding: quoted-printable Content-Disposition: attachment; filename="zfs-debug.patch" Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dbuf.c=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dbuf.c (revision = 252381)=0A= +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dbuf.c (working copy)=0A= @@ -1882,7 +1882,9 @@=0A= ASSERT3P(db->db.db_data, =3D=3D, db->db_buf->b_data);=0A= }=0A= =0A= +#ifdef ZFS_DEBUG=0A= ASSERT(db->db_buf =3D=3D NULL || arc_referenced(db->db_buf));=0A= +#endif=0A= =0A= /*=0A= * If this buffer is currently syncing out, and we are are=0A= Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dnode.c=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dnode.c (revision = 252381)=0A= +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dnode.c (working copy)=0A= @@ -1191,7 +1191,9 @@=0A= * other direct or indirect hold on the dnode must first drop the dnode=0A= * handle.=0A= */=0A= +#ifdef ZFS_DEBUG=0A= ASSERT(refs > 0 || dnh->dnh_zrlock.zr_owner !=3D curthread);=0A= +#endif=0A= =0A= /* NOTE: the DNODE_DNODE does not have a dn_dbuf */=0A= if (refs =3D=3D 0 && db !=3D NULL) {=0A= Index: sys/cddl/contrib/opensolaris/uts/common/sys/debug.h=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/cddl/contrib/opensolaris/uts/common/sys/debug.h (revision 252381)=0A= +++ sys/cddl/contrib/opensolaris/uts/common/sys/debug.h (working copy)=0A= @@ -40,6 +40,12 @@=0A= extern "C" {=0A= #endif=0A= =0A= +/* When running with INVARIANTS temporarily define DEBUG to enable = ASSERTS. */=0A= +#if !defined(DEBUG) && defined(_KERNEL) && defined(INVARIANTS)=0A= +#define _OPENSOLARIS_INVARIANTS 1=0A= +#define DEBUG 1=0A= +#endif=0A= +=0A= /*=0A= * ASSERT(ex) causes a panic or debugger entry if expression ex is not=0A= * true. ASSERT() is included only for debugging, and is a no-op in=0A= @@ -146,6 +152,12 @@=0A= #define STATIC static=0A= #endif=0A= =0A= +#if defined(_OPENSOLARIS_INVARIANTS)=0A= +/* Disable temporary DEBUG. */=0A= +#undef DEBUG=0A= +#undef _OPENSOLARIS_INVARIANTS=0A= +#endif=0A= +=0A= #ifdef __cplusplus=0A= }=0A= #endif=0A= Index: sys/modules/opensolaris/Makefile=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/modules/opensolaris/Makefile (revision 252381)=0A= +++ sys/modules/opensolaris/Makefile (working copy)=0A= @@ -24,6 +24,10 @@=0A= -I${.CURDIR}/../../cddl/contrib/opensolaris/uts/common \=0A= -I${.CURDIR}/../..=0A= =0A= +.if defined(ZFS_DEBUG)=0A= +CFLAGS+=3D -DDEBUG=0A= +.endif=0A= +=0A= IGNORE_PRAGMA=3D 1=0A= =0A= .include =0A= Index: sys/modules/zfs/Makefile=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/modules/zfs/Makefile (revision 252381)=0A= +++ sys/modules/zfs/Makefile (working copy)=0A= @@ -91,8 +91,9 @@=0A= CFLAGS+=3D-mminimal-toc=0A= .endif=0A= =0A= -#CFLAGS+=3D-DDEBUG=3D1=0A= -#DEBUG_FLAGS=3D-g=0A= +.if defined(ZFS_DEBUG)=0A= +CFLAGS+=3D -DDEBUG=0A= +.endif=0A= =0A= .include =0A= =0A= ------=_NextPart_000_04A5_01CE752B.5B0BDA30-- From owner-zfs-devel@FreeBSD.ORG Mon Jul 1 03:09:11 2013 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id C75BBC55; Mon, 1 Jul 2013 03:09:11 +0000 (UTC) (envelope-from gibbs@FreeBSD.org) Received: from aslan.scsiguy.com (ns1.scsiguy.com [70.89.174.89]) by mx1.freebsd.org (Postfix) with ESMTP id 889E1191C; Mon, 1 Jul 2013 03:09:11 +0000 (UTC) Received: from [192.168.0.99] (macbook.scsiguy.com [192.168.0.99]) (authenticated bits=0) by aslan.scsiguy.com (8.14.7/8.14.5) with ESMTP id r6136GK1026037 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NO); Mon, 1 Jul 2013 03:06:20 GMT (envelope-from gibbs@FreeBSD.org) Subject: Re: ZFS_DEBUG + enable ZFS ASSERTS with INVARIANTS any objections? Mime-Version: 1.0 (Mac OS X Mail 6.5 \(1508\)) Content-Type: text/plain; charset=iso-8859-1 From: "Justin T. Gibbs" X-Priority: 3 In-Reply-To: <5AD89226AC69454DA8297F8C90A4E7E2@multiplay.co.uk> Date: Sun, 30 Jun 2013 21:06:17 -0600 Content-Transfer-Encoding: 7bit Message-Id: References: <5AD89226AC69454DA8297F8C90A4E7E2@multiplay.co.uk> To: "Steven Hartland" X-Mailer: Apple Mail (2.1508) X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.4.3 (aslan.scsiguy.com [70.89.174.89]); Mon, 01 Jul 2013 03:06:20 +0000 (UTC) Cc: zfs-devel@FreeBSD.org X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Jul 2013 03:09:11 -0000 On Jun 29, 2013, at 5:47 PM, "Steven Hartland" wrote: > The attached patch adds a ZFS_DEBUG option as well > as enabling ZFS ASSERTS by default when the kernel > is compiled with INVARIANTS. > > This should enable us to get better test coverage > from CURRENT users and has already enabled me to > catch a number of issues when testing code + fix > for invalid ASSERT removed by r252390. > > So the question is there any objections to this > patch? > > Regards > Steve_______________________________________________ > zfs-devel@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/zfs-devel > To unsubscribe, send any mail to "zfs-devel-unsubscribe@freebsd.org" The patch seems to be only a partial solution. If DEBUG is the OpenSolaris equivalent of our INVARIANTS, shouldnt DEBUG be defined in all contexts when INVARIANTS is enabled? Otherwise, ASSERTS that depend on state maintained in #if DEBUG code in a .c file will unexpectedly fail. Just leaving DEBUG defined upon exit of sys/debug.h isn't enough. It appears that dtrace.c uses DEBUG without including (at least directly) this header. This usage (just like INVARIANTS) should be supported. In your patch, everyone does gets DEBUG if ZFS_DEBUG is set. While convenient for us ZFS developers, this should also work for someone working on DTrace, or a bug in the OpenSolaris nvpair code, etc. So I'd argue that the option is misnamed for at least the OpenSolaris module. On possible solution to this problem would be to grep opt_global.h from within the Makefile for OpenSolaris-ish modules: INVARIANTS_ENABLED!= grep INVARIANTS ${KERNBUILDDIR}/opt_global.h || true .if !empty(INVARIANTS_ENABLED) CFLAGS += -DDEBUG .endif If ZFS_DEBUG were also made a global option, similar logic could fail the ZFS module build if ZFS_DEBUG is enabled without INVARIANTS. -- Justin From owner-zfs-devel@FreeBSD.ORG Mon Jul 1 11:08:29 2013 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 5C900B1B for ; Mon, 1 Jul 2013 11:08:29 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) by mx1.freebsd.org (Postfix) with ESMTP id 347471227 for ; Mon, 1 Jul 2013 11:08:29 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.7/8.14.7) with ESMTP id r61B8TWa087807 for ; Mon, 1 Jul 2013 11:08:29 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.7/8.14.7/Submit) id r61B8SUe087805 for zfs-devel@FreeBSD.org; Mon, 1 Jul 2013 11:08:28 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 1 Jul 2013 11:08:28 GMT Message-Id: <201307011108.r61B8SUe087805@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: zfs-devel@FreeBSD.org Subject: Current problem reports assigned to zfs-devel@FreeBSD.org X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Jul 2013 11:08:29 -0000 Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- s kern/178467 zfs-devel [zfs] [request] Optimized Checksum Code for ZFS o kern/178387 zfs-devel [zfs] [patch] sparse files performance improvements 2 problems total. From owner-zfs-devel@FreeBSD.ORG Mon Jul 1 12:04:18 2013 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 02CEC4EC; Mon, 1 Jul 2013 12:04:18 +0000 (UTC) (envelope-from prvs=1894c479ef=killing@multiplay.co.uk) Received: from mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) by mx1.freebsd.org (Postfix) with ESMTP id 608A817C0; Mon, 1 Jul 2013 12:04:17 +0000 (UTC) Received: from r2d2 ([82.69.141.170]) by mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) (MDaemon PRO v10.0.4) with ESMTP id md50004656177.msg; Mon, 01 Jul 2013 13:04:10 +0100 X-Spam-Processed: mail1.multiplay.co.uk, Mon, 01 Jul 2013 13:04:10 +0100 (not processed: message from valid local sender) X-MDDKIM-Result: neutral (mail1.multiplay.co.uk) X-MDRemoteIP: 82.69.141.170 X-Return-Path: prvs=1894c479ef=killing@multiplay.co.uk X-Envelope-From: killing@multiplay.co.uk Message-ID: From: "Steven Hartland" To: "Justin T. Gibbs" References: <5AD89226AC69454DA8297F8C90A4E7E2@multiplay.co.uk> Subject: Re: ZFS_DEBUG + enable ZFS ASSERTS with INVARIANTS any objections? Date: Mon, 1 Jul 2013 13:04:20 +0100 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="iso-8859-1"; reply-type=original Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.5931 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.6157 Cc: zfs-devel@FreeBSD.org X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Jul 2013 12:04:18 -0000 ----- Original Message ----- From: "Justin T. Gibbs" To: "Steven Hartland" Cc: Sent: Monday, July 01, 2013 4:06 AM Subject: Re: ZFS_DEBUG + enable ZFS ASSERTS with INVARIANTS any objections? > On Jun 29, 2013, at 5:47 PM, "Steven Hartland" wrote: > >> The attached patch adds a ZFS_DEBUG option as well >> as enabling ZFS ASSERTS by default when the kernel >> is compiled with INVARIANTS. >> >> This should enable us to get better test coverage >> from CURRENT users and has already enabled me to >> catch a number of issues when testing code + fix >> for invalid ASSERT removed by r252390. >> >> So the question is there any objections to this >> patch? >> >> Regards >> Steve_______________________________________________ >> zfs-devel@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/zfs-devel >> To unsubscribe, send any mail to "zfs-devel-unsubscribe@freebsd.org" > > The patch seems to be only a partial solution. If DEBUG is the > OpenSolaris equivalent of our INVARIANTS, shouldnt DEBUG be defined > in all contexts when INVARIANTS is enabled? Otherwise, ASSERTS > that depend on state maintained in #if DEBUG code in a .c file will > unexpectedly fail. >From what I've seen ZFS code maintains additional state under ZFS_DEBUG not DEBUG although ZFS_DEBUG is itself triggered by DEBUG when compiling kernel code. This is why there's a few extra #ifdef ZFS_DEBUG's added so that ASSERTS which rely on this information aren't triggered. > Just leaving DEBUG defined upon exit of sys/debug.h isn't enough. > It appears that dtrace.c uses DEBUG without including (at least > directly) this header. This usage (just like INVARIANTS) should > be supported. This indeed could be done, however as I'm not a user of dtrace its not something I'm really setup to implement / test currently. > In your patch, everyone does gets DEBUG if ZFS_DEBUG is set. While > convenient for us ZFS developers, this should also work for someone > working on DTrace, or a bug in the OpenSolaris nvpair code, etc. > So I'd argue that the option is misnamed for at least the OpenSolaris > module. Maybe its best to remove the ZFS_DEBUG flag for the time being, as the key change is to get ASSERTS enabled under INVARIANTS? > On possible solution to this problem would be to grep opt_global.h > from within the Makefile for OpenSolaris-ish modules: > > INVARIANTS_ENABLED!= grep INVARIANTS ${KERNBUILDDIR}/opt_global.h || true > .if !empty(INVARIANTS_ENABLED) > CFLAGS += -DDEBUG > .endif > > If ZFS_DEBUG were also made a global option, similar logic could > fail the ZFS module build if ZFS_DEBUG is enabled without INVARIANTS. I'm not sure we want DEBUG enabled in modules triggered by INVARIANTS as this does add quite a bit more than just additional ASSERT checks, which as mentioned above is what the patch is designed to achieve. My reason for keeping it just to ASSERTs / VERIFYs was to gain the main benefit without enabling significant extra code. Perhaps adding an additional global flag such as OPENSOLARIS_DEBUG in addition to enabling ASSERT checks is the way to go? Regards Steve ================================================ This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmaster@multiplay.co.uk. From owner-zfs-devel@FreeBSD.ORG Mon Jul 1 15:34:49 2013 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id D0986274 for ; Mon, 1 Jul 2013 15:34:49 +0000 (UTC) (envelope-from gibbs@FreeBSD.org) Received: from aslan.scsiguy.com (aslan.scsiguy.com [70.89.174.89]) by mx1.freebsd.org (Postfix) with ESMTP id 9CC0D1102 for ; Mon, 1 Jul 2013 15:34:49 +0000 (UTC) Received: from [192.168.6.149] (207-225-98-3.dia.static.qwest.net [207.225.98.3]) (authenticated bits=0) by aslan.scsiguy.com (8.14.7/8.14.5) with ESMTP id r61FYlIb029366 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NO); Mon, 1 Jul 2013 15:34:48 GMT (envelope-from gibbs@FreeBSD.org) Subject: Re: ZFS_DEBUG + enable ZFS ASSERTS with INVARIANTS any objections? Mime-Version: 1.0 (Mac OS X Mail 6.5 \(1508\)) Content-Type: text/plain; charset=iso-8859-1 From: "Justin T. Gibbs" X-Priority: 3 In-Reply-To: Date: Mon, 1 Jul 2013 09:34:45 -0600 Content-Transfer-Encoding: quoted-printable Message-Id: References: <5AD89226AC69454DA8297F8C90A4E7E2@multiplay.co.uk> To: "Steven Hartland" X-Mailer: Apple Mail (2.1508) X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.4.3 (aslan.scsiguy.com [70.89.174.89]); Mon, 01 Jul 2013 15:34:48 +0000 (UTC) Cc: zfs-devel@FreeBSD.org X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Jul 2013 15:34:49 -0000 On Jul 1, 2013, at 6:04 AM, "Steven Hartland" = wrote: >=20 > ----- Original Message ----- From: "Justin T. Gibbs" = > To: "Steven Hartland" > Cc: > Sent: Monday, July 01, 2013 4:06 AM > Subject: Re: ZFS_DEBUG + enable ZFS ASSERTS with INVARIANTS any = objections? >=20 >=20 >> On Jun 29, 2013, at 5:47 PM, "Steven Hartland" = wrote: >>> The attached patch adds a ZFS_DEBUG option as well >>> as enabling ZFS ASSERTS by default when the kernel >>> is compiled with INVARIANTS. >>> This should enable us to get better test coverage >>> from CURRENT users and has already enabled me to >>> catch a number of issues when testing code + fix >>> for invalid ASSERT removed by r252390. >>> So the question is there any objections to this >>> patch? >>> Regards >>> = Steve_______________________________________________ >>> zfs-devel@freebsd.org mailing list >>> http://lists.freebsd.org/mailman/listinfo/zfs-devel >>> To unsubscribe, send any mail to "zfs-devel-unsubscribe@freebsd.org" >> The patch seems to be only a partial solution. If DEBUG is the >> OpenSolaris equivalent of our INVARIANTS, shouldnt DEBUG be defined >> in all contexts when INVARIANTS is enabled? Otherwise, ASSERTS >> that depend on state maintained in #if DEBUG code in a .c file will >> unexpectedly fail. >=20 > =46rom what I've seen ZFS code maintains additional state under = ZFS_DEBUG > not DEBUG although ZFS_DEBUG is itself triggered by DEBUG when = compiling > kernel code. >=20 > This is why there's a few extra #ifdef ZFS_DEBUG's added so that = ASSERTS > which rely on this information aren't triggered. Yes, I understand this. But there is no guarantee that this will continue to be true for future imports of ZFS or DTrace. I'm just trying to advocate for something that is easier to maintain than what you propose. >> Just leaving DEBUG defined upon exit of sys/debug.h isn't enough. >> It appears that dtrace.c uses DEBUG without including (at least >> directly) this header. This usage (just like INVARIANTS) should >> be supported. >=20 > This indeed could be done, however as I'm not a user of dtrace its > not something I'm really setup to implement / test currently. The goal of turning on INVARIANTS in head is to get crowd sourced coverage of diagnostic code. If this diagnostic code is currently broken, it will get noticed and fixed long before it hits production systems. We use DTrace at Spectra on ZFS_DEBUG kernels (so -DDEBUG is enabled in all "OpenSolaris" modules). We haven't noticed new bugs caused by this. I haven't tested on head recently, but if it does happen to be broken there, it would get noticed which would be a good thing! :-) >> In your patch, everyone does gets DEBUG if ZFS_DEBUG is set. While >> convenient for us ZFS developers, this should also work for someone >> working on DTrace, or a bug in the OpenSolaris nvpair code, etc. >> So I'd argue that the option is misnamed for at least the OpenSolaris >> module. >=20 > Maybe its best to remove the ZFS_DEBUG flag for the time being, as > the key change is to get ASSERTS enabled under INVARIANTS? >=20 >> On possible solution to this problem would be to grep opt_global.h >> from within the Makefile for OpenSolaris-ish modules: >> INVARIANTS_ENABLED!=3D grep INVARIANTS ${KERNBUILDDIR}/opt_global.h = || true >> .if !empty(INVARIANTS_ENABLED) >> CFLAGS +=3D -DDEBUG >> .endif >> If ZFS_DEBUG were also made a global option, similar logic could >> fail the ZFS module build if ZFS_DEBUG is enabled without INVARIANTS. >=20 > I'm not sure we want DEBUG enabled in modules triggered by INVARIANTS > as this does add quite a bit more than just additional ASSERT checks, > which as mentioned above is what the patch is designed to achieve. >=20 > My reason for keeping it just to ASSERTs / VERIFYs was to gain the > main benefit without enabling significant extra code. I would be okay with this if we were simply taking advantage of defined idioms in the OpenSolairs code for different levels of "diagnostic cost". Right now we have DEBUG and ZFS_DEBUG. You're proposing "half DEBUG", but since this doesn't exist upstream, it is a loaded gun waiting to go off on a future import (when an ASSERT is added that depends on #ifdef DEBUG .c file code). Folks already know that INVARIANTS compromises performance. I doubt the cost of turning on DEBUG would outweigh the benefit of finally having this code turned on in head. > Perhaps adding an additional global flag such as OPENSOLARIS_DEBUG > in addition to enabling ASSERT checks is the way to go? Only if you can upstream it. > Regards > Steve -- Justin= From owner-zfs-devel@FreeBSD.ORG Mon Jul 1 15:48:36 2013 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 329974AC for ; Mon, 1 Jul 2013 15:48:36 +0000 (UTC) (envelope-from will@firepipe.net) Received: from mail-vb0-f42.google.com (mail-vb0-f42.google.com [209.85.212.42]) by mx1.freebsd.org (Postfix) with ESMTP id ECF6E119A for ; Mon, 1 Jul 2013 15:48:35 +0000 (UTC) Received: by mail-vb0-f42.google.com with SMTP id i3so3738715vbh.15 for ; Mon, 01 Jul 2013 08:48:29 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:x-gm-message-state; bh=TUh+NkJXRELywOYL866PSmA4YhhCjmlv+WsXsCoaDhg=; b=HzHXG+Xlo0ajYUV52fXAxe9dWJB5WZnmluQPp/FnUkrjrGHZ7g/Ji3eOmad0c2Pbvv YXmd/zdK3lD1J3MswkQ/kvODtGM7ZAnbphNoB6LAOfpY76ikaGws1kLwT93mnDyjeuvo nTOkQ5kl3J0YPJBfvx7FC0cUqSZQ7/pKGQSJMjw3dQ6sj1cCGtibeHro6KDYmX74NyYM RzT4FHrYBPSJBcEnzlxQ8ZKRAjGkICaBJiovWa5zeK6/4o2JOgw6/E3vAK90t4QbBAs+ HqTB1GqKWBgCG9J0VudnEet/pZ8n3eBftGNbnrA0KlGZueiZfjaPmUtBaMjm9yvW6eOQ hMwg== MIME-Version: 1.0 X-Received: by 10.220.20.3 with SMTP id d3mr4992934vcb.55.1372693709241; Mon, 01 Jul 2013 08:48:29 -0700 (PDT) Received: by 10.58.226.66 with HTTP; Mon, 1 Jul 2013 08:48:29 -0700 (PDT) In-Reply-To: References: <5AD89226AC69454DA8297F8C90A4E7E2@multiplay.co.uk> Date: Mon, 1 Jul 2013 09:48:29 -0600 Message-ID: Subject: Re: ZFS_DEBUG + enable ZFS ASSERTS with INVARIANTS any objections? From: Will Andrews To: Steven Hartland Content-Type: text/plain; charset=UTF-8 X-Gm-Message-State: ALoCoQmZ/ik2G4rGthM+rLp06227IKcdmTW4T24h/3T3zncs/tnplMmAyYH6IgZes/pwhnUsXNAQ Cc: zfs-devel@freebsd.org X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Jul 2013 15:48:36 -0000 On Mon, Jul 1, 2013 at 6:04 AM, Steven Hartland wrote: > I'm not sure we want DEBUG enabled in modules triggered by INVARIANTS > as this does add quite a bit more than just additional ASSERT checks, > which as mentioned above is what the patch is designed to achieve. > > My reason for keeping it just to ASSERTs / VERIFYs was to gain the > main benefit without enabling significant extra code. > > Perhaps adding an additional global flag such as OPENSOLARIS_DEBUG > in addition to enabling ASSERT checks is the way to go? I personally really don't think it will be worthwhile. INVARIANTS is already a very expensive option to enable, and it is enabled by people who are trying to validate their kernel. In any case, we seriously need more people running *all* of the extra checking available in ZFS, assuming they are actually using ZFS on an INVARIANTS kernel. In SpectraBSD, we have a local patch that allows enabling ZFS_DEBUG via the kernel config, although the mechanism used is more of a hack (so I would not want it in FreeBSD :-)). But using a proper global option, if one wants to turn on ZFS_DEBUG specifically, would be nice. I think we should do that in addition to making INVARIANTS imply ZFS_DEBUG. Long term, FreeBSD will derive significant value from being able to easily merge changes from other ZFS ports. Therefore, it should be a high priority to either avoid making changes that can't reasonably be merged upstream, or to define an interface that can be merged upstream as a stub and implemented locally. --Will. From owner-zfs-devel@FreeBSD.ORG Tue Jul 2 06:51:13 2013 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id B1F2E737 for ; Tue, 2 Jul 2013 06:51:13 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: from mail-pa0-x231.google.com (mail-pa0-x231.google.com [IPv6:2607:f8b0:400e:c03::231]) by mx1.freebsd.org (Postfix) with ESMTP id 8D00B19BB for ; Tue, 2 Jul 2013 06:51:13 +0000 (UTC) Received: by mail-pa0-f49.google.com with SMTP id ld11so5867213pab.36 for ; Mon, 01 Jul 2013 23:51:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=delphix.com; s=google; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=wHGR5PxC9OQTBJ0lU9yJNQbZ9i5TWj4T9pdqIZ/vj2s=; b=ffzBCFhHlurpheZlQ4ob7KHKRN6VHELdS0ULn2lzTMosQnE67YVBbZ9wjZOR0YUBzv whioWLYPpTCWv8t9bb5ZPp2uzAIyrhvz+Ej577Tr4QsemYG50k0HD7wXe6LB/UCpeoCY LgqZff5bUsMmGOWbUa9AdE4Uej+7wWQ3lg1m4= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:x-gm-message-state; bh=wHGR5PxC9OQTBJ0lU9yJNQbZ9i5TWj4T9pdqIZ/vj2s=; b=cJEQ+HUU3nFl1spEk7o3G+/0G9ORMh4ZV7BXVxlaqtc6gsj3ePSPqTz1GatdhRontb hZjO9937M3FeuvJ4tMlJJhc4o2xJrksGwUZX7kur56+IZ+scKgFG2FNHG27m7gAVwmmM OJuQULD2dCa/99SOGPSbZguwj1my5KtJvOq9nPFasHBDRW8j7gFGNuRToOmd3DpnzTLz SUV6Y4X5TkVYrYW7ZSSHQBW3onULuGOrp8kqzXTCLS+7bTu9GZ9nRoFbW985oKpQL0U9 2gDhlPEg6kJA5qqSdOjbdFvromlavX1zOcsiORAgGISA8YGtIgXAbOOMfH+GUXfjl6aV 91hA== MIME-Version: 1.0 X-Received: by 10.66.26.179 with SMTP id m19mr27233459pag.4.1372747873397; Mon, 01 Jul 2013 23:51:13 -0700 (PDT) Received: by 10.70.132.66 with HTTP; Mon, 1 Jul 2013 23:51:13 -0700 (PDT) In-Reply-To: <5AD89226AC69454DA8297F8C90A4E7E2@multiplay.co.uk> References: <5AD89226AC69454DA8297F8C90A4E7E2@multiplay.co.uk> Date: Mon, 1 Jul 2013 23:51:13 -0700 Message-ID: Subject: Re: ZFS_DEBUG + enable ZFS ASSERTS with INVARIANTS any objections? From: Matthew Ahrens To: Steven Hartland X-Gm-Message-State: ALoCoQkoQMEJ7JposYDweuxsZCUU6BlAQeMg+6qYakCP+iJ3SzYCEnK+8r3lP/sqp+TkwjShm9X8 Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: zfs-devel@freebsd.org X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Jul 2013 06:51:13 -0000 If I understand correctly, INVARIANTS is a mechanism that enables additional runtime checks in the kernel, analogous to DEBUG enabling ASSERTions in Solaris-derived code. You want to turn on ZFS_DEBUG and assertions within ZFS without setting DEBUG in general. Why do you want to do this? Why not set DEBUG and turn on all assertions in Solaris-derived code? If it is performance, how much slow down do non-ZFS assertions cause? If it's too much, can we fix that? I'm not sure why ZFS_DEBUG exists, rather than just using DEBUG. It may be historical; now it seems that ZFS_DEBUG and DEBUG are always set together (including in libzpool). Unless there's a good reason for them to be separate, I'd rather see ZFS_DEBUG removed, and use DEBUG instead. --matt On Sat, Jun 29, 2013 at 4:47 PM, Steven Hartland wrote: > The attached patch adds a ZFS_DEBUG option as well > as enabling ZFS ASSERTS by default when the kernel > is compiled with INVARIANTS. > > This should enable us to get better test coverage > from CURRENT users and has already enabled me to > catch a number of issues when testing code + fix > for invalid ASSERT removed by r252390. > > So the question is there any objections to this > patch? > > Regards > Steve > _______________________________________________ > zfs-devel@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/zfs-devel > To unsubscribe, send any mail to "zfs-devel-unsubscribe@freebsd.org" > From owner-zfs-devel@FreeBSD.ORG Tue Jul 2 07:09:30 2013 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 4FDB09A2; Tue, 2 Jul 2013 07:09:30 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 676521A20; Tue, 2 Jul 2013 07:09:28 +0000 (UTC) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id KAA12262; Tue, 02 Jul 2013 10:09:26 +0300 (EEST) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1UtuiM-0004J9-2A; Tue, 02 Jul 2013 10:09:26 +0300 Message-ID: <51D27C6D.700@FreeBSD.org> Date: Tue, 02 Jul 2013 10:08:29 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:17.0) Gecko/20130405 Thunderbird/17.0.5 MIME-Version: 1.0 To: Matthew Ahrens Subject: Re: ZFS_DEBUG + enable ZFS ASSERTS with INVARIANTS any objections? References: <5AD89226AC69454DA8297F8C90A4E7E2@multiplay.co.uk> In-Reply-To: X-Enigmail-Version: 1.5.1 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: zfs-devel@FreeBSD.org X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Jul 2013 07:09:30 -0000 on 02/07/2013 09:51 Matthew Ahrens said the following: > If I understand correctly, INVARIANTS is a mechanism that enables > additional runtime checks in the kernel, analogous to DEBUG enabling > ASSERTions in Solaris-derived code. Yes. > You want to turn on ZFS_DEBUG and > assertions within ZFS without setting DEBUG in general. > > Why do you want to do this? Because in larger FreeBSD kernel DEBUG has already been used for other purposes. In some place it enables lots of printfs that typically are of no interest but can easily spam logs / message buffer. > Why not set DEBUG and turn on all assertions > in Solaris-derived code? If it is performance, how much slow down do > non-ZFS assertions cause? If it's too much, can we fix that? > > I'm not sure why ZFS_DEBUG exists, rather than just using DEBUG. It may be > historical; now it seems that ZFS_DEBUG and DEBUG are always set together > (including in libzpool). Unless there's a good reason for them to be > separate, I'd rather see ZFS_DEBUG removed, and use DEBUG instead. -- Andriy Gapon From owner-zfs-devel@FreeBSD.ORG Tue Jul 2 17:30:04 2013 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id C7CBE5C1 for ; Tue, 2 Jul 2013 17:30:04 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: from mail-la0-x236.google.com (mail-la0-x236.google.com [IPv6:2a00:1450:4010:c03::236]) by mx1.freebsd.org (Postfix) with ESMTP id 4E5A41C11 for ; Tue, 2 Jul 2013 17:30:03 +0000 (UTC) Received: by mail-la0-f54.google.com with SMTP id ec20so5868228lab.13 for ; Tue, 02 Jul 2013 10:30:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=delphix.com; s=google; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=lkTrwgyZJXTyZGMd56lD6m2+xopgseX7zLfsf4AN/LY=; b=LCkQTTZQYrgLYqweRiY78UCxalk6/26r2E5W/Lk+zomvJw29xgz7t9EaPD1Qebk6G4 GBjeon78ZDJXDv4pGE/euruCI4+zg2RgBH9oUysSbJC2QOf4uPStF8esY52zomYfEw9n 7kZPO4DC6bn6nNmAgkQxEpE1ZGntbc6sVaXgg= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:x-gm-message-state; bh=lkTrwgyZJXTyZGMd56lD6m2+xopgseX7zLfsf4AN/LY=; b=LxBt/MQS4gK6ds2UDnX9KWixcYd/cdylbwsrv6caC95kB/AWwFCa2MxVyGefvKYj0l nJVRAW7iV0iUrctGIRRezgh4lryRBjLKopE0GHTDumkbqbXNV/5KVktWz21PlikABxS2 G9z5pvdEvmHjyhfVFMVtgzjVoCoX8LksyQAg+4kWckXw/MJ6X+7Qyb9KIo+Xc+Sc2rM1 WzhcyIs+YTyH+NY9ej7fgC2GLPlCTNYKh5KFzpd0HI+9T70LdSgr6qHpYbRkQiJc6nrp kKra/OtoG5XrVpuXrq1EguybGAzrRQIP8LKZ6eQAlpOBC/t2i7qrNOKvTgs66aiJ+CR8 kKng== MIME-Version: 1.0 X-Received: by 10.112.157.226 with SMTP id wp2mr14277683lbb.65.1372786203007; Tue, 02 Jul 2013 10:30:03 -0700 (PDT) Received: by 10.114.57.3 with HTTP; Tue, 2 Jul 2013 10:30:02 -0700 (PDT) In-Reply-To: <51D27C6D.700@FreeBSD.org> References: <5AD89226AC69454DA8297F8C90A4E7E2@multiplay.co.uk> <51D27C6D.700@FreeBSD.org> Date: Tue, 2 Jul 2013 10:30:02 -0700 Message-ID: Subject: Re: ZFS_DEBUG + enable ZFS ASSERTS with INVARIANTS any objections? From: Matthew Ahrens To: Andriy Gapon X-Gm-Message-State: ALoCoQk5DQpJmSOHqZVwaE5tkuFOBi3bBYgXoNNpteWigDHS/8utyYDu65gGTHw/8pPDchlxJ1PF Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: zfs-devel@freebsd.org X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Jul 2013 17:30:04 -0000 On Tue, Jul 2, 2013 at 12:08 AM, Andriy Gapon wrote: > on 02/07/2013 09:51 Matthew Ahrens said the following: > > If I understand correctly, INVARIANTS is a mechanism that enables > > additional runtime checks in the kernel, analogous to DEBUG enabling > > ASSERTions in Solaris-derived code. > > Yes. > > > You want to turn on ZFS_DEBUG and > > assertions within ZFS without setting DEBUG in general. > > > > Why do you want to do this? > > Because in larger FreeBSD kernel DEBUG has already been used for other > purposes. > In some place it enables lots of printfs that typically are of no > interest but > can easily spam logs / message buffer. > Ah, so you'd be happy if Solaris had called it SOLARIS_DEBUG instead of simply DEBUG. You were just "lucky" that ZFS already renamed it to ZFS_DEBUG (but this is only used in a few cases). Seems like the possible solutions are: - change DEBUG in all your solaris-derived code to INVARIANTS (or something other than DEBUG) this introduces maintenance burden when merging with upstream - change your build process to use -DDEBUG on solaris-derived code when INVARIANTS is set I believe this is what Justin proposed in the first response in this thread. --matt > > > Why not set DEBUG and turn on all assertions > > in Solaris-derived code? If it is performance, how much slow down do > > non-ZFS assertions cause? If it's too much, can we fix that? > > > > I'm not sure why ZFS_DEBUG exists, rather than just using DEBUG. It may > be > > historical; now it seems that ZFS_DEBUG and DEBUG are always set together > > (including in libzpool). Unless there's a good reason for them to be > > separate, I'd rather see ZFS_DEBUG removed, and use DEBUG instead. > > > > -- > Andriy Gapon > From owner-zfs-devel@FreeBSD.ORG Mon Jul 8 11:40:46 2013 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 0A4C6269 for ; Mon, 8 Jul 2013 11:40:46 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) by mx1.freebsd.org (Postfix) with ESMTP id D79251321 for ; Mon, 8 Jul 2013 11:40:45 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.7/8.14.7) with ESMTP id r68BeXZw053981 for ; Mon, 8 Jul 2013 11:40:45 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.7/8.14.7/Submit) id r68B7n8N047384 for zfs-devel@FreeBSD.org; Mon, 8 Jul 2013 11:07:49 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 8 Jul 2013 11:07:49 GMT Message-Id: <201307081107.r68B7n8N047384@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: zfs-devel@FreeBSD.org Subject: Current problem reports assigned to zfs-devel@FreeBSD.org X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Jul 2013 11:40:46 -0000 Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- s kern/178467 zfs-devel [zfs] [request] Optimized Checksum Code for ZFS o kern/178387 zfs-devel [zfs] [patch] sparse files performance improvements 2 problems total. From owner-zfs-devel@FreeBSD.ORG Mon Jul 8 18:53:02 2013 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 1C086DA4 for ; Mon, 8 Jul 2013 18:53:02 +0000 (UTC) (envelope-from delphij@delphij.net) Received: from anubis.delphij.net (anubis.delphij.net [64.62.153.212]) by mx1.freebsd.org (Postfix) with ESMTP id 0934B124B for ; Mon, 8 Jul 2013 18:53:01 +0000 (UTC) Received: from zeta.ixsystems.com (drawbridge.ixsystems.com [206.40.55.65]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by anubis.delphij.net (Postfix) with ESMTPSA id 264F39D2D; Mon, 8 Jul 2013 11:53:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=delphij.net; s=anubis; t=1373309581; bh=bhs7Dh0iJIPL5/JsJSaUDtZDml5pZJXsvTHgaj+F9Ds=; h=Date:From:Reply-To:To:Subject; b=cHIRNgLo0TJr0FcLAqY+PEHDOEkn2Kt2bf6rmgXOhUcCR/gIkZIcEw0ha9ZIHgcMw k/fLfyJ3CMCcYEU+btzn0qh8x1pqFXVCl4GqQZGIVdYeiSl8jINvzn3QoxA02h9vCV Pc0Y7VRWNymMLSO2o5+X5gjhWBGx8R7sC2eHqWHo= Message-ID: <51DB0A8B.6040808@delphij.net> Date: Mon, 08 Jul 2013 11:52:59 -0700 From: Xin Li Organization: The FreeBSD Project MIME-Version: 1.0 To: zfs-devel@freebsd.org Subject: Use illumos-gate on github as upstream in the future? X-Enigmail-Version: 1.5.1 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list Reply-To: d@delphij.net List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Jul 2013 18:53:02 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 Hi, Is there any objection of using illumos-gate on github as upstream in the future? Cheers, - -- Xin LI https://www.delphij.net/ FreeBSD - The Power to Serve! Live free or die -----BEGIN PGP SIGNATURE----- iQEcBAEBCgAGBQJR2wqLAAoJEG80Jeu8UPuz+4MH/RmT649XeKaTnsQA70OVJqY8 iN1PHowepLZGO4zn7qruMctxVeQoxXb9NDyo/pyejaMJo0ezq/6dqrQqDyJm2mH1 jM/0teGfMXDdlfWtnlLWE8X9kXYCCGZytUXD3ntJ25i07LvE+/0dPg22Q4bNoOjV l1x4HFuQiPwx7cbB5G1QLk1n+tzPiwuAq64JR/D/+3IEhA9B9CMyd+a98L9XgcsE AblmIdty+zB5cZXrHFfZfh/bESd/w6X5lRuD52BWENTS9BssSh6aashzMgGG3uNZ TQptPYcW6QL3fMycvMx/EToL8jOItpMGCq/zcnktY4DewJRSQnMI7naJzq6UEVw= =Ge7J -----END PGP SIGNATURE----- From owner-zfs-devel@FreeBSD.ORG Mon Jul 8 20:06:10 2013 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id E6AD6FD8 for ; Mon, 8 Jul 2013 20:06:10 +0000 (UTC) (envelope-from will@firepipe.net) Received: from mail-vc0-f169.google.com (mail-vc0-f169.google.com [209.85.220.169]) by mx1.freebsd.org (Postfix) with ESMTP id AB5A116F7 for ; Mon, 8 Jul 2013 20:06:10 +0000 (UTC) Received: by mail-vc0-f169.google.com with SMTP id ia10so3685776vcb.14 for ; Mon, 08 Jul 2013 13:06:04 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:x-gm-message-state; bh=WtwforyzdYPB+pOktKO+Rs24nyVmktSrW54KZxFd48Q=; b=SQCT+hHp53/pDGr0lQB/PdYxp/1x8HukEN76gbJH72d32OmNv/w5M0/4axCQEpfmCG 1svuCDPzwSZgbrKTkIkl5jTG7Y+vsGOauOAouU+PkrhRBl+94SudhL66DcdVDja1bC5l zAfg47CkLbFJuDCKmtCFVI6LLvPtYIaTKTDfCQ8j29+vFoW3Pw1F2QIzcYDFAjHnoxgv nvCsLKz0sWPCGS+GEQwLdQIYhKdxi/3PMEf1z/Ru2/RM69dCnVg+GP43P5fyOqD1L4bG XLCbf11ZAYUUKSilXheNRu4b++lAsRMaWi946D2cEtLBffuEgNkY8N16PjA2dO7K9sIb +lfQ== MIME-Version: 1.0 X-Received: by 10.221.64.18 with SMTP id xg18mr14836800vcb.57.1373313964096; Mon, 08 Jul 2013 13:06:04 -0700 (PDT) Received: by 10.58.226.66 with HTTP; Mon, 8 Jul 2013 13:06:03 -0700 (PDT) In-Reply-To: <51DB0A8B.6040808@delphij.net> References: <51DB0A8B.6040808@delphij.net> Date: Mon, 8 Jul 2013 14:06:03 -0600 Message-ID: Subject: Re: Use illumos-gate on github as upstream in the future? From: Will Andrews To: Xin LI Content-Type: text/plain; charset=UTF-8 X-Gm-Message-State: ALoCoQnKpyKgFOixLWhZWQOad6Nw3xMvLOv3yhEYXZB5KqY4NBxyb5RCPTjAyMisj/od9y8F1WZ0 Cc: zfs-devel@freebsd.org X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Jul 2013 20:06:11 -0000 On Mon, Jul 8, 2013 at 12:52 PM, Xin Li wrote: > Is there any objection of using illumos-gate on github as upstream in > the future? Upstream in what sense? Isn't it already? --Will. From owner-zfs-devel@FreeBSD.ORG Mon Jul 8 21:03:02 2013 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id C0DC8995 for ; Mon, 8 Jul 2013 21:03:02 +0000 (UTC) (envelope-from delphij@delphij.net) Received: from anubis.delphij.net (anubis.delphij.net [IPv6:2001:470:1:117::25]) by mx1.freebsd.org (Postfix) with ESMTP id A6C7618EE for ; Mon, 8 Jul 2013 21:03:02 +0000 (UTC) Received: from zeta.ixsystems.com (drawbridge.ixsystems.com [206.40.55.65]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by anubis.delphij.net (Postfix) with ESMTPSA id A6ABC9424; Mon, 8 Jul 2013 14:03:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=delphij.net; s=anubis; t=1373317382; bh=ZNrcSfZegrYlo6ichw9d2iPw+bGn/nJZma7xmyXpfNI=; h=Date:From:Reply-To:To:CC:Subject:References:In-Reply-To; b=RUHt/Yv2jnKNFgtl9jZ5i8luobi3OYR/9QegXSAtnjkRANzwXsMgyqMDhIwQJfH/n nqzVGXKik4fn6D+9mduU1KXKf1RIpuFR/LaQRKDlikWarctL7BDF3/YXza9qxswvl5 VJrbOeo2TKANzyIvFnE1irLOUF2wqUGRLeQwfacI= Message-ID: <51DB2904.9040404@delphij.net> Date: Mon, 08 Jul 2013 14:03:00 -0700 From: Xin Li Organization: The FreeBSD Project MIME-Version: 1.0 To: Will Andrews Subject: Re: Use illumos-gate on github as upstream in the future? References: <51DB0A8B.6040808@delphij.net> In-Reply-To: X-Enigmail-Version: 1.5.1 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: zfs-devel@freebsd.org, Xin LI X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list Reply-To: d@delphij.net List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Jul 2013 21:03:02 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 On 07/08/13 13:06, Will Andrews wrote: > On Mon, Jul 8, 2013 at 12:52 PM, Xin Li > wrote: >> Is there any objection of using illumos-gate on github as >> upstream in the future? > > Upstream in what sense? Isn't it already? We currently reference revision numbers from the mercurial repository, the pro is that it's in part a linear version number (in the form of 14075:f73fc37ec88d, where 14075 is a "serial" number). By using the github as upstream, we would have revision numbers in the form of like, 57ff5e7 (it's still possible to do a "serial" by doing git log | grep ^commit | wc -l minus 1). The pros for this approach is that Illumos seems to have been moved to github lately, and git -> hg export sometimes lag for a week before catching up with the git repository. The benefit of doing it this way is that we do not get blocked if git -> hg conversion is stuck. Cheers, - -- Xin LI https://www.delphij.net/ FreeBSD - The Power to Serve! Live free or die -----BEGIN PGP SIGNATURE----- iQEcBAEBCgAGBQJR2ykEAAoJEG80Jeu8UPuzUVUIAIfY33saQRrZ7X2hyvpSTLoR SBta4pzJTs7VAGSwTRYKAPH2CVXQBAd1i1U8ux1GsjGgM5RoyaqYv/H6OpTeN/pX NyAEassHfbvZIigKDdI4xUk6SMN4ZmwjXGfLlgT3x3aaLmffNP+mOT7K9LsyADvu 36lZJxCuT4aX7strIrAwxQIR7p5Qm67/gP+eJNd8XtUbPcuMWbXTwcMUDy40NApn 5U3G1SNH6rW9iuRYe28ZORr5Y0z1QJEZmVTXhjBXixLNW8IWmK/Qg9eKqG6xE3fs pE39TOjn2vm/D6qLmhznfPytY2SC9lcM2gfuX7rv0Mm2XZCgXpv3XYTnZ/MKH5M= =veNg -----END PGP SIGNATURE----- From owner-zfs-devel@FreeBSD.ORG Wed Jul 10 09:25:27 2013 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id D675FEB5; Wed, 10 Jul 2013 09:25:27 +0000 (UTC) (envelope-from prvs=1903808b5b=killing@multiplay.co.uk) Received: from mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) by mx1.freebsd.org (Postfix) with ESMTP id BDCB21E65; Wed, 10 Jul 2013 09:25:26 +0000 (UTC) Received: from r2d2 ([82.69.141.170]) by mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) (MDaemon PRO v10.0.4) with ESMTP id md50004902843.msg; Wed, 10 Jul 2013 10:25:24 +0100 X-Spam-Processed: mail1.multiplay.co.uk, Wed, 10 Jul 2013 10:25:24 +0100 (not processed: message from valid local sender) X-MDDKIM-Result: neutral (mail1.multiplay.co.uk) X-MDRemoteIP: 82.69.141.170 X-Return-Path: prvs=1903808b5b=killing@multiplay.co.uk X-Envelope-From: killing@multiplay.co.uk Message-ID: <628C5D1AF6044488B708484203D70B7A@multiplay.co.uk> From: "Steven Hartland" To: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= , , , References: <86zjtupz3r.fsf@nine.des.no> Subject: Re: Make ZFS use the physical sector size when computing initial ashift Date: Wed, 10 Jul 2013 10:25:36 +0100 MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="----=_NextPart_000_1133_01CE7D57.D127DB40" X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.5931 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.6157 Cc: ivoras@freebsd.org X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 10 Jul 2013 09:25:28 -0000 This is a multi-part message in MIME format. ------=_NextPart_000_1133_01CE7D57.D127DB40 Content-Type: text/plain; format=flowed; charset="utf-8"; reply-type=original Content-Transfer-Encoding: 8bit Hi DES, unfortunately you need a quite bit more than this to work compatibly. I've had a patch here that does just this for quite some time but there's been some discussion on how we want additional control over this so its not been commited. If others are interested I've attached this as it achieves what we needed here so may also be of use for others too. There's also a big discussion on illumos about this very subject ATM so I'm monitoring that too. Hopefully there will be a nice conclusion come from that how people want to proceed and we'll be able to get a change in that works for everyone. Regards Steve ----- Original Message ----- From: "Dag-Erling Smørgrav" To: ; Cc: Sent: Wednesday, July 10, 2013 10:02 AM Subject: Make ZFS use the physical sector size when computing initial ashift The attached patch causes ZFS to base the minimum transfer size for a new vdev on the GEOM provider's stripesize (physical sector size) rather than sectorsize (logical sector size), provided that stripesize is a power of two larger than sectorsize and smaller than or equal to VDEV_PAD_SIZE. This should eliminate the need for ivoras@'s gnop trick when creating ZFS pools on Advanced Format drives. DES -- Dag-Erling Smørgrav - des@des.no -------------------------------------------------------------------------------- > _______________________________________________ > freebsd-fs@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" ================================================ This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmaster@multiplay.co.uk. ------=_NextPart_000_1133_01CE7D57.D127DB40 Content-Type: application/octet-stream; name="zzz-zfs-ashift-fix.patch" Content-Transfer-Encoding: quoted-printable Content-Disposition: attachment; filename="zzz-zfs-ashift-fix.patch" Changes zfs zpool initial / desired ashift to be based off stripesize=0A= instead of sectorsize making it compatible with drives marked with=0A= the 4k sector size quirk.=0A= =0A= Without the correct min block size BIO_DELETE requests passed to=0A= a large number of current SSD's via TRIM don't actually perform=0A= any LBA TRIM so its vital for the correct operation of TRIM to get=0A= the correct min block size.=0A= =0A= To do this we added the additional dashift (desired ashift) to=0A= vdev_open_func_t calls. This was needed as just updating ashift to=0A= be based off stripesize would mean that a devices reported minimum=0A= transfer size (ashift) could increase and that in turn would cause=0A= member devices to be unusable and hence break pools with error=0A= ZFS-8000-5E.=0A= =0A= The global minimum ashift used for new zpools can now also be=0A= tuned using the vfs.zfs.min_create_ashift sysctl. This defaults=0A= to 12 (4096 byte blocks) in order to optimise for newer disks which=0A= are migrating from 512 to 4096 byte sectors.=0A= =0A= The value of vfs.zfs.min_create_ashift is limited to min of=0A= SPA_MINBLOCKSHIFT (9) and a max of SPA_MAXBLOCKSHIFT (17).=0A= --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_disk.c.orig = 2011-06-06 09:36:46.000000000 +0000=0A= +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_disk.c = 2012-11-02 14:47:55.293668071 +0000=0A= @@ -32,6 +32,8 @@=0A= #include =0A= #include =0A= =0A= +extern int zfs_min_ashift;=0A= +=0A= /*=0A= * Virtual device vector for disks.=0A= */=0A= @@ -103,7 +105,7 @@=0A= }=0A= =0A= static int=0A= -vdev_disk_open(vdev_t *vd, uint64_t *psize, uint64_t *ashift)=0A= +vdev_disk_open(vdev_t *vd, uint64_t *psize, uint64_t *ashift, uint64_t = *dashift)=0A= {=0A= spa_t *spa =3D vd->vdev_spa;=0A= vdev_disk_t *dvd;=0A= @@ -284,7 +286,7 @@=0A= }=0A= =0A= /*=0A= - * Determine the device's minimum transfer size.=0A= + * Determine the device's minimum and desired transfer size.=0A= * If the ioctl isn't supported, assume DEV_BSIZE.=0A= */=0A= if (ldi_ioctl(dvd->vd_lh, DKIOCGMEDIAINFOEXT, (intptr_t)&dkmext,=0A= @@ -292,6 +294,7 @@=0A= dkmext.dki_pbsize =3D DEV_BSIZE;=0A= =0A= *ashift =3D highbit(MAX(dkmext.dki_pbsize, SPA_MINBLOCKSIZE)) - 1;=0A= + *dashift =3D highbit(MAX(dkmext.dki_pbsize, (1ULL << zfs_min_ashift))) = - 1;=0A= =0A= /*=0A= * Clear the nowritecache bit, so that on a vdev_reopen() we will=0A= --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_file.c.orig = 2012-01-05 22:31:25.000000000 +0000=0A= +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_file.c = 2012-11-02 14:47:38.252107541 +0000=0A= @@ -30,6 +30,8 @@=0A= #include =0A= #include =0A= =0A= +extern int zfs_min_ashift;=0A= +=0A= /*=0A= * Virtual device vector for files.=0A= */=0A= @@ -47,7 +49,7 @@=0A= }=0A= =0A= static int=0A= -vdev_file_open(vdev_t *vd, uint64_t *psize, uint64_t *ashift)=0A= +vdev_file_open(vdev_t *vd, uint64_t *psize, uint64_t *ashift, uint64_t = *dashift)=0A= {=0A= vdev_file_t *vf;=0A= vnode_t *vp;=0A= @@ -127,6 +129,7 @@=0A= =0A= *psize =3D vattr.va_size;=0A= *ashift =3D SPA_MINBLOCKSHIFT;=0A= + *dashift =3D zfs_min_ashift;=0A= =0A= return (0);=0A= }=0A= --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_geom.c.orig = 2012-11-02 12:20:15.918986181 +0000=0A= +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_geom.c = 2012-11-02 14:47:48.135273692 +0000=0A= @@ -36,6 +36,8 @@=0A= #include =0A= #include =0A= =0A= +extern int zfs_min_ashift;=0A= +=0A= /*=0A= * Virtual device vector for GEOM.=0A= */=0A= @@ -408,7 +410,7 @@=0A= }=0A= =0A= static int=0A= -vdev_geom_open(vdev_t *vd, uint64_t *psize, uint64_t *ashift)=0A= +vdev_geom_open(vdev_t *vd, uint64_t *psize, uint64_t *ashift, uint64_t = *dashift)=0A= {=0A= struct g_provider *pp;=0A= struct g_consumer *cp;=0A= @@ -494,9 +496,10 @@=0A= *psize =3D pp->mediasize;=0A= =0A= /*=0A= - * Determine the device's minimum transfer size.=0A= + * Determine the device's minimum and desired transfer size.=0A= */=0A= *ashift =3D highbit(MAX(pp->sectorsize, SPA_MINBLOCKSIZE)) - 1;=0A= + *dashift =3D highbit(MAX(pp->stripesize, (1ULL << zfs_min_ashift))) - = 1;=0A= =0A= /*=0A= * Clear the nowritecache settings, so that on a vdev_reopen()=0A= --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_mirror.c.orig = 2012-07-03 11:49:22.342245151 +0000=0A= +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_mirror.c = 2012-07-03 11:58:02.161948585 +0000=0A= @@ -127,7 +127,7 @@=0A= }=0A= =0A= static int=0A= -vdev_mirror_open(vdev_t *vd, uint64_t *asize, uint64_t *ashift)=0A= +vdev_mirror_open(vdev_t *vd, uint64_t *asize, uint64_t *ashift, = uint64_t *dashift)=0A= {=0A= int numerrors =3D 0;=0A= int lasterror =3D 0;=0A= @@ -150,6 +150,7 @@=0A= =0A= *asize =3D MIN(*asize - 1, cvd->vdev_asize - 1) + 1;=0A= *ashift =3D MAX(*ashift, cvd->vdev_ashift);=0A= + *dashift =3D MAX(*dashift, cvd->vdev_dashift);=0A= }=0A= =0A= if (numerrors =3D=3D vd->vdev_children) {=0A= --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_missing.c.orig = 2012-07-03 11:49:10.545275865 +0000=0A= +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_missing.c = 2012-07-03 11:58:07.670470640 +0000=0A= @@ -40,7 +40,7 @@=0A= =0A= /* ARGSUSED */=0A= static int=0A= -vdev_missing_open(vdev_t *vd, uint64_t *psize, uint64_t *ashift)=0A= +vdev_missing_open(vdev_t *vd, uint64_t *psize, uint64_t *ashift, = uint64_t *dashift)=0A= {=0A= /*=0A= * Really this should just fail. But then the root vdev will be in the=0A= @@ -50,6 +50,7 @@=0A= */=0A= *psize =3D 0;=0A= *ashift =3D 0;=0A= + *dashift =3D 0;=0A= return (0);=0A= }=0A= =0A= --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_raidz.c.orig = 2012-07-03 11:49:03.675875505 +0000=0A= +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_raidz.c = 2012-07-03 11:58:15.334806334 +0000=0A= @@ -1447,7 +1447,7 @@=0A= }=0A= =0A= static int=0A= -vdev_raidz_open(vdev_t *vd, uint64_t *asize, uint64_t *ashift)=0A= +vdev_raidz_open(vdev_t *vd, uint64_t *asize, uint64_t *ashift, uint64_t = *dashift)=0A= {=0A= vdev_t *cvd;=0A= uint64_t nparity =3D vd->vdev_nparity;=0A= @@ -1476,6 +1476,7 @@=0A= =0A= *asize =3D MIN(*asize - 1, cvd->vdev_asize - 1) + 1;=0A= *ashift =3D MAX(*ashift, cvd->vdev_ashift);=0A= + *dashift =3D MAX(*dashift, cvd->vdev_dashift);=0A= }=0A= =0A= *asize *=3D vd->vdev_children;=0A= --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_root.c.orig = 2012-07-03 11:49:27.901760380 +0000=0A= +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_root.c = 2012-07-03 11:58:19.704427068 +0000=0A= @@ -50,7 +50,7 @@=0A= }=0A= =0A= static int=0A= -vdev_root_open(vdev_t *vd, uint64_t *asize, uint64_t *ashift)=0A= +vdev_root_open(vdev_t *vd, uint64_t *asize, uint64_t *ashift, uint64_t = *dashift)=0A= {=0A= int lasterror =3D 0;=0A= int numerrors =3D 0;=0A= @@ -78,6 +78,7 @@=0A= =0A= *asize =3D 0;=0A= *ashift =3D 0;=0A= + *dashift =3D 0;=0A= =0A= return (0);=0A= }=0A= --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev.c.orig = 2012-10-22 20:41:50.234005351 +0000=0A= +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev.c 2012-10-22 = 20:42:16.355805894 +0000=0A= @@ -1125,6 +1125,7 @@=0A= uint64_t osize =3D 0;=0A= uint64_t asize, psize;=0A= uint64_t ashift =3D 0;=0A= + uint64_t dashift =3D 0;=0A= =0A= ASSERT(vd->vdev_open_thread =3D=3D curthread ||=0A= spa_config_held(spa, SCL_STATE_ALL, RW_WRITER) =3D=3D = SCL_STATE_ALL);=0A= @@ -1154,7 +1155,7 @@=0A= return (ENXIO);=0A= }=0A= =0A= - error =3D vd->vdev_ops->vdev_op_open(vd, &osize, &ashift);=0A= + error =3D vd->vdev_ops->vdev_op_open(vd, &osize, &ashift, &dashift);=0A= =0A= /*=0A= * Reset the vdev_reopening flag so that we actually close=0A= @@ -1255,14 +1256,16 @@=0A= */=0A= vd->vdev_asize =3D asize;=0A= vd->vdev_ashift =3D MAX(ashift, vd->vdev_ashift);=0A= + vd->vdev_dashift =3D MAX(dashift, vd->vdev_dashift);=0A= } else {=0A= /*=0A= * Make sure the alignment requirement hasn't increased.=0A= */=0A= if (ashift > vd->vdev_top->vdev_ashift) {=0A= + printf("ZFS ashift open failure of %s (%ld > %ld)\n", vd->vdev_path, = ashift, vd->vdev_top->vdev_ashift);=0A= vdev_set_state(vd, B_TRUE, VDEV_STATE_CANT_OPEN,=0A= VDEV_AUX_BAD_LABEL);=0A= return (EINVAL);=0A= }=0A= }=0A= =0A= --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_label.c.orig = 2012-11-05 15:27:52.092194343 +0000=0A= +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_label.c = 2012-11-05 15:53:26.449021023 +0000=0A= @@ -145,9 +145,12 @@=0A= #include =0A= =0A= static boolean_t vdev_trim_on_init =3D B_TRUE;=0A= +static boolean_t vdev_dashift_enable =3D B_TRUE;=0A= SYSCTL_DECL(_vfs_zfs_vdev);=0A= SYSCTL_INT(_vfs_zfs_vdev, OID_AUTO, trim_on_init, CTLFLAG_RW,=0A= &vdev_trim_on_init, 0, "Enable/disable full vdev trim on = initialisation");=0A= +SYSCTL_INT(_vfs_zfs_vdev, OID_AUTO, optimal_ashift, CTLFLAG_RW,=0A= + &vdev_dashift_enable, 0, "Enable/disable optimal ashift usage on = initialisation");=0A= =0A= /*=0A= * Basic routines to read and write from a vdev label.=0A= @@ -282,6 +285,16 @@=0A= vd->vdev_ms_array) =3D=3D 0);=0A= VERIFY(nvlist_add_uint64(nv, ZPOOL_CONFIG_METASLAB_SHIFT,=0A= vd->vdev_ms_shift) =3D=3D 0);=0A= + /*=0A= + * We use the max of ashift and dashift (the desired/optimal=0A= + * ashift), which is typically the stripesize of a device, to=0A= + * ensure we get the best performance from underlying devices.=0A= + * =0A= + * Its done here as it should only ever have an effect on new=0A= + * zpool creation.=0A= + */=0A= + if (vdev_dashift_enable)=0A= + vd->vdev_ashift =3D MAX(vd->vdev_ashift, vd->vdev_dashift);=0A= VERIFY(nvlist_add_uint64(nv, ZPOOL_CONFIG_ASHIFT,=0A= vd->vdev_ashift) =3D=3D 0);=0A= VERIFY(nvlist_add_uint64(nv, ZPOOL_CONFIG_ASIZE,=0A= --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/vdev_impl.h.orig = 2012-10-22 20:40:08.361577293 +0000=0A= +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/vdev_impl.h = 2012-10-22 21:02:52.447781800 +0000=0A= @@ -55,7 +55,7 @@=0A= /*=0A= * Virtual device operations=0A= */=0A= -typedef int vdev_open_func_t(vdev_t *vd, uint64_t *size, uint64_t = *ashift);=0A= +typedef int vdev_open_func_t(vdev_t *vd, uint64_t *size, uint64_t = *ashift, uint64_t *dashift);=0A= typedef void vdev_close_func_t(vdev_t *vd);=0A= typedef uint64_t vdev_asize_func_t(vdev_t *vd, uint64_t psize);=0A= typedef int vdev_io_start_func_t(zio_t *zio);=0A= @@ -119,6 +119,7 @@=0A= uint64_t vdev_asize; /* allocatable device capacity */=0A= uint64_t vdev_min_asize; /* min acceptable asize */=0A= uint64_t vdev_ashift; /* block alignment shift */=0A= + uint64_t vdev_dashift; /* desired blk alignment shift */=0A= uint64_t vdev_state; /* see VDEV_STATE_* #defines */=0A= uint64_t vdev_prevstate; /* used when reopening a vdev */=0A= vdev_ops_t *vdev_ops; /* vdev operations */=0A= --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dsl_pool.c.orig = 2012-11-02 14:56:29.474248887 +0000=0A= +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dsl_pool.c 2012-11-03 = 01:27:28.066912403 +0000=0A= @@ -41,6 +41,30 @@=0A= #include =0A= #include =0A= =0A= +#define ZFS_MIN_ASHIFT SPA_MINBLOCKSHIFT=0A= +/*=0A= + * Max ashift - limited by how labels are accessed by zio_read_phys = using offsets=0A= + * within vdev_label_t=0A= + *=0A= + * If label access is fixed to work with ashift properly then the max = should be=0A= + * set to SPA_MAXBLOCKSHIFT=0A= + */=0A= +#define ZFS_MAX_ASHIFT 13=0A= +/*=0A= + * Optimum ashift - defaults to 12 which results in a min block size of = 4096 as=0A= + * this is the optimum value for newer disks which are migrating from = 512 to 4096=0A= + * byte sectors=0A= + */=0A= +#define ZFS_OPTIMUM_ASHIFT 12 =0A= +=0A= +/*=0A= + * Minimum ashift used when creating new pools=0A= + *=0A= + * This can be tuned using the sysctl vfs.zfs.min_create_ashift but is = limited=0A= + * to a min of ZFS_MIN_ASHIFT and a max of ZFS_MAX_ASHIFT=0A= + * =0A= + */=0A= +int zfs_min_ashift =3D MAX(SPA_MINBLOCKSHIFT, ZFS_OPTIMUM_ASHIFT);=0A= int zfs_no_write_throttle =3D 0;=0A= int zfs_write_limit_shift =3D 3; /* 1/8th of physical memory */=0A= int zfs_txg_synctime_ms =3D 1000; /* target millisecs to sync a txg */=0A= @@ -54,6 +78,9 @@=0A= =0A= static pgcnt_t old_physmem =3D 0;=0A= =0A= +#ifdef _KERNEL=0A= +static int min_ashift_sysctl(SYSCTL_HANDLER_ARGS);=0A= +=0A= SYSCTL_DECL(_vfs_zfs);=0A= TUNABLE_INT("vfs.zfs.no_write_throttle", &zfs_no_write_throttle);=0A= SYSCTL_INT(_vfs_zfs, OID_AUTO, no_write_throttle, CTLFLAG_RDTUN,=0A= @@ -78,6 +105,32 @@=0A= TUNABLE_QUAD("vfs.zfs.write_limit_override", &zfs_write_limit_override);=0A= SYSCTL_QUAD(_vfs_zfs, OID_AUTO, write_limit_override, CTLFLAG_RDTUN,=0A= &zfs_write_limit_override, 0, "");=0A= +SYSCTL_PROC(_vfs_zfs, OID_AUTO, min_create_ashift, CTLTYPE_INT | = CTLFLAG_RW,=0A= + &zfs_min_ashift, 0, min_ashift_sysctl, "I",=0A= + "Minimum ashift used when creating new pools");=0A= +=0A= +static int=0A= +min_ashift_sysctl(SYSCTL_HANDLER_ARGS)=0A= +{=0A= + int error, value;=0A= +=0A= + value =3D *(int *)arg1;=0A= +=0A= + error =3D sysctl_handle_int(oidp, &value, 0, req);=0A= +=0A= + if ((error !=3D 0) || (req->newptr =3D=3D NULL))=0A= + return (error);=0A= +=0A= + if (value < ZFS_MIN_ASHIFT)=0A= + value =3D ZFS_MIN_ASHIFT;=0A= + else if (value > ZFS_MAX_ASHIFT)=0A= + value =3D ZFS_MAX_ASHIFT;=0A= +=0A= + *(int *)arg1 =3D value;=0A= +=0A= + return (0);=0A= +}=0A= +#endif=0A= =0A= int=0A= dsl_pool_open_special_dir(dsl_pool_t *dp, const char *name, dsl_dir_t = **ddp)=0A= ------=_NextPart_000_1133_01CE7D57.D127DB40-- From owner-zfs-devel@FreeBSD.ORG Wed Jul 10 10:46:27 2013 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 98129BB2; Wed, 10 Jul 2013 10:46:27 +0000 (UTC) (envelope-from des@des.no) Received: from smtp.des.no (smtp.des.no [194.63.250.102]) by mx1.freebsd.org (Postfix) with ESMTP id 5A20012B2; Wed, 10 Jul 2013 10:46:27 +0000 (UTC) Received: from nine.des.no (smtp.des.no [194.63.250.102]) by smtp-int.des.no (Postfix) with ESMTP id 2FD32447F; Wed, 10 Jul 2013 10:46:26 +0000 (UTC) Received: by nine.des.no (Postfix, from userid 1001) id 85689352D6; Wed, 10 Jul 2013 12:46:10 +0200 (CEST) From: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= To: "Steven Hartland" Subject: Re: Make ZFS use the physical sector size when computing initial ashift References: <86zjtupz3r.fsf@nine.des.no> <628C5D1AF6044488B708484203D70B7A@multiplay.co.uk> Date: Wed, 10 Jul 2013 12:46:10 +0200 In-Reply-To: <628C5D1AF6044488B708484203D70B7A@multiplay.co.uk> (Steven Hartland's message of "Wed, 10 Jul 2013 10:25:36 +0100") Message-ID: <86vc4ipua5.fsf@nine.des.no> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.3 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Cc: freebsd-fs@freebsd.org, freebsd-hackers@freebsd.org, zfs-devel@FreeBSD.org, ivoras@freebsd.org X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 10 Jul 2013 10:46:27 -0000 "Steven Hartland" writes: > Hi DES, unfortunately you need a quite bit more than this to work > compatibly. *chirp* *chirp* *chirp* DES --=20 Dag-Erling Sm=C3=B8rgrav - des@des.no From owner-zfs-devel@FreeBSD.ORG Wed Jul 10 11:03:20 2013 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id AD306958; Wed, 10 Jul 2013 11:03:20 +0000 (UTC) (envelope-from borjam@sarenet.es) Received: from proxypop04.sare.net (proxypop04.sare.net [194.30.0.65]) by mx1.freebsd.org (Postfix) with ESMTP id 6FF721453; Wed, 10 Jul 2013 11:03:20 +0000 (UTC) Received: from [172.16.2.2] (izaro.sarenet.es [192.148.167.11]) by proxypop04.sare.net (Postfix) with ESMTPSA id 0E5DE9DC4EE; Wed, 10 Jul 2013 13:03:13 +0200 (CEST) Subject: Re: Make ZFS use the physical sector size when computing initial ashift Mime-Version: 1.0 (Apple Message framework v1283) Content-Type: text/plain; charset=us-ascii From: Borja Marcos X-Priority: 3 In-Reply-To: <628C5D1AF6044488B708484203D70B7A@multiplay.co.uk> Date: Wed, 10 Jul 2013 13:03:09 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: <774B60E8-19C2-4A3A-880D-0D8726DC6727@sarenet.es> References: <86zjtupz3r.fsf@nine.des.no> <628C5D1AF6044488B708484203D70B7A@multiplay.co.uk> To: Steven Hartland X-Mailer: Apple Mail (2.1283) Cc: freebsd-fs@freebsd.org, =?iso-8859-1?Q?Dag-Erling_Sm=F8rgrav?= , zfs-devel@FreeBSD.org, ivoras@freebsd.org, freebsd-hackers@freebsd.org X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 10 Jul 2013 11:03:20 -0000 On Jul 10, 2013, at 11:25 AM, Steven Hartland wrote: > If others are interested I've attached this as it achieves what we = needed here so > may also be of use for others too. >=20 > There's also a big discussion on illumos about this very subject ATM = so I'm > monitoring that too. >=20 > Hopefully there will be a nice conclusion come from that how people = want to > proceed and we'll be able to get a change in that works for everyone. Hmm. I wonder if the simplest approach would be the better. I mean, = adding a flag to zpool. At home I have a playground FreeBSD machine with a ZFS zmirror, and, you = guessed it, I was careless when I purchased the components, I asked for two "1 TB drives" = and that I got, but different models, one of them "advanced format" and the other one "classic". I don't think it's that bad to create a pool on a classic disk using 4 = KB blocks, and it's quite likely that replacement disks will be 4 KB in the near future.=20 Also, if you use SSDs the situation is similar. Borja. From owner-zfs-devel@FreeBSD.ORG Wed Jul 10 11:10:44 2013 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 2686ED3A; Wed, 10 Jul 2013 11:10:44 +0000 (UTC) (envelope-from prvs=1903808b5b=killing@multiplay.co.uk) Received: from mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) by mx1.freebsd.org (Postfix) with ESMTP id 5096715E8; Wed, 10 Jul 2013 11:10:43 +0000 (UTC) Received: from r2d2 ([82.69.141.170]) by mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) (MDaemon PRO v10.0.4) with ESMTP id md50004904388.msg; Wed, 10 Jul 2013 12:10:35 +0100 X-Spam-Processed: mail1.multiplay.co.uk, Wed, 10 Jul 2013 12:10:35 +0100 (not processed: message from valid local sender) X-MDDKIM-Result: neutral (mail1.multiplay.co.uk) X-MDRemoteIP: 82.69.141.170 X-Return-Path: prvs=1903808b5b=killing@multiplay.co.uk X-Envelope-From: killing@multiplay.co.uk Message-ID: <197F9EAB64AD4A1FBC4DD75F7255D55D@multiplay.co.uk> From: "Steven Hartland" To: "Borja Marcos" References: <86zjtupz3r.fsf@nine.des.no> <628C5D1AF6044488B708484203D70B7A@multiplay.co.uk> <774B60E8-19C2-4A3A-880D-0D8726DC6727@sarenet.es> Subject: Re: Make ZFS use the physical sector size when computing initial ashift Date: Wed, 10 Jul 2013 12:10:54 +0100 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="iso-8859-1"; reply-type=original Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.5931 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.6157 Cc: freebsd-fs@freebsd.org, =?iso-8859-1?Q?Dag-Erling_Sm=F8rgrav?= , zfs-devel@FreeBSD.org, ivoras@freebsd.org, freebsd-hackers@freebsd.org X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 10 Jul 2013 11:10:44 -0000 There's lots more to consider when considering a way foward not least of all ashift isn't a zpool configuration option is per top level vdev, space consideration of moving from 512b to 4k, see previous and current discussions on zfs-devel@freebsd.org and zfs@lists.illumos.org for details. Regards Steve ----- Original Message ----- From: "Borja Marcos" On Jul 10, 2013, at 11:25 AM, Steven Hartland wrote: > If others are interested I've attached this as it achieves what we needed here so > may also be of use for others too. > > There's also a big discussion on illumos about this very subject ATM so I'm > monitoring that too. > > Hopefully there will be a nice conclusion come from that how people want to > proceed and we'll be able to get a change in that works for everyone. Hmm. I wonder if the simplest approach would be the better. I mean, adding a flag to zpool. At home I have a playground FreeBSD machine with a ZFS zmirror, and, you guessed it, I was careless when I purchased the components, I asked for two "1 TB drives" and that I got, but different models, one of them "advanced format" and the other one "classic". I don't think it's that bad to create a pool on a classic disk using 4 KB blocks, and it's quite likely that replacement disks will be 4 KB in the near future. Also, if you use SSDs the situation is similar. ================================================ This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmaster@multiplay.co.uk. From owner-zfs-devel@FreeBSD.ORG Wed Jul 10 11:49:31 2013 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 8AA23320 for ; Wed, 10 Jul 2013 11:49:31 +0000 (UTC) (envelope-from goran.lowkrantz@hidden-powers.com) Received: from mail.hidden-powers.com (mail.hidden-powers.com [IPv6:2a00:f680:101:fffe::5]) by mx1.freebsd.org (Postfix) with ESMTP id 449271859 for ; Wed, 10 Jul 2013 11:49:31 +0000 (UTC) Received: from mail.hidden-powers.com (localhost [127.0.0.1]) by dkim.hidden-powers.com (Postfix) with ESMTP id A24F837B512 for ; Wed, 10 Jul 2013 11:49:21 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=hidden-powers.com; h=date :from:to:subject:message-id:mime-version:content-type :content-transfer-encoding; s=selector1; bh=uoq1oCgLlTqpdDX/iUbL y7J1Wic=; b=Fwu1mma+HZD7KtUlWYY5TNrx14jO/h/mI25ffg+DkpAmnY9D1vPX T2JG7oO8WDwFQy08iqtIHdDJHYZk3TNV7EnVzQU/12a8JcvbEb2cwNYKMN1ah/Bn ndZU1FfGUeqNOEGquP2z8zycJxU1aSFJNiERhr/T9lDJkPSnifvxwpw= DomainKey-Signature: a=rsa-sha1; c=nofws; d=hidden-powers.com; h=date :from:to:subject:message-id:mime-version:content-type :content-transfer-encoding; q=dns; s=selector1; b=odGEzw9SgonwtD Or5Vgxhj6CGk35aMEL3AIw6UAmPJOfDUx7rqrNljJtdgGsgBFfCp1BOFGDdOiXpz jQErJe9aYFQ59uMnSp3IXc0kz43ACjQSoDY22r1+oez029H5x9DPYa5GpaEVOUFn mrO69rBW7nxe5EzHucZ+pZ2X7yjmg= Received: from [172.16.2.45] (unknown [176.57.193.190]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.hidden-powers.com (Postfix) with ESMTPSA id 3E40A37B510 for ; Wed, 10 Jul 2013 11:49:21 +0000 (UTC) Date: Wed, 10 Jul 2013 13:49:19 +0200 From: Goran Lowkrantz , Goran Lowkrantz To: zfs-devel@freebsd.org Subject: Subsctibe Message-ID: X-Mailer: Mulberry/4.0.8 (Mac OS X) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Content-Disposition: inline X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 10 Jul 2013 11:49:31 -0000 From owner-zfs-devel@FreeBSD.ORG Fri Jul 12 09:21:19 2013 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id A66229AB; Fri, 12 Jul 2013 09:21:19 +0000 (UTC) (envelope-from martin@matuska.org) Received: from mail.vx.sk (mail.vx.sk [IPv6:2a01:4f8:150:6101::4]) by mx1.freebsd.org (Postfix) with ESMTP id 6B56018C0; Fri, 12 Jul 2013 09:21:19 +0000 (UTC) Received: from core.vx.sk (localhost [127.0.0.2]) by mail.vx.sk (Postfix) with ESMTP id A0CA1359E7; Fri, 12 Jul 2013 11:21:18 +0200 (CEST) X-Virus-Scanned: amavisd-new at mail.vx.sk Received: from mail.vx.sk by core.vx.sk (amavisd-new, unix socket) with LMTP id Kuia8uWhm-gi; Fri, 12 Jul 2013 11:21:18 +0200 (CEST) Received: from localhost (localhost [127.0.0.2]) by mail.vx.sk (Postfix) with ESMTPSA id AD192359DA; Fri, 12 Jul 2013 11:21:17 +0200 (CEST) Received: from 145.243.194.209 ([145.243.194.209]) by mail.vx.sk (Horde Framework) with HTTP; Fri, 12 Jul 2013 11:21:17 +0200 Date: Fri, 12 Jul 2013 11:21:17 +0200 Message-ID: <20130712112117.Horde.JMB3fhsU1VfZRxdV4prj5Q6@mail.vx.sk> From: Martin =?utf-8?b?TWF0dcWha2E=?= To: "Ahrens, Matthew" , Xin Li , Steven Hartland Subject: N-way mirror read speedup in zfsonlinux User-Agent: Internet Messaging Program (IMP) H5 (6.1.2) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed; DelSp=Yes Content-Disposition: inline Content-Description: =?utf-8?b?U3Byw6F2YQ==?= s =?utf-8?b?xI1pc3TDvW0=?= textom X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: zfs-devel@freebsd.org X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 12 Jul 2013 09:21:19 -0000 Hi everyone, zfsonlinux has implemented a change in the N-way mirror device selection algorithm by selecting the device with the least pending I/O instead of random selection. They measured an increased read bandwidth increase up to 50% and IOPS increase up to 10%. this might be useful for common ZFS code and we might consider porting this to illumos and FreeBSD: https://github.com/zfsonlinux/zfs/issues/1461 https://github.com/zfsonlinux/zfs/commit/556011dbec2d10579819078559a77630fc559112 Cheers, mm From owner-zfs-devel@FreeBSD.ORG Fri Jul 12 14:18:42 2013 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 9D888944; Fri, 12 Jul 2013 14:18:42 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-qe0-x22e.google.com (mail-qe0-x22e.google.com [IPv6:2607:f8b0:400d:c02::22e]) by mx1.freebsd.org (Postfix) with ESMTP id 544EC190D; Fri, 12 Jul 2013 14:18:42 +0000 (UTC) Received: by mail-qe0-f46.google.com with SMTP id nd7so5100223qeb.19 for ; Fri, 12 Jul 2013 07:18:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=T7u5Nhmo5Pixf/Sv1ktnp+yvi6IerGJkrabuFc/ockc=; b=CxJQzvlOSzRAaVYHM9rO6+O/SD32ComCIszQAuPgp86F058rWWjZ+H9LD499GCMtrI TtoAqqfznmHfNEaXCTBD+1BOqlHau/HBtyixcC11aoe9Nan2FIR1uAAW7b8wi+C+hhEJ 7f9bJV01Wy7F5SKfu0V/23A6WB4A6Wh6dAva7Pp1o62qSrPUuWNkka3JrueWNBE3fVR4 XrlPtSuoPx5iyk0HIS6U8EVLfLhZlToV7hpZZO/SLzmKQhsbu6LCsvMeo8tCv+5xqB8k hwy1miktoesLBOh1s4FnLrpBurch5Yd7uUKuP/nsmh7gAbC+jFUBA8iXRAAfGEbmGGFq 6Tww== MIME-Version: 1.0 X-Received: by 10.224.121.77 with SMTP id g13mr37881736qar.49.1373638721895; Fri, 12 Jul 2013 07:18:41 -0700 (PDT) Received: by 10.49.37.226 with HTTP; Fri, 12 Jul 2013 07:18:41 -0700 (PDT) In-Reply-To: <20130712112117.Horde.JMB3fhsU1VfZRxdV4prj5Q6@mail.vx.sk> References: <20130712112117.Horde.JMB3fhsU1VfZRxdV4prj5Q6@mail.vx.sk> Date: Fri, 12 Jul 2013 08:18:41 -0600 Message-ID: Subject: Re: N-way mirror read speedup in zfsonlinux From: asomers@gmail.com To: =?UTF-8?Q?Martin_Matu=C5=A1ka?= Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: "Ahrens, Matthew" , Xin Li , zfs-devel@freebsd.org X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 12 Jul 2013 14:18:42 -0000 Looks pretty good. I'll put this on my list of stuff to test in the next couple of weeks as I start working more on performance. On Fri, Jul 12, 2013 at 3:21 AM, Martin Matu=C5=A1ka w= rote: > Hi everyone, > > zfsonlinux has implemented a change in the N-way mirror device selection > algorithm by selecting the device with the least pending I/O instead of > random selection. They measured an increased read bandwidth increase up t= o > 50% and IOPS increase up to 10%. > > this might be useful for common ZFS code and we might consider porting th= is > to illumos and FreeBSD: > https://github.com/zfsonlinux/zfs/issues/1461 > https://github.com/zfsonlinux/zfs/commit/556011dbec2d10579819078559a77630= fc559112 > > Cheers, > mm > _______________________________________________ > zfs-devel@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/zfs-devel > To unsubscribe, send any mail to "zfs-devel-unsubscribe@freebsd.org" From owner-zfs-devel@FreeBSD.ORG Fri Jul 12 18:43:02 2013 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id F14AF854 for ; Fri, 12 Jul 2013 18:43:02 +0000 (UTC) (envelope-from will@firepipe.net) Received: from mail-ve0-f181.google.com (mail-ve0-f181.google.com [209.85.128.181]) by mx1.freebsd.org (Postfix) with ESMTP id B82C9185B for ; Fri, 12 Jul 2013 18:43:02 +0000 (UTC) Received: by mail-ve0-f181.google.com with SMTP id db10so8350703veb.26 for ; Fri, 12 Jul 2013 11:42:56 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type:content-transfer-encoding:x-gm-message-state; bh=vQqSRB4eIR4vcqjMi+D1cLpQdrDujYdzbgHVDvJxkWs=; b=kY5V8ehITNihdB1epA9d0rLNljkOvWdlWDrNSEn/YoGvXjwvY3mSX98Xwcl3aHUVlk 8ySBnz11NLdN+6MbXlUUwSa+gAoiCHIdCSfrGdmPOFI4ceAHDrKri4W8epQADuzfbi1n yayFhu2rJHwohNKnIZD3ryd9CXW1MNJNANstWd8SUarpzUg0i2kHsvO4lmLhY0TtrFjj d34ctRVnv4XuGASK7lLbzeX1zBSAgeV3y+u+8cjjujgLUDVaBLGOpYZh+cUfC2XzLvq7 r4LejKJGAGoa/Aua55KTd2owIa86vptO63cw1Z7S+MOpip+gAAnhvDxptMSBIOAWccku FbDA== MIME-Version: 1.0 X-Received: by 10.58.54.70 with SMTP id h6mr25090489vep.36.1373654576509; Fri, 12 Jul 2013 11:42:56 -0700 (PDT) Received: by 10.58.226.66 with HTTP; Fri, 12 Jul 2013 11:42:56 -0700 (PDT) In-Reply-To: <20130712112117.Horde.JMB3fhsU1VfZRxdV4prj5Q6@mail.vx.sk> References: <20130712112117.Horde.JMB3fhsU1VfZRxdV4prj5Q6@mail.vx.sk> Date: Fri, 12 Jul 2013 12:42:56 -0600 Message-ID: Subject: Re: N-way mirror read speedup in zfsonlinux From: Will Andrews To: zfs-devel@freebsd.org Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Gm-Message-State: ALoCoQn8k8R+79FzjCqyIh4h20WbzwsKnS0dnZ6Qwv07tRYMytgDYZ2TJbqIgTYrLXJZZG3F+/II X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 12 Jul 2013 18:43:03 -0000 On Fri, Jul 12, 2013 at 3:21 AM, Martin Matu=C5=A1ka w= rote: > zfsonlinux has implemented a change in the N-way mirror device selection > algorithm by selecting the device with the least pending I/O instead of > random selection. They measured an increased read bandwidth increase up t= o > 50% and IOPS increase up to 10%. > > this might be useful for common ZFS code and we might consider porting th= is > to illumos and FreeBSD: > https://github.com/zfsonlinux/zfs/issues/1461 > https://github.com/zfsonlinux/zfs/commit/556011dbec2d10579819078559a77630= fc559112 Just a general comment: This is a well-documented improvement, much better than many I've seen. Not only was the research process documented, but so was much of the change itself. I think we would all benefit if all other ZFS changes had this level of documentation. --Will. From owner-zfs-devel@FreeBSD.ORG Mon Jul 15 11:08:15 2013 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 90EB273D for ; Mon, 15 Jul 2013 11:08:15 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) by mx1.freebsd.org (Postfix) with ESMTP id 6C32219F for ; Mon, 15 Jul 2013 11:08:15 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.7/8.14.7) with ESMTP id r6FB8FLC086465 for ; Mon, 15 Jul 2013 11:08:15 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.7/8.14.7/Submit) id r6FB8F3V086463 for zfs-devel@FreeBSD.org; Mon, 15 Jul 2013 11:08:15 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 15 Jul 2013 11:08:15 GMT Message-Id: <201307151108.r6FB8F3V086463@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: zfs-devel@FreeBSD.org Subject: Current problem reports assigned to zfs-devel@FreeBSD.org X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 Jul 2013 11:08:15 -0000 Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- s kern/178467 zfs-devel [zfs] [request] Optimized Checksum Code for ZFS o kern/178387 zfs-devel [zfs] [patch] sparse files performance improvements 2 problems total. From owner-zfs-devel@FreeBSD.ORG Mon Jul 15 22:07:17 2013 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id D302620B for ; Mon, 15 Jul 2013 22:07:17 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: from mail-la0-x233.google.com (mail-la0-x233.google.com [IPv6:2a00:1450:4010:c03::233]) by mx1.freebsd.org (Postfix) with ESMTP id 59382DA8 for ; Mon, 15 Jul 2013 22:07:17 +0000 (UTC) Received: by mail-la0-f51.google.com with SMTP id fq12so9707908lab.24 for ; Mon, 15 Jul 2013 15:07:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=delphix.com; s=google; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=hLkmdhFOJYf7jfk2iAIl8vN63cevj6ip1xJ3NbyXvpc=; b=OURMgSpfCqcVqKsANdlqE9EMyF/h/bo88P4LV9NF/qhkB8vowWgRTbYjU46iQgFD+D +aGMwkVGhrurAi7lssDJgzbsvV/egNIBFvAa/PTTCr2WlPmN8UIuh9RuelKBRBQ4US5/ 98DAr+LTmYAe670/Iws8HaR3SDsOG1kA0tjWE= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:x-gm-message-state; bh=hLkmdhFOJYf7jfk2iAIl8vN63cevj6ip1xJ3NbyXvpc=; b=mJZFu9S11oJ0pelTmxJsTqUOKJqkUY2dEipc2r52W8p8uFe1XEjrJZ/C7w73nz7dfz biD8U5QznVDQqoBPC0Q38MS8TayhvsLkSpHtJ3IuZz7cEqMCeXaaOeVqwnghTnbObCxe Vd3M/ZROS+26CezgH6afqmmxoQIzqe+UyoyZQhgPqYxe9jx/0A9xrRq/FRBtykw3gfA7 pqzgzlW/vpvI2VDaU4z9LI6dFfX4aGYkahUoYdXvSt270j1lFSn0uPRjfB8NqFxoAMrQ vN2gFhLqUZgzm48bvSdqXphlU5fF/oRUYe+q2HUT+tyRo6KKTQuk3OvzqMDiygs2/31l sfNw== MIME-Version: 1.0 X-Received: by 10.152.4.137 with SMTP id k9mr25929581lak.11.1373926035711; Mon, 15 Jul 2013 15:07:15 -0700 (PDT) Received: by 10.114.57.3 with HTTP; Mon, 15 Jul 2013 15:07:15 -0700 (PDT) In-Reply-To: <20130712112117.Horde.JMB3fhsU1VfZRxdV4prj5Q6@mail.vx.sk> References: <20130712112117.Horde.JMB3fhsU1VfZRxdV4prj5Q6@mail.vx.sk> Date: Mon, 15 Jul 2013 15:07:15 -0700 Message-ID: Subject: Re: N-way mirror read speedup in zfsonlinux From: Matthew Ahrens To: =?UTF-8?Q?Martin_Matu=C5=A1ka?= X-Gm-Message-State: ALoCoQlmSzY4K7z/GZ6c4z+Vzsem99b1NW6aFczXL7LgJgs8BoRJZRYYjTDlK51U1CZ4PP5VXhhM Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: zfs-devel@freebsd.org, Xin Li , George Wilson X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 Jul 2013 22:07:18 -0000 I'm not super familiar with this code, but I like the idea. --matt On Fri, Jul 12, 2013 at 2:21 AM, Martin Matu=C5=A1ka w= rote: > ** > > Hi everyone, > > zfsonlinux has implemented a change in the N-way mirror device selection > algorithm by selecting the device with the least pending I/O instead of > random selection. They measured an increased read bandwidth increase up t= o > 50% and IOPS increase up to 10%. > > this might be useful for common ZFS code and we might consider porting > this to illumos and FreeBSD: > https://github.com/zfsonlinux/zfs/issues/1461 > > https://github.com/zfsonlinux/zfs/commit/556011dbec2d10579819078559a77630= fc559112 > > Cheers, > mm > From owner-zfs-devel@FreeBSD.ORG Wed Jul 17 11:04:31 2013 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id EE5816A1; Wed, 17 Jul 2013 11:04:30 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 0040196E; Wed, 17 Jul 2013 11:04:29 +0000 (UTC) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id OAA02116; Wed, 17 Jul 2013 14:04:21 +0300 (EEST) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1UzPWu-00005Q-V1; Wed, 17 Jul 2013 14:04:21 +0300 Message-ID: <51E679FD.3040306@FreeBSD.org> Date: Wed, 17 Jul 2013 14:03:25 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:17.0) Gecko/20130708 Thunderbird/17.0.7 MIME-Version: 1.0 To: zfs-devel@FreeBSD.org, freebsd-fs@FreeBSD.org Subject: zfs_rename: another zfs+vfs deadlock X-Enigmail-Version: 1.5.1 Content-Type: text/plain; charset=X-VIET-VPS Content-Transfer-Encoding: 7bit X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Jul 2013 11:04:31 -0000 I received a report about what looked like a deadlock involving ZFS. Interesting bits from the report are: Thread 1156 (Thread 2038380): #0 sched_switch (td=0xfffffe01a9e56460, newtd=0xfffffe001c3ff000, flags=Variable "flags" is not available. ) at /usr/src/sys/kern/sched_ule.c:1860 #1 0xffffffff808ab51a in mi_switch (flags=260, newtd=0x0) at /usr/src/sys/kern/kern_synch.c:466 #2 0xffffffff808e3e63 in sleepq_switch (wchan=0xfffffe06d9df2310, pri=96) at /usr/src/sys/kern/subr_sleepqueue.c:538 #3 0xffffffff808e4a9d in sleepq_wait (wchan=0xfffffe06d9df2310, pri=96) at /usr/src/sys/kern/subr_sleepqueue.c:617 #4 0xffffffff8088aebb in __lockmgr_args (lk=0xfffffe06d9df2310, flags=524544, ilk=0xfffffe06d9df23d8, wmesg=Variable "wmesg" is not available. ) at /usr/src/sys/kern/kern_lock.c:214 #5 0xffffffff8092d349 in vop_stdlock (ap=Variable "ap" is not available. ) at lockmgr.h:97 #6 0xffffffff80bd62ab in VOP_LOCK1_APV (vop=0xffffffff8111cc80, a=0xffffff90729ee6e0) at vnode_if.c:1988 #7 0xffffffff8094cfa7 in _vn_lock (vp=0xfffffe06d9df2278, flags=524288, file=Variable "file" is not available. ) at vnode_if.h:859 #8 0xffffffff80942220 in vputx (vp=0xfffffe06d9df2278, func=1) at /usr/src/sys/kern/vfs_subr.c:2279 #9 0xffffffff816e75a4 in zfs_rename_unlock (zlpp=0xffffff90729ee878) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c:3501 #10 0xffffffff816e8df4 in zfs_freebsd_rename (ap=Variable "ap" is not available. ) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c:3927 #11 0xffffffff80bd67cb in VOP_RENAME_APV (vop=0xffffffff8175b900, a=0xffffff90729eeaa0) at vnode_if.c:1474 #12 0xffffffff80947844 in kern_renameat (td=Variable "td" is not available. ) at vnode_if.h:636 #13 0xffffffff80b4eff2 in amd64_syscall (td=0xfffffe01a9e56460, traced=0) at subr_syscall.c:135 #14 0xffffffff80b39b97 in Xfast_syscall () at /usr/src/sys/amd64/amd64/exception.S:387 fr 11 p *a $1 = {a_gen = {a_desc = 0xffffffff811594c0}, a_fdvp = 0xfffffe05fb094278, a_fvp = 0xfffffe04b7b62278, a_fcnp = 0xffffff90729eea58, a_tdvp = 0xfffffe0514137768, a_tvp = 0x0, a_tcnp = 0xffffff90729ee9a8} Thread 1158 (Thread 4174978): #0 sched_switch (td=0xfffffe088cbef000, newtd=0xfffffe001c40e000, flags=Variable "flags" is not available. ) at /usr/src/sys/kern/sched_ule.c:1860 #1 0xffffffff808ab51a in mi_switch (flags=260, newtd=0x0) at /usr/src/sys/kern/kern_synch.c:466 #2 0xffffffff808e3e63 in sleepq_switch (wchan=0xfffffe0514137800, pri=96) at /usr/src/sys/kern/subr_sleepqueue.c:538 #3 0xffffffff808e4a9d in sleepq_wait (wchan=0xfffffe0514137800, pri=96) at /usr/src/sys/kern/subr_sleepqueue.c:617 #4 0xffffffff8088b4e0 in __lockmgr_args (lk=0xfffffe0514137800, flags=2097152, ilk=0xfffffe05141378c8, wmesg=Variable "wmesg" is not available. ) at /usr/src/sys/kern/kern_lock.c:214 #5 0xffffffff8092d349 in vop_stdlock (ap=Variable "ap" is not available. ) at lockmgr.h:97 #6 0xffffffff80bd62ab in VOP_LOCK1_APV (vop=0xffffffff8111cc80, a=0xffffff9072813470) at vnode_if.c:1988 #7 0xffffffff8094cfa7 in _vn_lock (vp=0xfffffe0514137768, flags=2097152, file=Variable "file" is not available. ) at vnode_if.h:859 #8 0xffffffff816e5bdd in zfs_vnode_lock (vp=0xfffffe0514137768, flags=Variable "flags" is not available. ) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vfsops.c:1704 #9 0xffffffff816e6d70 in zfs_lookup (dvp=0xfffffe06d9df2278, nm=0xffffff90728135b0 "toBeDeleted", vpp=0xffffff9072813930, cnp=0xffffff9072813958, nameiop=0, cr=0xfffffe0ba89b0a00, td=0xfffffe088cbef000, flags=0) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c:1433 #10 0xffffffff816e7511 in zfs_freebsd_lookup (ap=0xffffff9072813710) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c:5758 #11 0xffffffff80bd5d3f in VOP_CACHEDLOOKUP_APV (vop=0xffffffff8175b900, a=0xffffff9072813710) at vnode_if.c:187 #12 0xffffffff8092b103 in vfs_cache_lookup (ap=Variable "ap" is not available. ) at vnode_if.h:80 #13 0xffffffff80bd7187 in VOP_LOOKUP_APV (vop=0xffffffff8175b900, a=0xffffff90728137d0) at vnode_if.c:123 #14 0xffffffff8093260a in lookup (ndp=0xffffff90728138f0) at vnode_if.h:54 #15 0xffffffff8093354e in namei (ndp=0xffffff90728138f0) at /usr/src/sys/kern/vfs_lookup.c:297 #16 0xffffffff80944213 in kern_statat_vnhook (td=0xfffffe088cbef000, flag=Variable "flag" is not available. ) at /usr/src/sys/kern/vfs_syscalls.c:2432 #17 0xffffffff809443b5 in kern_statat (td=Variable "td" is not available. ) at /usr/src/sys/kern/vfs_syscalls.c:2413 #18 0xffffffff8094455a in sys_stat (td=Variable "td" is not available. ) at /usr/src/sys/kern/vfs_syscalls.c:2374 #19 0xffffffff80b4eff2 in amd64_syscall (td=0xfffffe088cbef000, traced=0) at subr_syscall.c:135 #20 0xffffffff80b39b97 in Xfast_syscall () at /usr/src/sys/amd64/amd64/exception.S:387 As far as I understand the code, I think that zfs_rename_lock (called from zfs_rename) iterates up ancestor chain of target directory (tdzp) and obtains a reference on each of the ancestors via zfs_zget. zfs_rename_unlock does the opposite - it iterates in the reverse order and VN_RELE-s the ancestor znodes. As you can see above, on FreeBSD VN_RELE translates to vputx, which internally needs to obtain a vnode lock. The problem seems to be is that VOP_RENAME -> zfs_freebsd_rename is called with locked tdvp (and perhaps non-NULL and thus locked tvp). tdvp's vnode lock is released at the very end of zfs_freebsd_rename and so it is held over zfs_rename_unlock. And that means that vnode locks of tvp's ancestors can be acquired while tdvp's vnode lock is held. That violates the VFS lock ordering where a descendant's lock must always be acquired after an ancestor's lock. So that could lead to a deadlock with another VFS operation that acquires locks in the proper order. In the above snippet 0xfffffe06d9df2278 is a directory/ancestor of tdvp and 0xfffffe0514137768 is tdvp. VOP_LOOKUP -> zfs_lookup acquires the locks in the correct order (dvp is the ancestor while vp is the tdvp) while zfs_rename does it in the opposite order. A scenario to reproduce this bug could be like this. mkdir a mkdir a/b mv some-file a/b/ (in parallel with) stat a/b Of course it would have to be repeated many times to hit the right timing window. Also, namecache could interfere with this scenario, but I am not sure. -- Andriy Gapon From owner-zfs-devel@FreeBSD.ORG Wed Jul 17 19:46:05 2013 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id A8030E47; Wed, 17 Jul 2013 19:46:05 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) by mx1.freebsd.org (Postfix) with ESMTP id 0E2C07BC; Wed, 17 Jul 2013 19:46:04 +0000 (UTC) Received: from tom.home (kostik@localhost [127.0.0.1]) by kib.kiev.ua (8.14.7/8.14.7) with ESMTP id r6HJjvNV095405; Wed, 17 Jul 2013 22:45:57 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.8.3 kib.kiev.ua r6HJjvNV095405 Received: (from kostik@localhost) by tom.home (8.14.7/8.14.7/Submit) id r6HJjvUH095403; Wed, 17 Jul 2013 22:45:57 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Wed, 17 Jul 2013 22:45:57 +0300 From: Konstantin Belousov To: Andriy Gapon Subject: Re: zfs_rename: another zfs+vfs deadlock Message-ID: <20130717194557.GU5991@kib.kiev.ua> References: <51E679FD.3040306@FreeBSD.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="/Isdj7O9hWi8F9Bn" Content-Disposition: inline In-Reply-To: <51E679FD.3040306@FreeBSD.org> User-Agent: Mutt/1.5.21 (2010-09-15) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on tom.home Cc: freebsd-fs@FreeBSD.org, zfs-devel@FreeBSD.org X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Jul 2013 19:46:05 -0000 --/Isdj7O9hWi8F9Bn Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, Jul 17, 2013 at 02:03:25PM +0300, Andriy Gapon wrote: > A scenario to reproduce this bug could be like this. > mkdir a > mkdir a/b > mv some-file a/b/ (in parallel with) stat a/b > Of course it would have to be repeated many times to hit the right timing > window. Also, namecache could interfere with this scenario, but I am not= sure. >=20 There is no questions or proposals on how to approach the fix, JFYI mail ? I recommend you to look at the ufs_checkpath() and its use in the ufs_rename(). --/Isdj7O9hWi8F9Bn Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.20 (FreeBSD) iQIcBAEBAgAGBQJR5vR0AAoJEJDCuSvBvK1BevYP/03MlbINCVbX1tI9KuT02IPF KK1YPWykvf11h/GmeONiZv3qZjvYWe9jwkga4f9Hrb6DjAhIZS+3MuIwLK12yANd xfNNFF7XMHcoxyvuF4wDeufgn04ttRgREV0vaDFnODL+fMhzuz7sfjXI4lM9x6+0 nZaAjsS8eR2rYgC2z0oPRyBK+/mMldayM5FWUXBynLpkjgwlk7XP7A6BX9Fw7Mtp vFVKtGSg613ugUYZWwgI5gzJbUjtGCO7l6gQyYQCDGBeetWmyPLRHfz2aS+KsPEI cpG5vi7ruXcA9KMUg8jW9M+9qyMcCKWsnkkTUcpUOXNhbpDMaRKthGM1MVSu8HA6 Q1KfdVuXWPYgg8GJvrBXo6UjgPQmzp/Gw2a4SE/DcHhZ4ouusU0lxX0TOErf+wHW 4i8vWCJO4zk7HIpX546wLqF7eOzDSGJ3VdCkWNheeO6ca7f8wAW8f2/8mD1iBdZo s3wcGSfAKcYXJMX5J7SwTtFtv8V36lU4+XxOo0KiW/tDTu07sPyo7Zgw6iRwnlr+ +KYJzqTI0RftjD0lKlJPYZJTSYIPYffzu9fweiyrO9BbzQf/k+amDK00k30oy1D9 zf0olSwJN+2FhfnzQJf9P+3Urq10JilpmH4xJwuy3M8yKtqQ/eLh4no2ojAORErl nr17M0hGUNV9MUHmaDwL =zpSb -----END PGP SIGNATURE----- --/Isdj7O9hWi8F9Bn-- From owner-zfs-devel@FreeBSD.ORG Fri Jul 19 04:04:50 2013 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id BB00BF3; Fri, 19 Jul 2013 04:04:50 +0000 (UTC) (envelope-from olivier777a7@gmail.com) Received: from mail-qc0-x22e.google.com (mail-qc0-x22e.google.com [IPv6:2607:f8b0:400d:c01::22e]) by mx1.freebsd.org (Postfix) with ESMTP id 73D16D4A; Fri, 19 Jul 2013 04:04:50 +0000 (UTC) Received: by mail-qc0-f174.google.com with SMTP id m15so2109913qcq.5 for ; Thu, 18 Jul 2013 21:04:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=/VwWr052W/qjaNDzVlBXPIalwkJVETCK6q/NvH8p57o=; b=mXnFjTZ6k/sNdijpQV+vVpTfYysMyXnZuZvYq8mGEQs43UIX0AML6M7q7K3pCWmci/ mzhJEideaLIpnZwXMh2fSMZnZhU3EJxkyUtTEPLS4Ad/gZX9RQsXvTuCnbVlBUbDalke usQkxMJPk4x9CiXoiZ868EaXZz0mGKJEH/8EonyrsQ6qAVxVvqBcvtIruwy2INcJvqVs jR9UKtwpEf24ykKVQWI9+2Q6pYlOJwKgI3e24J8Uxu+b6Dn1k7G2zq5gxBrUDTOQGqLW iO4vDPSng3ryzs84SM8MsYtjIcLYW12BGCqCZPgDrZYxvZ5gjfGi6HipWIWnlKWVJVVX JEtg== MIME-Version: 1.0 X-Received: by 10.224.65.202 with SMTP id k10mr16754534qai.69.1374206689864; Thu, 18 Jul 2013 21:04:49 -0700 (PDT) Received: by 10.49.97.5 with HTTP; Thu, 18 Jul 2013 21:04:49 -0700 (PDT) Date: Thu, 18 Jul 2013 21:04:49 -0700 Message-ID: Subject: 9.2PRERELEASE ZFS panic in lzjb_compress From: olivier To: "freebsd-stable@freebsd.org" , zfs-devel@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Jul 2013 04:04:50 -0000 Hi, Running 9.2-PRERELEASE #19 r253313 I got the following panic Fatal trap 12: page fault while in kernel mode cpuid = 22; apic id = 46 fault virtual address = 0xffffff827ebca30c fault code = supervisor read data, page not present instruction pointer = 0x20:0xffffffff81983055 stack pointer = 0x28:0xffffffcf75bd60a0 frame pointer = 0x28:0xffffffcf75bd68f0 code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags = interrupt enabled, resume, IOPL = 0 current process = 0 (zio_write_issue_hig) trap number = 12 panic: page fault cpuid = 22 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2a/frame 0xffffffcf75bd5b30 kdb_backtrace() at kdb_backtrace+0x37/frame 0xffffffcf75bd5bf0 panic() at panic+0x1ce/frame 0xffffffcf75bd5cf0 trap_fatal() at trap_fatal+0x290/frame 0xffffffcf75bd5d50 trap_pfault() at trap_pfault+0x211/frame 0xffffffcf75bd5de0 trap() at trap+0x344/frame 0xffffffcf75bd5fe0 calltrap() at calltrap+0x8/frame 0xffffffcf75bd5fe0 --- trap 0xc, rip = 0xffffffff81983055, rsp = 0xffffffcf75bd60a0, rbp = 0xffffffcf75bd68f0 --- lzjb_compress() at lzjb_compress+0x185/frame 0xffffffcf75bd68f0 zio_compress_data() at zio_compress_data+0x92/frame 0xffffffcf75bd6920 zio_write_bp_init() at zio_write_bp_init+0x24b/frame 0xffffffcf75bd6970 zio_execute() at zio_execute+0xc3/frame 0xffffffcf75bd69b0 taskqueue_run_locked() at taskqueue_run_locked+0x74/frame 0xffffffcf75bd6a00 taskqueue_thread_loop() at taskqueue_thread_loop+0x46/frame 0xffffffcf75bd6a20 fork_exit() at fork_exit+0x11f/frame 0xffffffcf75bd6a70 fork_trampoline() at fork_trampoline+0xe/frame 0xffffffcf75bd6a70 --- trap 0, rip = 0, rsp = 0xffffffcf75bd6b30, rbp = 0 --- lzjb_compress+0x185 corresponds to line 85 in 80 cpy = src - offset; 81 if (cpy >= (uchar_t *)s_start && cpy != src && 82 src[0] == cpy[0] && src[1] == cpy[1] && src[2] == cpy[2]) { 83 *copymap |= copymask; 84 for (mlen = MATCH_MIN; mlen < MATCH_MAX; mlen++) 85 if (src[mlen] != cpy[mlen]) 86 break; 87 *dst++ = ((mlen - MATCH_MIN) << (NBBY - MATCH_BITS)) | 88 (offset >> NBBY); 89 *dst++ = (uchar_t)offset; I think it's the first time I've seen this panic. It happened while doing a send/receive. I have two pools with lzjb compression; I don't know which of these pools caused the problem, but one of them was the source of the send/receive. I only have a textdump but I'm happy to try to provide more information that could help anyone look into this. Thanks Olivier From owner-zfs-devel@FreeBSD.ORG Fri Jul 19 10:22:18 2013 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id BBB96A92; Fri, 19 Jul 2013 10:22:18 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id D466BED5; Fri, 19 Jul 2013 10:22:17 +0000 (UTC) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id NAA07981; Fri, 19 Jul 2013 13:22:15 +0300 (EEST) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1V07pG-0009d9-Vk; Fri, 19 Jul 2013 13:22:15 +0300 Message-ID: <51E9131F.1060707@FreeBSD.org> Date: Fri, 19 Jul 2013 13:21:19 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:17.0) Gecko/20130708 Thunderbird/17.0.7 MIME-Version: 1.0 To: Konstantin Belousov Subject: Re: zfs_rename: another zfs+vfs deadlock References: <51E679FD.3040306@FreeBSD.org> <20130717194557.GU5991@kib.kiev.ua> In-Reply-To: <20130717194557.GU5991@kib.kiev.ua> X-Enigmail-Version: 1.5.1 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: freebsd-fs@FreeBSD.org, zfs-devel@FreeBSD.org X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Jul 2013 10:22:18 -0000 on 17/07/2013 22:45 Konstantin Belousov said the following: > On Wed, Jul 17, 2013 at 02:03:25PM +0300, Andriy Gapon wrote: >> A scenario to reproduce this bug could be like this. >> mkdir a >> mkdir a/b >> mv some-file a/b/ (in parallel with) stat a/b >> Of course it would have to be repeated many times to hit the right timing >> window. Also, namecache could interfere with this scenario, but I am not sure. >> > > There is no questions or proposals on how to approach the fix, JFYI mail ? I was just reporting the problem and my analysis of it. A question of "how to fix" was implied. > I recommend you to look at the ufs_checkpath() and its use in the > ufs_rename(). Thank you. That code is enlightening. I do not think that the approach is directly applicable to zfs_rename, unfortunately. But I will try to see if the same kind of approach could be used. Also, I noticed that ufs_rename() checks for cross-device rename. Should all filesystems do that or should that check belong to VFS layer (if not already done there)? -- Andriy Gapon From owner-zfs-devel@FreeBSD.ORG Fri Jul 19 13:52:52 2013 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 06D9CBF8; Fri, 19 Jul 2013 13:52:52 +0000 (UTC) (envelope-from c.kworr@gmail.com) Received: from mail-lb0-x22c.google.com (mail-lb0-x22c.google.com [IPv6:2a00:1450:4010:c04::22c]) by mx1.freebsd.org (Postfix) with ESMTP id 5F81BB51; Fri, 19 Jul 2013 13:52:51 +0000 (UTC) Received: by mail-lb0-f172.google.com with SMTP id v20so3493511lbc.3 for ; Fri, 19 Jul 2013 06:52:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:content-type:content-transfer-encoding; bh=bgM3m8esa5/XraltbhB5pSqLrutuTLAVM8S3aIkkBfg=; b=KlZVpCBoP95A/aIPcuUO1JZLZ/2cGbbxpOF25d6Rh1VcVSOMCm1yuqxTu2M1/AMDoQ RehASXurcEUlEyljT67n/Tfv3vzfaEVWEqR9oK1pCtPjwmWpS6ZEhbtADLjvwhesyA31 vhBWIpywJHLKqEmd1MWqI4QEA1w9PFOf1BYW+dGG8q/Vtx88M5POXVhSgH0/I26/1ta6 /3pHrOV3nFUnLUpPlmaEzXrQAcwahHTg46D3hzcIejfN+Nutfxy1zNsRAswbQCGgNAfz Vzq9JjKaAUNezbtogxdb8v9xjeBr1Xyh6klpn6KUN/52IQjq0jlat/EI/0A66WyIjkJW uNKg== X-Received: by 10.112.143.162 with SMTP id sf2mr7072684lbb.1.1374241970326; Fri, 19 Jul 2013 06:52:50 -0700 (PDT) Received: from [192.168.1.139] (mau.donbass.com. [92.242.127.250]) by mx.google.com with ESMTPSA id p7sm6145025lbi.15.2013.07.19.06.52.49 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Fri, 19 Jul 2013 06:52:49 -0700 (PDT) Message-ID: <51E944B0.5080409@gmail.com> Date: Fri, 19 Jul 2013 16:52:48 +0300 From: Volodymyr Kostyrko User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:17.0) Gecko/20130710 Thunderbird/17.0.7 MIME-Version: 1.0 To: olivier Subject: Re: 9.2PRERELEASE ZFS panic in lzjb_compress References: In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Cc: "freebsd-stable@freebsd.org" , zfs-devel@freebsd.org X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Jul 2013 13:52:52 -0000 19.07.2013 07:04, olivier wrote: > Hi, > Running 9.2-PRERELEASE #19 r253313 I got the following panic > > Fatal trap 12: page fault while in kernel mode > cpuid = 22; apic id = 46 > fault virtual address = 0xffffff827ebca30c > fault code = supervisor read data, page not present > instruction pointer = 0x20:0xffffffff81983055 > stack pointer = 0x28:0xffffffcf75bd60a0 > frame pointer = 0x28:0xffffffcf75bd68f0 > code segment = base 0x0, limit 0xfffff, type 0x1b > = DPL 0, pres 1, long 1, def32 0, gran 1 > processor eflags = interrupt enabled, resume, IOPL = 0 > current process = 0 (zio_write_issue_hig) > trap number = 12 > panic: page fault > cpuid = 22 > KDB: stack backtrace: > db_trace_self_wrapper() at db_trace_self_wrapper+0x2a/frame > 0xffffffcf75bd5b30 > kdb_backtrace() at kdb_backtrace+0x37/frame 0xffffffcf75bd5bf0 > panic() at panic+0x1ce/frame 0xffffffcf75bd5cf0 > trap_fatal() at trap_fatal+0x290/frame 0xffffffcf75bd5d50 > trap_pfault() at trap_pfault+0x211/frame 0xffffffcf75bd5de0 > trap() at trap+0x344/frame 0xffffffcf75bd5fe0 > calltrap() at calltrap+0x8/frame 0xffffffcf75bd5fe0 > --- trap 0xc, rip = 0xffffffff81983055, rsp = 0xffffffcf75bd60a0, rbp = > 0xffffffcf75bd68f0 --- > lzjb_compress() at lzjb_compress+0x185/frame 0xffffffcf75bd68f0 > zio_compress_data() at zio_compress_data+0x92/frame 0xffffffcf75bd6920 > zio_write_bp_init() at zio_write_bp_init+0x24b/frame 0xffffffcf75bd6970 > zio_execute() at zio_execute+0xc3/frame 0xffffffcf75bd69b0 > taskqueue_run_locked() at taskqueue_run_locked+0x74/frame 0xffffffcf75bd6a00 > taskqueue_thread_loop() at taskqueue_thread_loop+0x46/frame > 0xffffffcf75bd6a20 > fork_exit() at fork_exit+0x11f/frame 0xffffffcf75bd6a70 > fork_trampoline() at fork_trampoline+0xe/frame 0xffffffcf75bd6a70 > --- trap 0, rip = 0, rsp = 0xffffffcf75bd6b30, rbp = 0 --- > > lzjb_compress+0x185 corresponds to line 85 in > 80 cpy = src - offset; > 81 if (cpy >= (uchar_t *)s_start && cpy != src && > 82 src[0] == cpy[0] && src[1] == cpy[1] && src[2] == cpy[2]) { > 83 *copymap |= copymask; > 84 for (mlen = MATCH_MIN; mlen < MATCH_MAX; mlen++) > 85 if (src[mlen] != cpy[mlen]) > 86 break; > 87 *dst++ = ((mlen - MATCH_MIN) << (NBBY - MATCH_BITS)) | > 88 (offset >> NBBY); > 89 *dst++ = (uchar_t)offset; > > I think it's the first time I've seen this panic. It happened while doing a > send/receive. I have two pools with lzjb compression; I don't know which of > these pools caused the problem, but one of them was the source of the > send/receive. > > I only have a textdump but I'm happy to try to provide more information > that could help anyone look into this. > Thanks > Olivier Oh, I can add to this one. I have a full core dump of the same problem caused by copying large set of files from lzjb compressed pool to lz4 compressed pool. vfs.zfs.recover was set. #1 0xffffffff8039d954 in kern_reboot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:449 #2 0xffffffff8039ddce in panic (fmt=) at /usr/src/sys/kern/kern_shutdown.c:637 #3 0xffffffff80620a6a in trap_fatal (frame=, eva=) at /usr/src/sys/amd64/amd64/trap.c:879 #4 0xffffffff80620d25 in trap_pfault (frame=0x0, usermode=0) at /usr/src/sys/amd64/amd64/trap.c:700 #5 0xffffffff806204f6 in trap (frame=0xffffff821ca43600) at /usr/src/sys/amd64/amd64/trap.c:463 #6 0xffffffff8060a032 in calltrap () at /usr/src/sys/amd64/amd64/exception.S:232 #7 0xffffffff805a9367 in vm_page_alloc (object=0xffffffff80a34030, pindex=16633, req=97) at /usr/src/sys/vm/vm_page.c:1445 #8 0xffffffff8059c42e in kmem_back (map=0xfffffe00010000e8, addr=18446743524021862400, size=16384, flags=) at /usr/src/sys/vm/vm_kern.c:362 #9 0xffffffff8059c2ac in kmem_malloc (map=0xfffffe00010000e8, size=16384, flags=257) at /usr/src/sys/vm/vm_kern.c:313 #10 0xffffffff80595104 in uma_large_malloc (size=, wait=257) at /usr/src/sys/vm/uma_core.c:994 #11 0xffffffff80386b80 in malloc (size=16384, mtp=0xffffffff80ea7c40, flags=0) at /usr/src/sys/kern/kern_malloc.c:492 #12 0xffffffff80c9e13c in lz4_compress (s_start=0xffffff80d0b19000, d_start=0xffffff8159445000, s_len=131072, d_len=114688, n=-2) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/lz4.c:843 #13 0xffffffff80cdde25 in zio_compress_data (c=, src=, dst=0xffffff8159445000, s_len=131072) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zio_compress.c:109 #14 0xffffffff80cda012 in zio_write_bp_init (zio=0xfffffe0143a12000) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1107 #15 0xffffffff80cd8ec6 in zio_execute (zio=0xfffffe0143a12000) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1305 #16 0xffffffff803e25e6 in taskqueue_run_locked (queue=0xfffffe00060ca300) at /usr/src/sys/kern/subr_taskqueue.c:312 #17 0xffffffff803e2e38 in taskqueue_thread_loop (arg=) at /usr/src/sys/kern/subr_taskqueue.c:501 #18 0xffffffff8036f40a in fork_exit ( callout=0xffffffff803e2da0 , arg=0xfffffe00060cc3d0, frame=0xffffff821ca43a80) at /usr/src/sys/kern/kern_fork.c:988 #19 0xffffffff8060a56e in fork_trampoline () at /usr/src/sys/amd64/amd64/exception.S:606 I have a full crash dump in case someone wants to look at it. -- Sphinx of black quartz, judge my vow. From owner-zfs-devel@FreeBSD.ORG Fri Jul 19 18:35:04 2013 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id D9C407A5; Fri, 19 Jul 2013 18:35:04 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) by mx1.freebsd.org (Postfix) with ESMTP id 501CDB06; Fri, 19 Jul 2013 18:35:04 +0000 (UTC) Received: from tom.home (kostik@localhost [127.0.0.1]) by kib.kiev.ua (8.14.7/8.14.7) with ESMTP id r6JIZ0E7029586; Fri, 19 Jul 2013 21:35:00 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.8.3 kib.kiev.ua r6JIZ0E7029586 Received: (from kostik@localhost) by tom.home (8.14.7/8.14.7/Submit) id r6JIZ0Kr029585; Fri, 19 Jul 2013 21:35:00 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Fri, 19 Jul 2013 21:35:00 +0300 From: Konstantin Belousov To: Andriy Gapon Subject: Re: zfs_rename: another zfs+vfs deadlock Message-ID: <20130719183500.GL5991@kib.kiev.ua> References: <51E679FD.3040306@FreeBSD.org> <20130717194557.GU5991@kib.kiev.ua> <51E9131F.1060707@FreeBSD.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="Vo48LVc30GAQuLuW" Content-Disposition: inline In-Reply-To: <51E9131F.1060707@FreeBSD.org> User-Agent: Mutt/1.5.21 (2010-09-15) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on tom.home Cc: freebsd-fs@FreeBSD.org, zfs-devel@FreeBSD.org X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Jul 2013 18:35:04 -0000 --Vo48LVc30GAQuLuW Content-Type: text/plain; charset=us-ascii Content-Disposition: inline On Fri, Jul 19, 2013 at 01:21:19PM +0300, Andriy Gapon wrote: > Also, I noticed that ufs_rename() checks for cross-device rename. Should all > filesystems do that or should that check belong to VFS layer (if not already > done there)? In principle yes, this sounds right. The only concern I see is layered filesystems like nullfs interaction with filesystems below the bypass. In other words, if any bypass provided the aggregation, this should be checked at the bypass layer too, in addition to the kern_renameat(). --Vo48LVc30GAQuLuW Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.20 (FreeBSD) iQIcBAEBAgAGBQJR6YbTAAoJEJDCuSvBvK1BhGsQAJ7PmCb57BzSzDUJCwydMcr3 fOD9M8UwR1NKxAWboxmIbhqjL7bfzzzeNGTHwhPj6NqQZQeBg8Lq0lvoKEqKTT6Z lPl0acR7+V2IIwBD5wj7NBN6LkZvztXc92pUt7PmLOTi7sbNOC2r8eUIvEjyMjCC O1tN4/eZiKGOk3F6ityRNjn4h2JUkwAhfn85gMrJOQvOuxVvo/AgARcxdplZdZIv 1WzZFtfWYrRGCjNwxQ0w4qE2amZ5aJudcXJdU3qiKh8Ss9s9TkLV+ZDj6+kofng+ YCbVuQ3xD9N8EpG/bmYnZV4gzWuD4hDsHBYf3Ba3DE7rdJfek7/K4TRVLnQxBCa6 toTkJijznXFjM33qpjORaNwOvFu+dWnWKmzgDMs6Ky32eeRPPqQz7Fe8IgJMD1C9 JDMZbGHJ/wqCR+vNKGaGrlZO4EL/L54IhqY2i1r3f2/fyMKBVq5bxwIxs3c3F2sw qqF64vwsnfd1aeKUTtgCVVdaSRmrsG6hdjfgri4sMqX6GfjppAcqXf6sah9SzEcv ibNiMut4q8Z6lfb9xPwsYrzubmyelQilf111bB9g7VzZuEDsEfTJoSbIPWKXPikP 6tn29wD6+E04zvL0KrCB5QtMHUhoS1l6mfrEMPXrm2wDgGhI1zBGNIdvZfSE4XxG ToPDy+vHlxrf9lDmwM3J =ojft -----END PGP SIGNATURE----- --Vo48LVc30GAQuLuW-- From owner-zfs-devel@FreeBSD.ORG Mon Jul 22 11:08:19 2013 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 1F1828E0 for ; Mon, 22 Jul 2013 11:08:19 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id EAB5525C1 for ; Mon, 22 Jul 2013 11:08:18 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.7/8.14.7) with ESMTP id r6MB8In7055617 for ; Mon, 22 Jul 2013 11:08:18 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.7/8.14.7/Submit) id r6MB8ICm055615 for zfs-devel@FreeBSD.org; Mon, 22 Jul 2013 11:08:18 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 22 Jul 2013 11:08:18 GMT Message-Id: <201307221108.r6MB8ICm055615@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: zfs-devel@FreeBSD.org Subject: Current problem reports assigned to zfs-devel@FreeBSD.org X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Jul 2013 11:08:19 -0000 Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- s kern/178467 zfs-devel [zfs] [request] Optimized Checksum Code for ZFS o kern/178387 zfs-devel [zfs] [patch] sparse files performance improvements 2 problems total. From owner-zfs-devel@FreeBSD.ORG Wed Jul 24 20:04:03 2013 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 5CD014DB for ; Wed, 24 Jul 2013 20:04:03 +0000 (UTC) (envelope-from delphij@delphij.net) Received: from anubis.delphij.net (anubis.delphij.net [IPv6:2001:470:1:117::25]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 3564C290A for ; Wed, 24 Jul 2013 20:04:03 +0000 (UTC) Received: from zeta.ixsystems.com (drawbridge.ixsystems.com [206.40.55.65]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by anubis.delphij.net (Postfix) with ESMTPSA id BB35D1688E; Wed, 24 Jul 2013 13:04:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=delphij.net; s=anubis; t=1374696241; bh=PzZ3HdBOZkA8s2dX4VZP02ktX5zzjoB30ZeT/SSGF0A=; h=Date:From:Reply-To:To:CC:Subject; b=Ky/+vfmQDWjEMolSBWaOIZsD/dIucxGNxqqLS4mhVZgTnH8NGZ3Vf7p1xJmEEGRuC knrv+M4VaItxdqauQJDO5s15BHeFOiBzcrjUZEj1hmiix5NCDQooGmfLIUv4YibxY/ CxCgnbHuMfv3bt+w3tR7wZwTK4KyeM28O98saQME= Message-ID: <51F03331.6060403@delphij.net> Date: Wed, 24 Jul 2013 13:04:01 -0700 From: Xin Li Organization: The FreeBSD Project MIME-Version: 1.0 To: Matthew Ahrens Subject: Is 'zpool clear' supposed to work in case when pool I/O is suspended? X-Enigmail-Version: 1.5.1 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: zfs-devel@freebsd.org X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list Reply-To: d@delphij.net List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Jul 2013 20:04:03 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 Hi, Matthew, Looking at zfs_ioc_clear(), it seems like we wanted to use it as a way of waking up the pool from suspended I/O: %%% /* * Resume any suspended I/Os. */ if (zio_resume(spa) != 0) error = SET_ERROR(EIO); spa_close(spa, FTAG); %%% However, it's marked as POOL_CHECK_SUSPENDED: %%% zfs_ioctl_register_pool(ZFS_IOC_CLEAR, zfs_ioc_clear, zfs_secpolicy_config, B_TRUE, POOL_CHECK_SUSPENDED); %%% And the change was introduced in Illumos revision 4445fffb (libzfs_core). Is this part intentional? Before this it was POOL_CHECK_NONE. Cheers, - -- Xin LI https://www.delphij.net/ FreeBSD - The Power to Serve! Live free or die -----BEGIN PGP SIGNATURE----- iQEcBAEBCgAGBQJR8DMxAAoJEG80Jeu8UPuzpLcH/iZD7lkMyvEaKrzs37UgqPHH +gxvqozX9U620Bog8IQ+vKsV+8G6zWpFWfLb3OhNewNJwgojynaRShToPwr70sBR 72SuJ8LaUq4FafgOPQhJRyLHYnvF4986S93JIgOuHprzRFhWsLzBP8OSaKrfbAFp 89YE8EIMoc91L+c6gvGcfcDAdb25J4xeRlv4ZLeUf6pMmQ6IRnoZ15XrKhkT9J8b ZRNQO1g6Xy3Ub8XjxuANAVLH+lUq8APKBhoQO82vsfVgnm6U4U4o1MKDI2ooV/94 hf+bwPZ/p63z9b5cf0wrI1B6lZ06JBjp8WTf7DbufhNik6Kg+orlYvaMyVt9kMk= =bBCp -----END PGP SIGNATURE----- From owner-zfs-devel@FreeBSD.ORG Fri Jul 26 08:05:23 2013 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id BA4B255E; Fri, 26 Jul 2013 08:05:23 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 9BAFE2B3E; Fri, 26 Jul 2013 08:05:22 +0000 (UTC) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id LAA22527; Fri, 26 Jul 2013 11:05:20 +0300 (EEST) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1V2d1c-000HcJ-DG; Fri, 26 Jul 2013 11:05:20 +0300 Message-ID: <51F22D6E.1010507@FreeBSD.org> Date: Fri, 26 Jul 2013 11:03:58 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:17.0) Gecko/20130708 Thunderbird/17.0.7 MIME-Version: 1.0 To: "Justin T. Gibbs" , Steven Hartland Subject: Re: ZFS_DEBUG + enable ZFS ASSERTS with INVARIANTS any objections? References: <5AD89226AC69454DA8297F8C90A4E7E2@multiplay.co.uk> In-Reply-To: X-Enigmail-Version: 1.5.1 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: zfs-devel@FreeBSD.org X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Jul 2013 08:05:23 -0000 Justin, Steven, all, do you have any news on this topic? I think that having something in the tree would be better than having nothing at all (as it is now), unless that something would make future work very hard. BTW, I've thought just a little bit about this issue and here is how I would implement INVARIANTS support for code that has come and will come from OpenSolaris and its descendants. However, I am not sure how feasible and hard this approach would be in a technical sense. So, add something like the following at the end of auto-generated opt_global.h: #if defined(INVARIANTS) && defined(OPENSOLARIS_FILE) #ifndef DEBUG #define DEBUG #endif #ifndef ZFS_DEBUG #define ZFS_DEBUG #endif #endif Then all that remains to be done is to tag each file that uses OpenSolaris DEBUG semantics with OPENSOLARIS_FILE. That's easy, right? :-) Seriously though, there is already ZFS_C / ZFS_CFLAGS defined to be used with files that compiled into kernel. We could just add -DOPENSOLARIS_FILE there. For all the modules, we'd have to amend CFLAGS with -DOPENSOLARIS_FILE. Tedious, but doable. Alternatively, we could have a special header file that would contain: #if defined(INVARIANTS) #ifndef DEBUG #define DEBUG #endif #ifndef ZFS_DEBUG #define ZFS_DEBUG #endif #endif And then pass -include the/special/file.h in CFLAGS for all the relevant files/modules. What do you think? -- Andriy Gapon From owner-zfs-devel@FreeBSD.ORG Fri Jul 26 09:40:36 2013 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 149796C4 for ; Fri, 26 Jul 2013 09:40:36 +0000 (UTC) (envelope-from mavbsd@gmail.com) Received: from mail-bk0-x231.google.com (mail-bk0-x231.google.com [IPv6:2a00:1450:4008:c01::231]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 8B7A22EC4 for ; Fri, 26 Jul 2013 09:40:35 +0000 (UTC) Received: by mail-bk0-f49.google.com with SMTP id r7so667619bkg.22 for ; Fri, 26 Jul 2013 02:40:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:message-id:date:from:user-agent:mime-version:to:subject :content-type:content-transfer-encoding; bh=e5bNfii+iodxzVUY4asTdx8wDocSmLKLLE6FROTX2+w=; b=A+FkNjuVn7gp/jUMDd6/gf8H/r0UFZBqBixFmjHEXqoqe7OG7nFMjYpkreti/g5p9q 71gTAgUTkeLOLwqo0IE960JzSabQY/q7QgcQ8jDfMM7dD6HgMWwaWtlZJX0Fdvufq0/C f7PlhACwEUbf3s5yExL2rrx/AKv1qMjK5f1kxcUYe3oOajoXXC/FbEt0+Rm3AMRsVBk8 3TDP0YtnI4aYgmZZdUWmzfSLrecTHB83qj/FKvcwTvkbaBPjJYfxlJWUqsHYSl/j930p yEpgyBEL+adGvLFy4IJOLQ6LsTOE8CMuGlVEVVFMQkne7MtItSVqP9UU37+vciTPFmm7 wJYA== X-Received: by 10.205.3.5 with SMTP id nw5mr6991164bkb.137.1374831633699; Fri, 26 Jul 2013 02:40:33 -0700 (PDT) Received: from mavbook.mavhome.dp.ua (mavhome.mavhome.dp.ua. [213.227.240.37]) by mx.google.com with ESMTPSA id ct12sm12139444bkb.12.2013.07.26.02.40.31 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Fri, 26 Jul 2013 02:40:32 -0700 (PDT) Sender: Alexander Motin Message-ID: <51F2440E.5050107@FreeBSD.org> Date: Fri, 26 Jul 2013 12:40:30 +0300 From: Alexander Motin User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:17.0) Gecko/20130616 Thunderbird/17.0.6 MIME-Version: 1.0 To: zfs-devel@FreeBSD.org Subject: [RFC] Few patches about suspended pools Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Jul 2013 09:40:36 -0000 Hi. I would like to request for some comments about few patches I've made to improve handling of pools that lost devices, becoming incomplete and suspended: http://people.freebsd.org/~mav/zfs_patches/. cmd_on_suspend.patch -- allows three IOCTLs to be used on suspended pool. Looking on SVN log, this patch should restore the state of things we had up to some code drop from illumos, that seems also imported their issues. This change allows `zpool clear` to be used to recover suspended pool, `zpool status -v` print list of errors for the suspended pool, the third IOCTL looks like some name resolution not very clear to me, but that doesn't look like obviously suspend-related. remove_last.patch -- makes SPA_ASYNC_REMOVE async events in ZFS to be processed even after pool was suspended. Without that, async events are handled only after successful txg commit, that is obviously impossible without disks. In fact patch makes async thread to be called immediately on SPA_ASYNC_REMOVE async events. I haven't found any reason why async events (especially this) should be handled only once per commit. online_on_suspend.patch -- is somewhat questionable and possibly incomplete patch. The idea was to make `zpool online` command work for suspended pools, as some sources recommend. The problem is that this command tries to change pool content (like updating history) or wait for completion. That required some hacks to make it not stuck with pool still suspended. -- Alexander Motin From owner-zfs-devel@FreeBSD.ORG Fri Jul 26 14:08:40 2013 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 2E0001C5; Fri, 26 Jul 2013 14:08:40 +0000 (UTC) (envelope-from prvs=1919536ae1=killing@multiplay.co.uk) Received: from mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) (using TLSv1 with cipher RC4-MD5 (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 76C7E2D2D; Fri, 26 Jul 2013 14:08:38 +0000 (UTC) Received: from r2d2 ([82.69.141.170]) by mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) (MDaemon PRO v10.0.4) with ESMTP id md50005166700.msg; Fri, 26 Jul 2013 15:08:30 +0100 X-Spam-Processed: mail1.multiplay.co.uk, Fri, 26 Jul 2013 15:08:30 +0100 (not processed: message from valid local sender) X-MDDKIM-Result: neutral (mail1.multiplay.co.uk) X-MDRemoteIP: 82.69.141.170 X-Return-Path: prvs=1919536ae1=killing@multiplay.co.uk X-Envelope-From: killing@multiplay.co.uk Message-ID: <1BF5D3CF73474D9984E97A30B41A10FC@multiplay.co.uk> From: "Steven Hartland" To: "Andriy Gapon" , "Justin T. Gibbs" References: <5AD89226AC69454DA8297F8C90A4E7E2@multiplay.co.uk> <51F22D6E.1010507@FreeBSD.org> Subject: Re: ZFS_DEBUG + enable ZFS ASSERTS with INVARIANTS any objections? Date: Fri, 26 Jul 2013 15:09:03 +0100 MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="----=_NextPart_000_0B78_01CE8A12.10FCCB10" X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.5931 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.6157 Cc: zfs-devel@FreeBSD.org X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Jul 2013 14:08:40 -0000 This is a multi-part message in MIME format. ------=_NextPart_000_0B78_01CE8A12.10FCCB10 Content-Type: text/plain; format=flowed; charset="iso-8859-1"; reply-type=original Content-Transfer-Encoding: 7bit I tried an alternative version (attached) based on Justin's feedback, which but enables DEBUG in all opensolaris modules when INVARIANTS is in effect. Unfortunately it results in all the LOR being printed so I'm not a fan TBH, I prefer the option to enable ASSERTS ony by changing debug.h Regards Steve ----- Original Message ----- From: "Andriy Gapon" > Justin, Steven, all, > > do you have any news on this topic? > I think that having something in the tree would be better than having nothing at > all (as it is now), unless that something would make future work very hard. > > BTW, I've thought just a little bit about this issue and here is how I would > implement INVARIANTS support for code that has come and will come from > OpenSolaris and its descendants. However, I am not sure how feasible and hard > this approach would be in a technical sense. > > So, add something like the following at the end of auto-generated opt_global.h: > #if defined(INVARIANTS) && defined(OPENSOLARIS_FILE) > #ifndef DEBUG > #define DEBUG > #endif > #ifndef ZFS_DEBUG > #define ZFS_DEBUG > #endif > #endif > > Then all that remains to be done is to tag each file that uses OpenSolaris DEBUG > semantics with OPENSOLARIS_FILE. That's easy, right? :-) > Seriously though, there is already ZFS_C / ZFS_CFLAGS defined to be used with > files that compiled into kernel. We could just add -DOPENSOLARIS_FILE there. > For all the modules, we'd have to amend CFLAGS with -DOPENSOLARIS_FILE. > Tedious, but doable. > > Alternatively, we could have a special header file that would contain: > #if defined(INVARIANTS) > #ifndef DEBUG > #define DEBUG > #endif > #ifndef ZFS_DEBUG > #define ZFS_DEBUG > #endif > #endif > > And then pass -include the/special/file.h in CFLAGS for all the relevant > files/modules. > > What do you think? > > -- > Andriy Gapon > ================================================ This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmaster@multiplay.co.uk. ------=_NextPart_000_0B78_01CE8A12.10FCCB10 Content-Type: application/octet-stream; name="zfs-debug2.patch" Content-Transfer-Encoding: quoted-printable Content-Disposition: attachment; filename="zfs-debug2.patch" Index: sys/modules/dtrace/Makefile=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/modules/dtrace/Makefile (revision 253401)=0A= +++ sys/modules/dtrace/Makefile (working copy)=0A= @@ -26,4 +26,9 @@=0A= SUBDIR+=3D systrace_freebsd32=0A= .endif=0A= =0A= +_INVARIANTS_ENABLED!=3D grep INVARIANTS ${KERNBUILDDIR}/opt_global.h || = true=0A= +.if !empty(_INVARIANTS_ENABLED)=0A= +CFLAGS +=3D -DDEBUG=0A= +.endif=0A= +=0A= .include =0A= Index: sys/modules/opensolaris/Makefile=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/modules/opensolaris/Makefile (revision 253401)=0A= +++ sys/modules/opensolaris/Makefile (working copy)=0A= @@ -24,6 +24,11 @@=0A= -I${.CURDIR}/../../cddl/contrib/opensolaris/uts/common \=0A= -I${.CURDIR}/../..=0A= =0A= +_INVARIANTS_ENABLED!=3D grep INVARIANTS ${KERNBUILDDIR}/opt_global.h || = true=0A= +.if !empty(_INVARIANTS_ENABLED)=0A= +CFLAGS +=3D -DDEBUG=0A= +.endif=0A= +=0A= IGNORE_PRAGMA=3D 1=0A= =0A= .include =0A= Index: sys/modules/zfs/Makefile=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/modules/zfs/Makefile (revision 253401)=0A= +++ sys/modules/zfs/Makefile (working copy)=0A= @@ -91,8 +91,10 @@=0A= CFLAGS+=3D-mminimal-toc=0A= .endif=0A= =0A= -#CFLAGS+=3D-DDEBUG=3D1=0A= -#DEBUG_FLAGS=3D-g=0A= +_INVARIANTS_ENABLED!=3D grep INVARIANTS ${KERNBUILDDIR}/opt_global.h || = true=0A= +.if !empty(_INVARIANTS_ENABLED)=0A= +CFLAGS +=3D -DDEBUG=0A= +.endif=0A= =0A= .include =0A= ------=_NextPart_000_0B78_01CE8A12.10FCCB10-- From owner-zfs-devel@FreeBSD.ORG Fri Jul 26 14:13:14 2013 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 4D5AF237; Fri, 26 Jul 2013 14:13:14 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 62EC22D5E; Fri, 26 Jul 2013 14:13:12 +0000 (UTC) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id RAA26848; Fri, 26 Jul 2013 17:13:10 +0300 (EEST) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1V2ila-000I7W-EO; Fri, 26 Jul 2013 17:13:10 +0300 Message-ID: <51F283BE.3030006@FreeBSD.org> Date: Fri, 26 Jul 2013 17:12:14 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:17.0) Gecko/20130708 Thunderbird/17.0.7 MIME-Version: 1.0 To: Steven Hartland Subject: Re: ZFS_DEBUG + enable ZFS ASSERTS with INVARIANTS any objections? References: <5AD89226AC69454DA8297F8C90A4E7E2@multiplay.co.uk> <51F22D6E.1010507@FreeBSD.org> <1BF5D3CF73474D9984E97A30B41A10FC@multiplay.co.uk> In-Reply-To: <1BF5D3CF73474D9984E97A30B41A10FC@multiplay.co.uk> X-Enigmail-Version: 1.5.1 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: zfs-devel@FreeBSD.org X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Jul 2013 14:13:14 -0000 on 26/07/2013 17:09 Steven Hartland said the following: > I tried an alternative version (attached) based on Justin's feedback, > which but enables DEBUG in all opensolaris modules when INVARIANTS is in > effect. > > Unfortunately it results in all the LOR being printed so I'm not a fan > TBH, I prefer the option to enable ASSERTS ony by changing debug.h Hmm, I am not sure what exactly is the problem... LOR detection and reporting is controlled by WITNESS, as I understand, and WITNESS is orthogonal to INVARIANTS. -- Andriy Gapon From owner-zfs-devel@FreeBSD.ORG Fri Jul 26 14:46:06 2013 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 3B21A616; Fri, 26 Jul 2013 14:46:06 +0000 (UTC) (envelope-from prvs=1919536ae1=killing@multiplay.co.uk) Received: from mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) (using TLSv1 with cipher RC4-MD5 (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 865E72F46; Fri, 26 Jul 2013 14:46:05 +0000 (UTC) Received: from r2d2 ([82.69.141.170]) by mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) (MDaemon PRO v10.0.4) with ESMTP id md50005166979.msg; Fri, 26 Jul 2013 15:46:04 +0100 X-Spam-Processed: mail1.multiplay.co.uk, Fri, 26 Jul 2013 15:46:04 +0100 (not processed: message from valid local sender) X-MDDKIM-Result: neutral (mail1.multiplay.co.uk) X-MDRemoteIP: 82.69.141.170 X-Return-Path: prvs=1919536ae1=killing@multiplay.co.uk X-Envelope-From: killing@multiplay.co.uk Message-ID: <03354BD289064923B1C39EF4CD4F822E@multiplay.co.uk> From: "Steven Hartland" To: "Andriy Gapon" References: <5AD89226AC69454DA8297F8C90A4E7E2@multiplay.co.uk> <51F22D6E.1010507@FreeBSD.org> <1BF5D3CF73474D9984E97A30B41A10FC@multiplay.co.uk> <51F283BE.3030006@FreeBSD.org> Subject: Re: ZFS_DEBUG + enable ZFS ASSERTS with INVARIANTS any objections? Date: Fri, 26 Jul 2013 15:46:34 +0100 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="iso-8859-1"; reply-type=original Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.5931 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.6157 Cc: zfs-devel@FreeBSD.org X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Jul 2013 14:46:06 -0000 ----- Original Message ----- From: "Andriy Gapon" > on 26/07/2013 17:09 Steven Hartland said the following: >> I tried an alternative version (attached) based on Justin's feedback, >> which but enables DEBUG in all opensolaris modules when INVARIANTS is in >> effect. >> >> Unfortunately it results in all the LOR being printed so I'm not a fan >> TBH, I prefer the option to enable ASSERTS ony by changing debug.h > > Hmm, I am not sure what exactly is the problem... > LOR detection and reporting is controlled by WITNESS, as I understand, and > WITNESS is orthogonal to INVARIANTS. I believe theirs a connection between DEBUG and LOR reporting, specificaly in rwlock.h when not under DEBUG RW_FLAGS includes SX_NOWITNESS. Regards Steve ================================================ This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmaster@multiplay.co.uk. From owner-zfs-devel@FreeBSD.ORG Fri Jul 26 15:01:40 2013 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 50C30AF8; Fri, 26 Jul 2013 15:01:40 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 621322013; Fri, 26 Jul 2013 15:01:38 +0000 (UTC) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id SAA27204; Fri, 26 Jul 2013 18:01:37 +0300 (EEST) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1V2jWS-000IBv-Qh; Fri, 26 Jul 2013 18:01:36 +0300 Message-ID: <51F28F00.9050705@FreeBSD.org> Date: Fri, 26 Jul 2013 18:00:16 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:17.0) Gecko/20130708 Thunderbird/17.0.7 MIME-Version: 1.0 To: Steven Hartland Subject: Re: ZFS_DEBUG + enable ZFS ASSERTS with INVARIANTS any objections? References: <5AD89226AC69454DA8297F8C90A4E7E2@multiplay.co.uk> <51F22D6E.1010507@FreeBSD.org> <1BF5D3CF73474D9984E97A30B41A10FC@multiplay.co.uk> <51F283BE.3030006@FreeBSD.org> <03354BD289064923B1C39EF4CD4F822E@multiplay.co.uk> In-Reply-To: <03354BD289064923B1C39EF4CD4F822E@multiplay.co.uk> X-Enigmail-Version: 1.5.1 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: zfs-devel@FreeBSD.org X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Jul 2013 15:01:40 -0000 on 26/07/2013 17:46 Steven Hartland said the following: > ----- Original Message ----- From: "Andriy Gapon" > > >> on 26/07/2013 17:09 Steven Hartland said the following: >>> I tried an alternative version (attached) based on Justin's feedback, >>> which but enables DEBUG in all opensolaris modules when INVARIANTS is in >>> effect. >>> >>> Unfortunately it results in all the LOR being printed so I'm not a fan >>> TBH, I prefer the option to enable ASSERTS ony by changing debug.h >> >> Hmm, I am not sure what exactly is the problem... >> LOR detection and reporting is controlled by WITNESS, as I understand, and >> WITNESS is orthogonal to INVARIANTS. > > I believe theirs a connection between DEBUG and LOR reporting, specificaly > in rwlock.h when not under DEBUG RW_FLAGS includes SX_NOWITNESS. ( I guessed that you mean sys/cddl/compat/opensolaris/sys/rwlock.h of all rwlock.h files under sys/) Yes. But I mean that if you don't have WITNESS in the first place, then you don't get any LOR reports. And if you have WITNESS you should expect that ZFS is covered too. On the other hand, I also think that DEBUG should not have grown any new meanings during porting. E.g. in your example it should have been something like ZFS_WITNESS or OPENSOLARIS_WITNESS, but not DEBUG, because DEBUG already had specific meaning for osol code. I think that this should be fixed as well. -- Andriy Gapon From owner-zfs-devel@FreeBSD.ORG Fri Jul 26 15:41:48 2013 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id D37F158D; Fri, 26 Jul 2013 15:41:48 +0000 (UTC) (envelope-from prvs=1919536ae1=killing@multiplay.co.uk) Received: from mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) (using TLSv1 with cipher RC4-MD5 (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 28A1422DC; Fri, 26 Jul 2013 15:41:47 +0000 (UTC) Received: from r2d2 ([82.69.141.170]) by mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) (MDaemon PRO v10.0.4) with ESMTP id md50005167656.msg; Fri, 26 Jul 2013 16:41:44 +0100 X-Spam-Processed: mail1.multiplay.co.uk, Fri, 26 Jul 2013 16:41:44 +0100 (not processed: message from valid local sender) X-MDDKIM-Result: neutral (mail1.multiplay.co.uk) X-MDRemoteIP: 82.69.141.170 X-Return-Path: prvs=1919536ae1=killing@multiplay.co.uk X-Envelope-From: killing@multiplay.co.uk Message-ID: <0C3C764BA0B344678A0D9EBB36697478@multiplay.co.uk> From: "Steven Hartland" To: "Andriy Gapon" References: <5AD89226AC69454DA8297F8C90A4E7E2@multiplay.co.uk> <51F22D6E.1010507@FreeBSD.org> <1BF5D3CF73474D9984E97A30B41A10FC@multiplay.co.uk> <51F283BE.3030006@FreeBSD.org> <03354BD289064923B1C39EF4CD4F822E@multiplay.co.uk> <51F28F00.9050705@FreeBSD.org> Subject: Re: ZFS_DEBUG + enable ZFS ASSERTS with INVARIANTS any objections? Date: Fri, 26 Jul 2013 16:42:14 +0100 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="iso-8859-1"; reply-type=original Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.5931 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.6157 Cc: zfs-devel@FreeBSD.org X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Jul 2013 15:41:48 -0000 ----- Original Message ----- From: "Andriy Gapon" >>> on 26/07/2013 17:09 Steven Hartland said the following: >>>> I tried an alternative version (attached) based on Justin's feedback, >>>> which but enables DEBUG in all opensolaris modules when INVARIANTS is in >>>> effect. >>>> >>>> Unfortunately it results in all the LOR being printed so I'm not a fan >>>> TBH, I prefer the option to enable ASSERTS ony by changing debug.h >>> >>> Hmm, I am not sure what exactly is the problem... >>> LOR detection and reporting is controlled by WITNESS, as I understand, and >>> WITNESS is orthogonal to INVARIANTS. >> >> I believe theirs a connection between DEBUG and LOR reporting, specificaly >> in rwlock.h when not under DEBUG RW_FLAGS includes SX_NOWITNESS. > > ( I guessed that you mean sys/cddl/compat/opensolaris/sys/rwlock.h of all > rwlock.h files under sys/) > > Yes. But I mean that if you don't have WITNESS in the first place, then you > don't get any LOR reports. And if you have WITNESS you should expect that ZFS > is covered too. > > On the other hand, I also think that DEBUG should not have grown any new > meanings during porting. E.g. in your example it should have been something > like ZFS_WITNESS or OPENSOLARIS_WITNESS, but not DEBUG, because DEBUG already > had specific meaning for osol code. I think that this should be fixed as well. That may be the best solution. The problem with LOR reporting in ZFS, from what pjd said, is that the LOR's are false positives and there's a lot of them hence why they are only reported under DEBUG. Regards Steve ================================================ This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmaster@multiplay.co.uk. From owner-zfs-devel@FreeBSD.ORG Fri Jul 26 15:56:17 2013 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id CC69E9F6; Fri, 26 Jul 2013 15:56:17 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id DE55C23A1; Fri, 26 Jul 2013 15:56:16 +0000 (UTC) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id SAA27655; Fri, 26 Jul 2013 18:56:08 +0300 (EEST) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1V2kNE-000IHU-30; Fri, 26 Jul 2013 18:56:08 +0300 Message-ID: <51F29BDF.9020404@FreeBSD.org> Date: Fri, 26 Jul 2013 18:55:11 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:17.0) Gecko/20130708 Thunderbird/17.0.7 MIME-Version: 1.0 To: Steven Hartland Subject: Re: ZFS_DEBUG + enable ZFS ASSERTS with INVARIANTS any objections? References: <5AD89226AC69454DA8297F8C90A4E7E2@multiplay.co.uk> <51F22D6E.1010507@FreeBSD.org> <1BF5D3CF73474D9984E97A30B41A10FC@multiplay.co.uk> <51F283BE.3030006@FreeBSD.org> <03354BD289064923B1C39EF4CD4F822E@multiplay.co.uk> <51F28F00.9050705@FreeBSD.org> <0C3C764BA0B344678A0D9EBB36697478@multiplay.co.uk> In-Reply-To: <0C3C764BA0B344678A0D9EBB36697478@multiplay.co.uk> X-Enigmail-Version: 1.5.1 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: zfs-devel@FreeBSD.org X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Jul 2013 15:56:17 -0000 on 26/07/2013 18:42 Steven Hartland said the following: > That may be the best solution. The problem with LOR reporting in ZFS, from what > pjd said, is that the LOR's are false positives and there's a lot of them hence > why they are only reported under DEBUG. I agree that ZFS reports quite a large number of LORs. FreeBSD without ZFS reports a number of false LORs as well. So it's the difference in LORs (not their absolute count) that matters. For this reason it makes sense to report ZFS LORs as well - just to see if a new one appears after a change. Personally for me, couple hundreds of extra lines during boot do not pose any problem or annoyance. But that's me :-) -- Andriy Gapon From owner-zfs-devel@FreeBSD.ORG Mon Jul 29 11:08:18 2013 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 6E12263F for ; Mon, 29 Jul 2013 11:08:18 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 3FCE42EE0 for ; Mon, 29 Jul 2013 11:08:18 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.7/8.14.7) with ESMTP id r6TB8Iw7063654 for ; Mon, 29 Jul 2013 11:08:18 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.7/8.14.7/Submit) id r6TB8H6o063652 for zfs-devel@FreeBSD.org; Mon, 29 Jul 2013 11:08:17 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 29 Jul 2013 11:08:17 GMT Message-Id: <201307291108.r6TB8H6o063652@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: zfs-devel@FreeBSD.org Subject: Current problem reports assigned to zfs-devel@FreeBSD.org X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 29 Jul 2013 11:08:18 -0000 Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- s kern/178467 zfs-devel [zfs] [request] Optimized Checksum Code for ZFS o kern/178387 zfs-devel [zfs] [patch] sparse files performance improvements 2 problems total. From owner-zfs-devel@FreeBSD.ORG Mon Jul 29 14:25:50 2013 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 29707A96; Mon, 29 Jul 2013 14:25:50 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id CD6D329B8; Mon, 29 Jul 2013 14:25:48 +0000 (UTC) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id RAA06580; Mon, 29 Jul 2013 17:25:40 +0300 (EEST) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1V3oOJ-00072l-ND; Mon, 29 Jul 2013 17:25:39 +0300 Message-ID: <51F67B2A.3040302@FreeBSD.org> Date: Mon, 29 Jul 2013 17:24:42 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:17.0) Gecko/20130708 Thunderbird/17.0.7 MIME-Version: 1.0 To: freebsd-arch@FreeBSD.org Subject: translate INVARIANTS to DEBUG for code from OpenSolaris X-Enigmail-Version: 1.5.1 Content-Type: text/plain; charset=X-VIET-VPS Content-Transfer-Encoding: 7bit X-Mailman-Approved-At: Mon, 29 Jul 2013 17:40:30 +0000 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 29 Jul 2013 14:25:50 -0000 [zfs-devel@, fs@, dtrace@ are Bcc-ed] In OpenSolaris and its descendants DEBUG is used in a fashion similar to our INVARIANTS. For example, ASSERT macros are enabled by it. In our kernel code DEBUG has a different meaning and enables far too verbose or far too obscure code and, as such, it is very rarely enabled. The idea of a change that I would like to propose is to translate INVARIANTS kernel option into DEBUG for the files that originated from OpenSolaris (and hopefully only for them). The change: opensolaris code: translate INVARIANTS to DEBUG and ZFS_DEBUG do this by forcing inclusion of sys/cddl/compat/opensolaris/sys/debug_compat.h via -include option into all source files from OpenSolaris. Note that this -include option must always be after -include opt_global.h. Additionally, remove forced definition of DEBUG for some modules and fix their build without DEBUG. Also, meaning of DEBUG was overloaded to enable WITNESS support for some OpenSolaris (primarily ZFS) locks. Now this overloading is removed and that use of DEBUG is replaced with a new option OPENSOLARIS_WITNESS. http://people.freebsd.org/~avg/osol-invariants-debug.diff I would like to ask for your feedback on the soundness of the whole idea. Also on the name, location and style of inclusion for sys/cddl/compat/opensolaris/sys/debug_compat.h. And on any other details of the proposed change. Testing is also welcome, of course. Thank you very much. -- Andriy Gapon From owner-zfs-devel@FreeBSD.ORG Tue Jul 30 07:52:06 2013 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 870237D7 for ; Tue, 30 Jul 2013 07:52:06 +0000 (UTC) (envelope-from mavbsd@gmail.com) Received: from mail-la0-x229.google.com (mail-la0-x229.google.com [IPv6:2a00:1450:4010:c03::229]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 0B2582A4D for ; Tue, 30 Jul 2013 07:52:05 +0000 (UTC) Received: by mail-la0-f41.google.com with SMTP id ec20so1179302lab.0 for ; Tue, 30 Jul 2013 00:52:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:message-id:date:from:user-agent:mime-version:to:subject :references:in-reply-to:content-type:content-transfer-encoding; bh=yPc6cAKIYxEL/CU7fHrZnpt9M2EHvAniwsynZDejcbw=; b=wrNXW06u2zsiStSCwuy/NTyo0Nkbj0okuxKYHRAmpa8pJyG1DEOprTOsbZq8xNx7mE WJA4YMzBBYzR4Jc2TeTQdhvjgcMLLJrH1dyQTiOP3584BWZ++ASu41wN08dgOyGqX+pr h3QlEHgLBvF15Roy7yLgIcg7Dt7SexWyycP/WNF3jbkfuLt1AAsfb1zOShKwTyucepAS isQ3Jpy4/0Lc1UEsqf26xxOGzWCQuTE3/ksRbcmulUsy3NDT4KrreWi5D9ITGq5Hn2LT JixZDSg+JzsgxcRMY3/wHz2l6n88+zvU1uQNp7+P36RhOSzWKJJqIWUczsGl5HYxhWux WkPw== X-Received: by 10.152.5.227 with SMTP id v3mr28552975lav.31.1375170723853; Tue, 30 Jul 2013 00:52:03 -0700 (PDT) Received: from mavbook.mavhome.dp.ua (mavhome.mavhome.dp.ua. [213.227.240.37]) by mx.google.com with ESMTPSA id w9sm7520101lbd.9.2013.07.30.00.52.01 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 30 Jul 2013 00:52:02 -0700 (PDT) Sender: Alexander Motin Message-ID: <51F7709F.6030800@FreeBSD.org> Date: Tue, 30 Jul 2013 10:51:59 +0300 From: Alexander Motin User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:17.0) Gecko/20130616 Thunderbird/17.0.6 MIME-Version: 1.0 To: zfs-devel@FreeBSD.org Subject: Re: [RFC] Few patches about suspended pools References: <51F2440E.5050107@FreeBSD.org> In-Reply-To: <51F2440E.5050107@FreeBSD.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 Jul 2013 07:52:06 -0000 Hoping somebody reads this, here is some addition. On 26.07.2013 12:40, Alexander Motin wrote: > I would like to request for some comments about few patches I've made to > improve handling of pools that lost devices, becoming incomplete and > suspended: http://people.freebsd.org/~mav/zfs_patches/. > > cmd_on_suspend.patch -- allows three IOCTLs to be used on suspended > pool. Looking on SVN log, this patch should restore the state of things > we had up to some code drop from illumos, that seems also imported their > issues. This change allows `zpool clear` to be used to recover suspended > pool, `zpool status -v` print list of errors for the suspended pool, the > third IOCTL looks like some name resolution not very clear to me, but > that doesn't look like obviously suspend-related. > > remove_last.patch -- makes SPA_ASYNC_REMOVE async events in ZFS to be > processed even after pool was suspended. Without that, async events are > handled only after successful txg commit, that is obviously impossible > without disks. In fact patch makes async thread to be called immediately > on SPA_ASYNC_REMOVE async events. I haven't found any reason why async > events (especially this) should be handled only once per commit. I've found that calling async event in arbitrary time increases always existing chances to get stuck with attempt to get configuration write lock while some read lock holder blocked infinitely waiting for I/O, blocking zpool completely. So I've rewritten this patch to handle SPA_ASYNC_REMOVE in separate thread since it is more critical for the whole system behavior and same time does not require configuration lock: remove_last2.patch > online_on_suspend.patch -- is somewhat questionable and possibly > incomplete patch. The idea was to make `zpool online` command work for > suspended pools, as some sources recommend. The problem is that this > command tries to change pool content (like updating history) or wait for > completion. That required some hacks to make it not stuck with pool > still suspended. Also I've found a way to hang ZFS by device detach up to the state of freezing `zpool status`. Issue was introduced with pool features implementation. `zpool` tool reads pool configuration on every pool opening. Previously there was no problem since the configuration was recreated by kernel from data that seems always present in memory. But newly introduced feature use counters may not. They are stored in ZAP and in normal case just live in ARC. But if system has strong ARC pressure, that information may be evicted, and then. if we are loosing devices, we are stuck. I am not sure what would be a proper fix for this situation. I've created workaround (no_features.patch) that blocks features reporting to user-level if pool is suspended. That is a bit overkill, but helps with one predictable flaw: `zpool upgrade` always wish to upgrade suspended pools, but fortunately it can't due to the same suspension. -- Alexander Motin From owner-zfs-devel@FreeBSD.ORG Wed Jul 31 00:15:34 2013 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id B2819FC4; Wed, 31 Jul 2013 00:15:34 +0000 (UTC) (envelope-from prvs=19244eee1d=killing@multiplay.co.uk) Received: from mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) (using TLSv1 with cipher RC4-MD5 (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 2E3892121; Wed, 31 Jul 2013 00:15:33 +0000 (UTC) Received: from r2d2 ([82.69.141.170]) by mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) (MDaemon PRO v10.0.4) with ESMTP id md50005230608.msg; Wed, 31 Jul 2013 01:15:25 +0100 X-Spam-Processed: mail1.multiplay.co.uk, Wed, 31 Jul 2013 01:15:25 +0100 (not processed: message from valid local sender) X-MDDKIM-Result: neutral (mail1.multiplay.co.uk) X-MDRemoteIP: 82.69.141.170 X-Return-Path: prvs=19244eee1d=killing@multiplay.co.uk X-Envelope-From: killing@multiplay.co.uk Message-ID: <95286A3C69D44769973FB0E072A84AB0@multiplay.co.uk> From: "Steven Hartland" To: , "Xin LI" , References: <201307180022.r6I0MgeS094364@svn.freebsd.org> <251C370CF0FF4EE686D571A1235D4C90@multiplay.co.uk> <51E73BCA.3030404@delphij.net> Subject: Invalid ashift increase allowed by r253441 (was: Re: svn commit: r253441 ...) Date: Wed, 31 Jul 2013 01:16:03 +0100 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="iso-8859-1"; reply-type=response Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.5931 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.6157 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 31 Jul 2013 00:15:34 -0000 ----- Original Message ----- From: "Steven Hartland" > ----- Original Message ----- > From: "Xin Li" >> -----BEGIN PGP SIGNED MESSAGE----- >> Hash: SHA512 >> >> On 07/17/13 17:34, Steven Hartland wrote: >>> This is an interesting change, could this not cause serious issues >>> when we try to read / write to a disk with an incompatible block >>> size? >> >> No, it's safe to use larger ashift to access pool formatted with >> smaller ashift, it's not optimal but better than marking the pool >> FALUTERED, and yes, the operator still have to recreate the pool if >> performance is a concern. > > Maybe I'm missing something but if the device is truely using a larger > sectorsize than that which the zpool was created with then surely > attempts to address it will fail in g_io_check when bio_offset is > not a factor of sectorsize? > > For example on a native 4k sectorsize disk its not possible to directly > access the bytes at offset 512 instead you would need to read offset 0 > and scan in 512 bytes to the returned data block. I didn't think > geom / zfs dealt with this case? > > The only case I can see this happening is if an old 512b sectorsize > disk was dd'ed to a new 4k sectorsize one but thats effectively what > this change is allowing. Just wanted to chase up on this one to see if I'm barking up the wrong tree or if this could cause serious breakage like I think it could. Looking at the illumos issue and commit for this: https://www.illumos.org/issues/2671 https://github.com/illumos/illumos-gate/commit/2384d9f8fcca0a7ef8b3ae674d94df82832c0fce I suspect this came about because of changes to how ashift was being reported / faked, very simular to the patches many of us have for FreeBSD. If I'm correct this strengthens my view that this change is incorrect, as what was trying to be done was support for faked sector size on devices such as SSD's and HD's which report and importantly support 512b sectors even if they are reported to ZFS as 4k. This is very different from supporting a device which ONLY supports a large sector size and hence smaller accesses would fail which this change allows. Thoughts? Regards Steve ================================================ This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmaster@multiplay.co.uk. From owner-zfs-devel@FreeBSD.ORG Wed Jul 31 11:59:15 2013 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id C7E6858A; Wed, 31 Jul 2013 11:59:15 +0000 (UTC) (envelope-from mavbsd@gmail.com) Received: from mail-la0-x22f.google.com (mail-la0-x22f.google.com [IPv6:2a00:1450:4010:c03::22f]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id E3C9E2D7D; Wed, 31 Jul 2013 11:59:14 +0000 (UTC) Received: by mail-la0-f47.google.com with SMTP id eo20so427795lab.34 for ; Wed, 31 Jul 2013 04:59:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:message-id:date:from:user-agent:mime-version:to:subject :content-type:content-transfer-encoding; bh=ZowcteYKPR//jV7pptNQ66cF/xHL1tO24PBAFHXbva4=; b=BqpIpRGs3kWBw4qWvrcgYOrW4F2WvqosdKWFslcVJfoxt18Qttan6ap+uNv09ra729 fHjF3655nnVbl6dQF8+QENIVDaITRKfxMA9A65f6byY/lCIufT1DX70CtFLdKE6D6r8F SAe2AtHOFRhbyULj+M+JFhNlJ/rBMSl6ERFDfji8ytAuYpkgblXloY0S0QaXv0K17HTN c1xuRVYs8XBJW545y3t/q0Rrb9ZPQY7FS3jhaLOrbBi/pkJenXS9ExhArSrBk1MGH/0Q +y8mH7QN5H/RVU553NGQZTCRXf5HL+Ou9+57Eg7li2c9FcqulrEXIFoxZg9pMnyHxB1s w5vA== X-Received: by 10.112.20.200 with SMTP id p8mr2179977lbe.87.1375271952728; Wed, 31 Jul 2013 04:59:12 -0700 (PDT) Received: from mavbook.mavhome.dp.ua (mavhome.mavhome.dp.ua. [213.227.240.37]) by mx.google.com with ESMTPSA id rx1sm1190212lbb.0.2013.07.31.04.59.10 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Wed, 31 Jul 2013 04:59:11 -0700 (PDT) Sender: Alexander Motin Message-ID: <51F8FC0C.2030703@FreeBSD.org> Date: Wed, 31 Jul 2013 14:59:08 +0300 From: Alexander Motin User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:17.0) Gecko/20130616 Thunderbird/17.0.6 MIME-Version: 1.0 To: Martin Matuska , zfs-devel@FreeBSD.org, Steven Hartland Subject: HEAD r252840 (illumos bug 3836) and our TRIM are incompatible, causing deadlocks Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 31 Jul 2013 11:59:15 -0000 Hi. With some experiments I've got to believe that HEAD r252840 (illumos bug 3836) and our TRIM implementation are mutually incompatible. I have found 100% repeatable scenario how to cause deadlock when these changes are applied together. All that needed is to create significant write load (I've used `iozone -t 16 -s 8G` on 8-core system with 2GB RAM and 2 striped SSDs) and run `zpool clear poolname`. After that system is effectively dead: all I/O are stuck and even zpool commands are no longer functioning. I think triggering event is not necessary should be `zpool clear`, any event/action that takes SCL_ZIO lock for writing should do the same. r252840 (illumos bug 3836) is based on assumption that zio_free_sync() has no lock dependencies and should complete immediately. Unfortunately, with our TRIM implementation that is not true due to ZIO_STAGE_VDEV_IO_START added to the ZIO_FREE_PIPELINE, which, while not really accessing devices, still acquires SCL_ZIO lock for read. As result, we are getting such deadlock: `zpool clear` asks for SC_ZIO for writing and waits for all read locks to be dropped; SC_ZIO is held for read by regular I/Os that were running and are completed now, but to drop the lock they require free zio_write_intr thread; unfortunately all zio_write_intr treads under high load are stuck inside modified zio_free(), that is now trying to directly execute zio_free_sync(), that with our TRIM implementation tries to obtain SCL_ZIO for read; BANG! DEADLOCK! Reverting r252840 fixes the situation for me. If somebody have ideas how to fix the situation without reverting either changes -- welcome. -- Alexander Motin From owner-zfs-devel@FreeBSD.ORG Thu Aug 1 05:10:58 2013 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 16FC24B6; Thu, 1 Aug 2013 05:10:58 +0000 (UTC) (envelope-from delphij@delphij.net) Received: from anubis.delphij.net (anubis.delphij.net [IPv6:2001:470:1:117::25]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id DBA962823; Thu, 1 Aug 2013 05:10:57 +0000 (UTC) Received: from delphij-macbook.local (c-67-188-85-47.hsd1.ca.comcast.net [67.188.85.47]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by anubis.delphij.net (Postfix) with ESMTPSA id EFF8416753; Wed, 31 Jul 2013 22:10:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=delphij.net; s=anubis; t=1375333857; bh=qzhQ3I9qlhqCPyjDv7axc5l+MFZdaaTUxCzDYYzNZdE=; h=Date:From:Reply-To:To:CC:Subject:References:In-Reply-To; b=YEhA221+PSas61CdqWMCcHrkZyDy8tKLWsCnhoDNHs0zmLgEJbCWW4MMxKviDA3Mx RbW5tj+rCyYwfyv65OiSCkyqIzUOs4gK1YS2uBDlgF7OkFp/kfFR7fQ414VK/OpmJL r7X7HECnfKX+HUXR9LT2HevpOag2SIz7SJs21ErU= Message-ID: <51F9EDDF.4080100@delphij.net> Date: Wed, 31 Jul 2013 22:10:55 -0700 From: Xin Li Organization: The FreeBSD Project MIME-Version: 1.0 To: Steven Hartland Subject: Re: Invalid ashift increase allowed by r253441 References: <201307180022.r6I0MgeS094364@svn.freebsd.org> <251C370CF0FF4EE686D571A1235D4C90@multiplay.co.uk> <51E73BCA.3030404@delphij.net> <95286A3C69D44769973FB0E072A84AB0@multiplay.co.uk> In-Reply-To: <95286A3C69D44769973FB0E072A84AB0@multiplay.co.uk> X-Enigmail-Version: 1.5.1 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: Xin LI , zfs-devel@FreeBSD.org, d@delphij.net X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list Reply-To: d@delphij.net List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 01 Aug 2013 05:10:58 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 Hi, Steven, On 7/30/13 5:16 PM, Steven Hartland wrote: [...] >> For example on a native 4k sectorsize disk its not possible to >> directly access the bytes at offset 512 instead you would need to >> read offset 0 and scan in 512 bytes to the returned data block. I >> didn't think geom / zfs dealt with this case? No, GEOM does not allow this. >> The only case I can see this happening is if an old 512b >> sectorsize disk was dd'ed to a new 4k sectorsize one but thats >> effectively what this change is allowing. I think your analysis is correct. However, in my opinion, ZFS should make sure that the access is aligned when it issues the operation to the underlying GEOM provider, and banning import of pools with larger ashift is just papering problems out, which is not desirable. [...] > I suspect this came about because of changes to how ashift was > being reported / faked, very simular to the patches many of us have > for FreeBSD. > > If I'm correct this strengthens my view that this change is > incorrect, as what was trying to be done was support for faked > sector size on devices such as SSD's and HD's which report and > importantly support 512b sectors even if they are reported to ZFS > as 4k. This is very different from supporting a device which ONLY > supports a large sector size and hence smaller accesses would fail > which this change allows. Well, we actually need to answer two questions, not just one: a) should a pool be importable when ashift increases? I think the answer is "Yes" here, which is done by r253441. No, this is not optimal, but it does not really hurt anything if we make ZFS do the right thing, which is the second issue: b) should ZFS access smaller block when ashift demands larger ones? My answer would be "No" and in my opinion, this is the real underlying issue that needs to be addressed -- I would not really mind if we revert r253441 for now, but it would make us diverge further from upstream and, on the other hand, doesn't really buy us anything and papers the real issue out. Cheers, -----BEGIN PGP SIGNATURE----- iQEcBAEBCAAGBQJR+e3fAAoJEG80Jeu8UPuzkdYIAIUCjLGT+9t/qn3r8/7Lce7f zU85TuPnANnfNoD3xxq2rLqrLu+23w4fHdlzPTWG2GGdHS1s5VeVt3zl0SO3uhuW 43l+ecJL8Kd3cl2n1PTiHPwENnebm6OOPg8S7RDdxSJUX/Jvz5Of9UJRPtuh6J9p s/DwLDvPMGHVEVuZ0XkJbxvz/NkqMy08iizLBI3MJnZ61z4OjmFlhCJ1UiZiBPSV SU7zKRtjU221RHhSV0xm21fD+b5fRjBOgMS88uZPF27VI/NT0NfpAgX9fMtBzcSw faQSfEExb00NBiVsDooaZPrwuQXuvtHfaDK3y0w4JR/3kwV1wA9TJU5d1cVDLXY= =7R36 -----END PGP SIGNATURE----- From owner-zfs-devel@FreeBSD.ORG Thu Aug 1 08:23:35 2013 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id B83EA52B; Thu, 1 Aug 2013 08:23:35 +0000 (UTC) (envelope-from prvs=19253618d7=killing@multiplay.co.uk) Received: from mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) (using TLSv1 with cipher RC4-MD5 (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 30B632178; Thu, 1 Aug 2013 08:23:34 +0000 (UTC) Received: from r2d2 ([82.69.141.170]) by mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) (MDaemon PRO v10.0.4) with ESMTP id md50005255389.msg; Thu, 01 Aug 2013 09:23:27 +0100 X-Spam-Processed: mail1.multiplay.co.uk, Thu, 01 Aug 2013 09:23:27 +0100 (not processed: message from valid local sender) X-MDDKIM-Result: neutral (mail1.multiplay.co.uk) X-MDRemoteIP: 82.69.141.170 X-Return-Path: prvs=19253618d7=killing@multiplay.co.uk X-Envelope-From: killing@multiplay.co.uk Message-ID: From: "Steven Hartland" To: References: <201307180022.r6I0MgeS094364@svn.freebsd.org> <251C370CF0FF4EE686D571A1235D4C90@multiplay.co.uk> <51E73BCA.3030404@delphij.net> <95286A3C69D44769973FB0E072A84AB0@multiplay.co.uk> <51F9EDDF.4080100@delphij.net> Subject: Re: Invalid ashift increase allowed by r253441 Date: Thu, 1 Aug 2013 09:24:01 +0100 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="iso-8859-1"; reply-type=original Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.5931 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.6157 Cc: Xin LI , zfs-devel@FreeBSD.org, d@delphij.net X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 01 Aug 2013 08:23:35 -0000 ----- Original Message ----- From: "Xin Li" > Hi, Steven, > > On 7/30/13 5:16 PM, Steven Hartland wrote: > [...] >>> For example on a native 4k sectorsize disk its not possible to >>> directly access the bytes at offset 512 instead you would need to >>> read offset 0 and scan in 512 bytes to the returned data block. I >>> didn't think geom / zfs dealt with this case? > > No, GEOM does not allow this. > >>> The only case I can see this happening is if an old 512b >>> sectorsize disk was dd'ed to a new 4k sectorsize one but thats >>> effectively what this change is allowing. > > I think your analysis is correct. However, in my opinion, ZFS should > make sure that the access is aligned when it issues the operation to > the underlying GEOM provider, and banning import of pools with larger > ashift is just papering problems out, which is not desirable. > > [...] >> I suspect this came about because of changes to how ashift was >> being reported / faked, very simular to the patches many of us have >> for FreeBSD. >> >> If I'm correct this strengthens my view that this change is >> incorrect, as what was trying to be done was support for faked >> sector size on devices such as SSD's and HD's which report and >> importantly support 512b sectors even if they are reported to ZFS >> as 4k. This is very different from supporting a device which ONLY >> supports a large sector size and hence smaller accesses would fail >> which this change allows. > > Well, we actually need to answer two questions, not just one: > > a) should a pool be importable when ashift increases? > > I think the answer is "Yes" here, which is done by r253441. No, this > is not optimal, but it does not really hurt anything if we make ZFS do > the right thing, which is the second issue: > > b) should ZFS access smaller block when ashift demands larger ones? > > My answer would be "No" and in my opinion, this is the real underlying > issue that needs to be addressed -- I would not really mind if we > revert r253441 for now, but it would make us diverge further from > upstream and, on the other hand, doesn't really buy us anything and > papers the real issue out. When I first thought about this I was in agreement that it really should deal with this case but then I thought about the overhead this would add for and every single IO request, and was would that really be worth it? Given the performance impact that is very evident when you use SSD's that lie about their sector size I'd have to say I don't think so. Given this I currently plan to raise this upstream with the view to have this change backed out. If iilumos wants to allow disk sector size quirks it should do so properly like we've been talking about at i.e. by using a combination of physical and logical sector size reporting. Regards Steve Regards Steve ================================================ This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmaster@multiplay.co.uk. From owner-zfs-devel@FreeBSD.ORG Thu Aug 1 08:46:36 2013 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id CF2DBE6C; Thu, 1 Aug 2013 08:46:36 +0000 (UTC) (envelope-from delphij@delphij.net) Received: from anubis.delphij.net (anubis.delphij.net [64.62.153.212]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id B5125229C; Thu, 1 Aug 2013 08:46:36 +0000 (UTC) Received: from delphij-macbook.local (c-67-188-85-47.hsd1.ca.comcast.net [67.188.85.47]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by anubis.delphij.net (Postfix) with ESMTPSA id 2A0C116508; Thu, 1 Aug 2013 01:46:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=delphij.net; s=anubis; t=1375346790; bh=SBBUPbmX41g8UejOf332/uP+cB75WDLgTN42p1VwgaE=; h=Date:From:Reply-To:To:CC:Subject:References:In-Reply-To; b=XVERMS/NpPuUZllM4VJHWHHl0X4CiHUJM/1rSey4SfDGPCLX71ZtzMt5xafPTiuPQ q/H0dBKdlFAiVEUQ3a71yiU18kEYX3/JENjXxEpTRlnyXm+l/ObMZFOGPxAMI00rP3 s6HVnOPl0BR974FGiauRG3eErKWvubs+o/IBMTWY= Message-ID: <51FA2063.30709@delphij.net> Date: Thu, 01 Aug 2013 01:46:27 -0700 From: Xin Li Organization: The FreeBSD Project MIME-Version: 1.0 To: Steven Hartland Subject: Re: Invalid ashift increase allowed by r253441 References: <201307180022.r6I0MgeS094364@svn.freebsd.org> <251C370CF0FF4EE686D571A1235D4C90@multiplay.co.uk> <51E73BCA.3030404@delphij.net> <95286A3C69D44769973FB0E072A84AB0@multiplay.co.uk> <51F9EDDF.4080100@delphij.net> In-Reply-To: X-Enigmail-Version: 1.5.1 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: Xin LI , zfs-devel@FreeBSD.org, d@delphij.net X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list Reply-To: d@delphij.net List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 01 Aug 2013 08:46:36 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 On 8/1/13 1:24 AM, Steven Hartland wrote: > When I first thought about this I was in agreement that it really > should deal with this case but then I thought about the overhead > this would add for and every single IO request, and was would that > really be worth it? > > Given the performance impact that is very evident when you use > SSD's that lie about their sector size I'd have to say I don't > think so. I'm not sure if I have followed -- could you be more specific on what kind of overhead? (Speaking for the escalated read or read before write, I think we just can't avoid it without recreating the pool, assuming an ashift=9 image is dd'ed into an ashift=12 storage.) Or are you talking about something that I have overlooked? Cheers, -----BEGIN PGP SIGNATURE----- iQEcBAEBCAAGBQJR+iBiAAoJEG80Jeu8UPuz2KAIAJCCUmoj5d6TjclpVg+wdjT+ QZ9hO7sI8Awoes8k4YGUjWAJHia7IgFxrASVgdKJlDBXUB92QoFGnXnHvaIZxLzY XhIL9vgVi/2roLzG/mlkmaXVPuuso1nCjE+6wVu9tl9+SkaGOYhVEZLsbdqftLHl MlwxqbCNIWXPmc1JkpPWDnU6fUEPe6iyjVtZqMZnNkWvrt4R6Uc63DTUL/NrVnrJ N/cbItQlvQb1CEyoVyW/1lybQXJERlRdit4XVicF8EMX9YMHX42XpkpDt0YjdaTM lLE0vOUaK9igmdCiFqB86pH/SWEg/DiLJBv80FBNxCF0i+kE40RfQ9vMvIkB7z4= =o2h2 -----END PGP SIGNATURE----- From owner-zfs-devel@FreeBSD.ORG Thu Aug 1 09:56:50 2013 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 0C84B1E0; Thu, 1 Aug 2013 09:56:50 +0000 (UTC) (envelope-from prvs=19253618d7=killing@multiplay.co.uk) Received: from mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) (using TLSv1 with cipher RC4-MD5 (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 7B0D62676; Thu, 1 Aug 2013 09:56:49 +0000 (UTC) Received: from r2d2 ([82.69.141.170]) by mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) (MDaemon PRO v10.0.4) with ESMTP id md50005257160.msg; Thu, 01 Aug 2013 10:56:46 +0100 X-Spam-Processed: mail1.multiplay.co.uk, Thu, 01 Aug 2013 10:56:46 +0100 (not processed: message from valid local sender) X-MDDKIM-Result: neutral (mail1.multiplay.co.uk) X-MDRemoteIP: 82.69.141.170 X-Return-Path: prvs=19253618d7=killing@multiplay.co.uk X-Envelope-From: killing@multiplay.co.uk Message-ID: <3AF8D53CF58049FABBFB6A69757BE4EF@multiplay.co.uk> From: "Steven Hartland" To: References: <201307180022.r6I0MgeS094364@svn.freebsd.org> <251C370CF0FF4EE686D571A1235D4C90@multiplay.co.uk> <51E73BCA.3030404@delphij.net> <95286A3C69D44769973FB0E072A84AB0@multiplay.co.uk> <51F9EDDF.4080100@delphij.net> <51FA2063.30709@delphij.net> Subject: Re: Invalid ashift increase allowed by r253441 Date: Thu, 1 Aug 2013 10:57:23 +0100 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="iso-8859-1"; reply-type=original Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.5931 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.6157 Cc: Xin LI , zfs-devel@FreeBSD.org, d@delphij.net X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 01 Aug 2013 09:56:50 -0000 ----- Original Message ----- From: "Xin Li" To: "Steven Hartland" Cc: ; "Xin LI" ; Sent: Thursday, August 01, 2013 9:46 AM Subject: Re: Invalid ashift increase allowed by r253441 > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA256 > > On 8/1/13 1:24 AM, Steven Hartland wrote: >> When I first thought about this I was in agreement that it really >> should deal with this case but then I thought about the overhead >> this would add for and every single IO request, and was would that >> really be worth it? >> >> Given the performance impact that is very evident when you use >> SSD's that lie about their sector size I'd have to say I don't >> think so. > > I'm not sure if I have followed -- could you be more specific on what > kind of overhead? (Speaking for the escalated read or read before > write, I think we just can't avoid it without recreating the pool, > assuming an ashift=9 image is dd'ed into an ashift=12 storage.) > > Or are you talking about something that I have overlooked? I see two use cases for this:- 1. Disk quirks to make a 4K native disk report as such. 2. Manually copying a pool disk from one device to another with a larger sector size. #1 is going to be the most common case and this should be dealt with by correctly reporting the two values to ZFS, no other changes are needed for this to work. The performance won't be optimal but thats to be expected. #2 is where you're going to need ZFS to do validation / conversion on every IO to check if its needs alignment changes, escalate reads or add read before write for small writes. This is going to be petty rare and I don't believe the rareness justifies slowing down for the common case (as you'd always need to check every IO if this was supported, even if said device didn't require it). That said it may be possible add a transformation stage into the zio pipeline on pool load for devices that require it, so the impact to devices which don't require transformation would be zero. I don't think this will a simple task though so I believe the current way forward should be:- 1. Revert the ashift check so it fails devices with larger ashift. 2. Add proper support for physical and logical sector sizes reporting for ashift. 3. Look at adding translation support for larger logical sector size devices than the current ashift. Regards Steve ================================================ This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmaster@multiplay.co.uk. From owner-zfs-devel@FreeBSD.ORG Thu Aug 1 14:03:13 2013 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 894177E6; Thu, 1 Aug 2013 14:03:13 +0000 (UTC) (envelope-from mavbsd@gmail.com) Received: from mail-lb0-x232.google.com (mail-lb0-x232.google.com [IPv6:2a00:1450:4010:c04::232]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 6826324E5; Thu, 1 Aug 2013 14:03:12 +0000 (UTC) Received: by mail-lb0-f178.google.com with SMTP id z5so1526432lbh.23 for ; Thu, 01 Aug 2013 07:03:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:content-type:content-transfer-encoding; bh=Zb0CmKGjHwVkkDOHv/dX1sDo31uat/pYjDT1J9i7TXE=; b=1BiNgflyDtvYp7u2AcCdB4TER0QeqGhO9Rbq+FeEYf7dVyG8aH+dZkK0QZZqS5RnZr zLtXzkLafD+a7o07hrjgzVYJoj9foSsxCZ/qzWTZwLPE2kii0w+DNVnp2NgzmvAQXRuS DjlHyth/LBNWi3OJV+SmdKOSgKWeSLS9D/dNWBGMGJoQeCwQ/tD6zCqfGKFoKm8YJyxh ZGqENhln6pcVByONJkQUm8+MZP5WuOwxLVqS8hvImzwgshgft67OP3CCrHaeC8qvWW23 vWytsOGrWhTWutmXKVDQrePa0sQjs7WBBI4taeeZfHnPdgJypL8HpjLySgLq/gELFvXp CubQ== X-Received: by 10.152.9.194 with SMTP id c2mr795147lab.83.1375365790207; Thu, 01 Aug 2013 07:03:10 -0700 (PDT) Received: from mavbook.mavhome.dp.ua (mavhome.mavhome.dp.ua. [213.227.240.37]) by mx.google.com with ESMTPSA id v18sm1330498lbd.5.2013.08.01.07.03.07 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Thu, 01 Aug 2013 07:03:09 -0700 (PDT) Sender: Alexander Motin Message-ID: <51FA6A99.8020300@FreeBSD.org> Date: Thu, 01 Aug 2013 17:03:05 +0300 From: Alexander Motin User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:17.0) Gecko/20130616 Thunderbird/17.0.6 MIME-Version: 1.0 To: Martin Matuska , Pawel Jakub Dawidek Subject: Re: HEAD r252840 (illumos bug 3836) and our TRIM are incompatible, causing deadlocks References: <51F8FC0C.2030703@FreeBSD.org> In-Reply-To: <51F8FC0C.2030703@FreeBSD.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: zfs-devel@FreeBSD.org X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 01 Aug 2013 14:03:13 -0000 On 31.07.2013 14:59, Alexander Motin wrote: > With some experiments I've got to believe that HEAD r252840 (illumos bug > 3836) and our TRIM implementation are mutually incompatible. I have > found 100% repeatable scenario how to cause deadlock when these changes > are applied together. All that needed is to create significant write > load (I've used `iozone -t 16 -s 8G` on 8-core system with 2GB RAM and 2 > striped SSDs) and run `zpool clear poolname`. After that system is > effectively dead: all I/O are stuck and even zpool commands are no > longer functioning. I think triggering event is not necessary should be > `zpool clear`, any event/action that takes SCL_ZIO lock for writing > should do the same. > > r252840 (illumos bug 3836) is based on assumption that zio_free_sync() > has no lock dependencies and should complete immediately. Unfortunately, > with our TRIM implementation that is not true due to > ZIO_STAGE_VDEV_IO_START added to the ZIO_FREE_PIPELINE, which, while not > really accessing devices, still acquires SCL_ZIO lock for read. As > result, we are getting such deadlock: `zpool clear` asks for SC_ZIO for > writing and waits for all read locks to be dropped; SC_ZIO is held for > read by regular I/Os that were running and are completed now, but to > drop the lock they require free zio_write_intr thread; unfortunately all > zio_write_intr treads under high load are stuck inside modified > zio_free(), that is now trying to directly execute zio_free_sync(), that > with our TRIM implementation tries to obtain SCL_ZIO for read; BANG! > DEADLOCK! > > Reverting r252840 fixes the situation for me. If somebody have ideas how > to fix the situation without reverting either changes -- welcome. This patch workarounds the problem: http://people.freebsd.org/~mav/zfs_patches/direct_free_and_trim.patch It disables r252840 when ZFS TRIM is enabled (vfs.zfs.trim.enabled=1). When TRIM is disabled, patch enables direct free execution from r252840 and removes ZIO_STAGE_VDEV_IO_START and ZIO_STAGE_VDEV_IO_ASSESS stages from the pipeline. -- Alexander Motin From owner-zfs-devel@FreeBSD.ORG Sun Aug 4 09:25:52 2013 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id EC9F797C; Sun, 4 Aug 2013 09:25:52 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from mail.vx.sk (mail.vx.sk [IPv6:2a01:4f8:150:6101::4]) by mx1.freebsd.org (Postfix) with ESMTP id 4EDC4236B; Sun, 4 Aug 2013 09:25:52 +0000 (UTC) Received: from core.vx.sk (localhost [127.0.0.2]) by mail.vx.sk (Postfix) with ESMTP id 794D33EA99; Sun, 4 Aug 2013 11:25:51 +0200 (CEST) X-Virus-Scanned: amavisd-new at mail.vx.sk Received: from mail.vx.sk by core.vx.sk (amavisd-new, unix socket) with LMTP id sD3jABiuasa2; Sun, 4 Aug 2013 11:25:51 +0200 (CEST) Received: from [192.168.2.103] (dslb-178-005-090-069.pools.arcor-ip.net [178.5.90.69]) by mail.vx.sk (Postfix) with ESMTPSA id D05DB3EA91; Sun, 4 Aug 2013 11:25:50 +0200 (CEST) Message-ID: <51FE1E1E.8080502@FreeBSD.org> Date: Sun, 04 Aug 2013 11:25:50 +0200 From: Martin Matuska User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:17.0) Gecko/20130623 Thunderbird/17.0.7 MIME-Version: 1.0 To: zfs-devel@freebsd.org Subject: Re: N-way mirror read speedup in zfsonlinux References: <20130712112117.Horde.JMB3fhsU1VfZRxdV4prj5Q6@mail.vx.sk> In-Reply-To: <20130712112117.Horde.JMB3fhsU1VfZRxdV4prj5Q6@mail.vx.sk> Content-Type: multipart/mixed; boundary="------------030902000700020407030307" Cc: Xin Li X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 04 Aug 2013 09:25:53 -0000 This is a multi-part message in MIME format. --------------030902000700020407030307 Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 8bit Attached is a FreeBSD version of this patch for testing and comments, including sysctl tunable: http://people.freebsd.org/~mm/patches/zfs/vdev_mirror.c.patch On 2013-07-12 11:21, Martin Matuka wrote: > Hi everyone, > > zfsonlinux has implemented a change in the N-way mirror device selection > algorithm by selecting the device with the least pending I/O instead of > random selection. They measured an increased read bandwidth increase > up to > 50% and IOPS increase up to 10%. > > this might be useful for common ZFS code and we might consider porting > this > to illumos and FreeBSD: > https://github.com/zfsonlinux/zfs/issues/1461 > https://github.com/zfsonlinux/zfs/commit/556011dbec2d10579819078559a77630fc559112 > --------------030902000700020407030307 Content-Type: text/x-patch; name="vdev_mirror.c.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="vdev_mirror.c.patch" Index: sys/cddl/compat/opensolaris/sys/time.h =================================================================== --- sys/cddl/compat/opensolaris/sys/time.h (revision 253925) +++ sys/cddl/compat/opensolaris/sys/time.h (working copy) @@ -50,6 +50,8 @@ #define SEC_TO_TICK(sec) ((sec) * hz) #define NSEC_TO_TICK(usec) ((usec) / (NANOSEC / hz)) +#define NSEC_PER_USEC 1000 + #ifdef _KERNEL static __inline hrtime_t gethrtime(void) { Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_mirror.c =================================================================== --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_mirror.c (revision 253925) +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_mirror.c (working copy) @@ -33,6 +33,8 @@ #include #include +SYSCTL_DECL(_vfs_zfs_vdev); + /* * Virtual device vector for mirroring. */ @@ -41,6 +43,7 @@ vdev_t *mc_vd; uint64_t mc_offset; int mc_error; + int mc_pending; uint8_t mc_tried; uint8_t mc_skipped; uint8_t mc_speculative; @@ -54,7 +57,20 @@ mirror_child_t mm_child[1]; } mirror_map_t; -int vdev_mirror_shift = 21; +/* + * When the children are equally busy queue incoming requests to a single + * child for N microseconds. + * Otherwise, requests are queued to the least busy device. + * + * For fast SSDs it may make sense to decrease zfs_vdev_mirror_switch_us + * significantly to bound the worst case latencies. It would probably be + * ideal to calculate a decaying average of the last observed latencies and + * use that to dynamically adjust the zfs_vdev_mirror_switch_us time. + */ +int zfs_vdev_mirror_switch_us = 10000; +TUNABLE_INT("vfs.zfs.vdev.mirror_switch_us", &zfs_vdev_mirror_switch_us); +SYSCTL_INT(_vfs_zfs_vdev, OID_AUTO, mirror_switch_us, CTLFLAG_RW, + &zfs_vdev_mirror_switch_us, 0, "Switch mirrors every N usecs"); static void vdev_mirror_map_free(zio_t *zio) @@ -69,6 +85,19 @@ zio_vsd_default_cksum_report }; +static int +vdev_mirror_pending(vdev_t *vd) +{ + vdev_queue_t *vq = &vd->vdev_queue; + int pending; + + mutex_enter(&vq->vq_lock); + pending = avl_numnodes(&vq->vq_pending_tree); + mutex_exit(&vq->vq_lock); + + return (pending); +} + static mirror_map_t * vdev_mirror_map_alloc(zio_t *zio) { @@ -108,6 +137,9 @@ mc->mc_offset = DVA_GET_OFFSET(&dva[c]); } } else { + int lowest_pending = INT_MAX; + int lowest_nr = 1; + c = vd->vdev_children; mm = kmem_zalloc(offsetof(mirror_map_t, mm_child[c]), KM_SLEEP); @@ -114,8 +146,7 @@ mm->mm_children = c; mm->mm_replacing = (vd->vdev_ops == &vdev_replacing_ops || vd->vdev_ops == &vdev_spare_ops); - mm->mm_preferred = mm->mm_replacing ? 0 : - (zio->io_offset >> vdev_mirror_shift) % c; + mm->mm_preferred = 0; mm->mm_root = B_FALSE; for (c = 0; c < mm->mm_children; c++) { @@ -122,7 +153,40 @@ mc = &mm->mm_child[c]; mc->mc_vd = vd->vdev_child[c]; mc->mc_offset = zio->io_offset; + + if (mm->mm_replacing) + continue; + + if (!vdev_readable(mc->mc_vd)) { + mc->mc_error = ENXIO; + mc->mc_tried = 1; + mc->mc_skipped = 1; + mc->mc_pending = INT_MAX; + continue; + } + + mc->mc_pending = vdev_mirror_pending(mc->mc_vd); + if (mc->mc_pending < lowest_pending) { + lowest_pending = mc->mc_pending; + lowest_nr = 1; + } else if (mc->mc_pending == lowest_pending) { + lowest_nr++; + } } + + d = gethrtime() / (NSEC_PER_USEC * zfs_vdev_mirror_switch_us); + d = (d % lowest_nr) + 1; + + for (c = 0; c < mm->mm_children; c++) { + mc = &mm->mm_child[c]; + + if (mm->mm_child[c].mc_pending == lowest_pending) { + if (--d == 0) { + mm->mm_preferred = c; + break; + } + } + } } zio->io_vsd = mm; --------------030902000700020407030307-- From owner-zfs-devel@FreeBSD.ORG Sun Aug 4 12:56:07 2013 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id DAD3C38D; Sun, 4 Aug 2013 12:56:06 +0000 (UTC) (envelope-from prvs=1928f37ee5=killing@multiplay.co.uk) Received: from mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) (using TLSv1 with cipher RC4-MD5 (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 99A1827C1; Sun, 4 Aug 2013 12:56:05 +0000 (UTC) Received: from r2d2 ([82.69.141.170]) by mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) (MDaemon PRO v10.0.4) with ESMTP id md50005325686.msg; Sun, 04 Aug 2013 13:56:02 +0100 X-Spam-Processed: mail1.multiplay.co.uk, Sun, 04 Aug 2013 13:56:02 +0100 (not processed: message from valid local sender) X-MDDKIM-Result: neutral (mail1.multiplay.co.uk) X-MDRemoteIP: 82.69.141.170 X-Return-Path: prvs=1928f37ee5=killing@multiplay.co.uk X-Envelope-From: killing@multiplay.co.uk Message-ID: From: "Steven Hartland" To: "Alexander Motin" , "Martin Matuska" , "Pawel Jakub Dawidek" References: <51F8FC0C.2030703@FreeBSD.org> <51FA6A99.8020300@FreeBSD.org> Subject: Re: HEAD r252840 (illumos bug 3836) and our TRIM are incompatible, causing deadlocks Date: Sun, 4 Aug 2013 13:56:40 +0100 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="iso-8859-1"; reply-type=response Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.5931 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.6157 Cc: zfs-devel@FreeBSD.org X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 04 Aug 2013 12:56:07 -0000 ----- Original Message ----- From: "Alexander Motin" To: "Martin Matuska" ; "Pawel Jakub Dawidek" Cc: ; "Steven Hartland" Sent: Thursday, August 01, 2013 3:03 PM Subject: Re: HEAD r252840 (illumos bug 3836) and our TRIM are incompatible, causing deadlocks > On 31.07.2013 14:59, Alexander Motin wrote: >> With some experiments I've got to believe that HEAD r252840 (illumos bug >> 3836) and our TRIM implementation are mutually incompatible. I have >> found 100% repeatable scenario how to cause deadlock when these changes >> are applied together. All that needed is to create significant write >> load (I've used `iozone -t 16 -s 8G` on 8-core system with 2GB RAM and 2 >> striped SSDs) and run `zpool clear poolname`. After that system is >> effectively dead: all I/O are stuck and even zpool commands are no >> longer functioning. I think triggering event is not necessary should be >> `zpool clear`, any event/action that takes SCL_ZIO lock for writing >> should do the same. >> >> r252840 (illumos bug 3836) is based on assumption that zio_free_sync() >> has no lock dependencies and should complete immediately. Unfortunately, >> with our TRIM implementation that is not true due to >> ZIO_STAGE_VDEV_IO_START added to the ZIO_FREE_PIPELINE, which, while not >> really accessing devices, still acquires SCL_ZIO lock for read. As >> result, we are getting such deadlock: `zpool clear` asks for SC_ZIO for >> writing and waits for all read locks to be dropped; SC_ZIO is held for >> read by regular I/Os that were running and are completed now, but to >> drop the lock they require free zio_write_intr thread; unfortunately all >> zio_write_intr treads under high load are stuck inside modified >> zio_free(), that is now trying to directly execute zio_free_sync(), that >> with our TRIM implementation tries to obtain SCL_ZIO for read; BANG! >> DEADLOCK! >> >> Reverting r252840 fixes the situation for me. If somebody have ideas how >> to fix the situation without reverting either changes -- welcome. > > This patch workarounds the problem: > http://people.freebsd.org/~mav/zfs_patches/direct_free_and_trim.patch > > It disables r252840 when ZFS TRIM is enabled (vfs.zfs.trim.enabled=1). > When TRIM is disabled, patch enables direct free execution from r252840 > and removes ZIO_STAGE_VDEV_IO_START and ZIO_STAGE_VDEV_IO_ASSESS stages > from the pipeline. I assume you removed the vdev stages when trim is disabled as an optimization, due to the fact that a free wouldn't result in any physical IO? Regards Steve ================================================ This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmaster@multiplay.co.uk. From owner-zfs-devel@FreeBSD.ORG Sun Aug 4 13:53:00 2013 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 6DF5490; Sun, 4 Aug 2013 13:53:00 +0000 (UTC) (envelope-from mavbsd@gmail.com) Received: from mail-ee0-x235.google.com (mail-ee0-x235.google.com [IPv6:2a00:1450:4013:c00::235]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id A4FED29D2; Sun, 4 Aug 2013 13:52:59 +0000 (UTC) Received: by mail-ee0-f53.google.com with SMTP id b15so1143391eek.26 for ; Sun, 04 Aug 2013 06:52:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:content-type:content-transfer-encoding; bh=kkmwQ4SNDAWZTPCEABteQrS6Nn+/4XhYRrsd0yxM/6M=; b=hV7/P9Ot6fdQypHOKscoihosqPHPsa3r9IwBNWedj90S6RLe6KHy+nkMz6T8naL2RW SMuDYBOEoK3SJu+RbDst3nJ9v5IXXaqpGWXsFgFOK9ZkBhyJ6uqoFNYvR8Gdhp7qs8S8 fq4IFM+Z8ALhxFeyEuPyznrNzSoN9UmqR11i6T84HxfgEW89f9vUV9dC+R5dXHREbIgg +IxHoLBRCwQ1Q7g6EuKmabxkWB8p3iRJPVxxxgIIh9QJaNe7R8SLuWY41zEp2k4VqMsV Hpv7T0IVGBG/Doj8HdZQ3MqyJGMZd8skdm4/Z9feVLUxkT0FPPJekAdUEzSzLtpaHCQj 5jXw== X-Received: by 10.14.102.72 with SMTP id c48mr13203048eeg.52.1375624377840; Sun, 04 Aug 2013 06:52:57 -0700 (PDT) Received: from mavbook.mavhome.dp.ua ([37.229.21.195]) by mx.google.com with ESMTPSA id t6sm16863168eel.12.2013.08.04.06.52.55 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Sun, 04 Aug 2013 06:52:56 -0700 (PDT) Sender: Alexander Motin Message-ID: <51FE5CB6.5060703@FreeBSD.org> Date: Sun, 04 Aug 2013 16:52:54 +0300 From: Alexander Motin User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:17.0) Gecko/20130616 Thunderbird/17.0.6 MIME-Version: 1.0 To: Steven Hartland Subject: Re: HEAD r252840 (illumos bug 3836) and our TRIM are incompatible, causing deadlocks References: <51F8FC0C.2030703@FreeBSD.org> <51FA6A99.8020300@FreeBSD.org> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: zfs-devel@FreeBSD.org X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 04 Aug 2013 13:53:00 -0000 On 04.08.2013 15:56, Steven Hartland wrote: >> On 31.07.2013 14:59, Alexander Motin wrote: >>> With some experiments I've got to believe that HEAD r252840 (illumos bug >>> 3836) and our TRIM implementation are mutually incompatible. I have >>> found 100% repeatable scenario how to cause deadlock when these changes >>> are applied together. All that needed is to create significant write >>> load (I've used `iozone -t 16 -s 8G` on 8-core system with 2GB RAM and 2 >>> striped SSDs) and run `zpool clear poolname`. After that system is >>> effectively dead: all I/O are stuck and even zpool commands are no >>> longer functioning. I think triggering event is not necessary should be >>> `zpool clear`, any event/action that takes SCL_ZIO lock for writing >>> should do the same. >>> >>> r252840 (illumos bug 3836) is based on assumption that zio_free_sync() >>> has no lock dependencies and should complete immediately. Unfortunately, >>> with our TRIM implementation that is not true due to >>> ZIO_STAGE_VDEV_IO_START added to the ZIO_FREE_PIPELINE, which, while not >>> really accessing devices, still acquires SCL_ZIO lock for read. As >>> result, we are getting such deadlock: `zpool clear` asks for SC_ZIO for >>> writing and waits for all read locks to be dropped; SC_ZIO is held for >>> read by regular I/Os that were running and are completed now, but to >>> drop the lock they require free zio_write_intr thread; unfortunately all >>> zio_write_intr treads under high load are stuck inside modified >>> zio_free(), that is now trying to directly execute zio_free_sync(), that >>> with our TRIM implementation tries to obtain SCL_ZIO for read; BANG! >>> DEADLOCK! >>> >>> Reverting r252840 fixes the situation for me. If somebody have ideas how >>> to fix the situation without reverting either changes -- welcome. >> >> This patch workarounds the problem: >> http://people.freebsd.org/~mav/zfs_patches/direct_free_and_trim.patch >> >> It disables r252840 when ZFS TRIM is enabled (vfs.zfs.trim.enabled=1). >> When TRIM is disabled, patch enables direct free execution from >> r252840 and removes ZIO_STAGE_VDEV_IO_START and >> ZIO_STAGE_VDEV_IO_ASSESS stages from the pipeline. > > I assume you removed the vdev stages when trim is disabled as an > optimization, due to the fact that a free wouldn't result in any > physical IO? As I have written above, zio_vdev_io_start() called on top level takes SCL_ZIO, that causes described deadlock. That is why I needed to block those stages, useless in that case any way, just at the entrance point. I've decided it will be more effective to remove whole stages then add checks inside. -- Alexander Motin From owner-zfs-devel@FreeBSD.ORG Sun Aug 4 14:02:33 2013 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 345BC2AD; Sun, 4 Aug 2013 14:02:33 +0000 (UTC) (envelope-from prvs=1928f37ee5=killing@multiplay.co.uk) Received: from mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) (using TLSv1 with cipher RC4-MD5 (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 76A192A1A; Sun, 4 Aug 2013 14:02:32 +0000 (UTC) Received: from r2d2 ([82.69.141.170]) by mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) (MDaemon PRO v10.0.4) with ESMTP id md50005326199.msg; Sun, 04 Aug 2013 15:02:31 +0100 X-Spam-Processed: mail1.multiplay.co.uk, Sun, 04 Aug 2013 15:02:31 +0100 (not processed: message from valid local sender) X-MDDKIM-Result: neutral (mail1.multiplay.co.uk) X-MDRemoteIP: 82.69.141.170 X-Return-Path: prvs=1928f37ee5=killing@multiplay.co.uk X-Envelope-From: killing@multiplay.co.uk Message-ID: From: "Steven Hartland" To: "Alexander Motin" References: <51F8FC0C.2030703@FreeBSD.org> <51FA6A99.8020300@FreeBSD.org> <51FE5CB6.5060703@FreeBSD.org> Subject: Re: HEAD r252840 (illumos bug 3836) and our TRIM are incompatible, causing deadlocks Date: Sun, 4 Aug 2013 15:03:09 +0100 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="iso-8859-1"; reply-type=response Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.5931 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.6157 Cc: zfs-devel@FreeBSD.org X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 04 Aug 2013 14:02:33 -0000 ----- Original Message ----- From: "Alexander Motin" ... >>> It disables r252840 when ZFS TRIM is enabled (vfs.zfs.trim.enabled=1). >>> When TRIM is disabled, patch enables direct free execution from >>> r252840 and removes ZIO_STAGE_VDEV_IO_START and >>> ZIO_STAGE_VDEV_IO_ASSESS stages from the pipeline. >> >> I assume you removed the vdev stages when trim is disabled as an >> optimization, due to the fact that a free wouldn't result in any >> physical IO? > > As I have written above, zio_vdev_io_start() called on top level takes > SCL_ZIO, that causes described deadlock. That is why I needed to block > those stages, useless in that case any way, just at the entrance point. > I've decided it will be more effective to remove whole stages then add > checks inside. Thanks for the clarification, I'll continue testing :) Regards Steve ================================================ This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmaster@multiplay.co.uk. From owner-zfs-devel@FreeBSD.ORG Sun Aug 4 16:37:45 2013 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 040438D3; Sun, 4 Aug 2013 16:37:45 +0000 (UTC) (envelope-from mavbsd@gmail.com) Received: from mail-ee0-x234.google.com (mail-ee0-x234.google.com [IPv6:2a00:1450:4013:c00::234]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 3A96E2F0D; Sun, 4 Aug 2013 16:37:44 +0000 (UTC) Received: by mail-ee0-f52.google.com with SMTP id c41so1194951eek.25 for ; Sun, 04 Aug 2013 09:37:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:content-type:content-transfer-encoding; bh=B98qB2UXkP3eezMh1y9Mpar8gDEpGXUkGK5RMj0EHY0=; b=ba8UuIclW+5zmltJSqQ578eDZgr2Lfeqxrg1a5RhqFeAyUgRkRn4p6urxAYN3bkmTl kGdjJElrNR5iUoIvvstn+UaQ6GsXzt4HWBh9oq+67JBp3uLga6v+xQd7Xsuo05l+RpSA iMncD/J3UlbxNR3nZfYO2b81C0uqhJajYFDj9hUJywIwOCOn3/J9+RIuniENAzrco0Nh +sQG9xR3AlcJKWthUMivJOHPlexL9D/E2djCgMI5fzO82NvqCUe10tOV1cuoEiGF6Ycj ujHTpK2a5SfqjmE+XrOAqAoh15zhx+mozEPrsiyL5JQ0piFkoPDmUak0+MAWcku+VL3j u/PQ== X-Received: by 10.15.36.133 with SMTP id i5mr9083872eev.149.1375634262487; Sun, 04 Aug 2013 09:37:42 -0700 (PDT) Received: from mavbook.mavhome.dp.ua ([37.229.21.195]) by mx.google.com with ESMTPSA id cg12sm27152064eeb.7.2013.08.04.09.37.40 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Sun, 04 Aug 2013 09:37:41 -0700 (PDT) Sender: Alexander Motin Message-ID: <51FE8353.3070609@FreeBSD.org> Date: Sun, 04 Aug 2013 19:37:39 +0300 From: Alexander Motin User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:17.0) Gecko/20130616 Thunderbird/17.0.6 MIME-Version: 1.0 To: Martin Matuska Subject: Re: N-way mirror read speedup in zfsonlinux References: <20130712112117.Horde.JMB3fhsU1VfZRxdV4prj5Q6@mail.vx.sk> <51FE1E1E.8080502@FreeBSD.org> In-Reply-To: <51FE1E1E.8080502@FreeBSD.org> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 8bit Cc: zfs-devel@freebsd.org, Xin Li X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 04 Aug 2013 16:37:45 -0000 Hi. I like the idea of load-aware balancing. I went that way in both gmirror and graid and seen substantial performance improvements. But pure load-based balance may be ineffective in case of spinning HDDs and sequential I/Os. Doing that way will mitigate effect of the device read-ahead. If sequential read operations will be handled on triple mirror as ABCABC, we may get situation when all three devices are busy, but giving only read speed of one. vdev_mirror_shift used in the original balance code has its reason -- it will make all I/Os to the consequential offsets (within 2MB) go to the same disk. You may find G_RAID_SUBDISK_TRACK_SIZE constant in graid and TRACK_SIZE in gmirror doing exactly the same. I am not have doubt in effectiveness of proposed time-based magic. If there is only one request issued at a time, this algorithm indeed may give benefit by more effective using of the device read-ahead. But if we have more then one request active, the result will become purely random since first request sent will affect later ones and (d % lowest_nr) will give precedence to completely random device. Also I would like to note that getting precise time still can be expensive operation on some hardware to do it on every I/O. I think in many cases there will be only one device with lowest load, and reading of time could be skipped. On 04.08.2013 12:25, Martin Matuska wrote: > Attached is a FreeBSD version of this patch for testing and comments, > including sysctl tunable: > http://people.freebsd.org/~mm/patches/zfs/vdev_mirror.c.patch > > On 2013-07-12 11:21, Martin Matuka wrote: >> zfsonlinux has implemented a change in the N-way mirror device selection >> algorithm by selecting the device with the least pending I/O instead of >> random selection. They measured an increased read bandwidth increase >> up to >> 50% and IOPS increase up to 10%. >> >> this might be useful for common ZFS code and we might consider porting >> this >> to illumos and FreeBSD: >> https://github.com/zfsonlinux/zfs/issues/1461 >> https://github.com/zfsonlinux/zfs/commit/556011dbec2d10579819078559a77630fc559112 -- Alexander Motin From owner-zfs-devel@FreeBSD.ORG Sun Aug 4 16:40:08 2013 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id B1B6B904; Sun, 4 Aug 2013 16:40:08 +0000 (UTC) (envelope-from mavbsd@gmail.com) Received: from mail-ee0-x22e.google.com (mail-ee0-x22e.google.com [IPv6:2a00:1450:4013:c00::22e]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id E8A982F1E; Sun, 4 Aug 2013 16:40:07 +0000 (UTC) Received: by mail-ee0-f46.google.com with SMTP id c13so1179417eek.5 for ; Sun, 04 Aug 2013 09:40:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:content-type:content-transfer-encoding; bh=Hn2TTtjN65scFlFWV4m8BHQqEM2I5noHUFFjQrYse00=; b=CuHckHLPss6DueAD4MqaKm/0ZipwSMS0I1GNx80h/D6OSG86dDvbUkIoUfWZtBFDQ2 6zlRKdS7BgV6xg6iWe5FEoo10LO3QUV7I/TdEws0TbBycp2cA87XFMQC1YU/V4KENdmv q9eP1KeMMKgLqKWF4WEyz53Q1T3Ddzc7DHvHphSD0VUXWKJzjfEqxSm+vhrE6Po+Atnd 51HAyx1u607Fv+5jI7iCPT8MauO5BUPxHlZEdQvEEjM4OnaIuJOQJ/16YXXKEZ4D6PeM i/QosO7W2kvHBQ0RuH01AU5jmJxpTA4bYzF1mXA6Tej9pSPfu/MQNeJlhrwT/WeymGtT x9Ow== X-Received: by 10.15.63.8 with SMTP id l8mr13609529eex.23.1375634406310; Sun, 04 Aug 2013 09:40:06 -0700 (PDT) Received: from mavbook.mavhome.dp.ua ([37.229.21.195]) by mx.google.com with ESMTPSA id j2sm27166519eep.6.2013.08.04.09.40.04 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Sun, 04 Aug 2013 09:40:05 -0700 (PDT) Sender: Alexander Motin Message-ID: <51FE83E3.2070209@FreeBSD.org> Date: Sun, 04 Aug 2013 19:40:03 +0300 From: Alexander Motin User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:17.0) Gecko/20130616 Thunderbird/17.0.6 MIME-Version: 1.0 To: Martin Matuska Subject: Re: N-way mirror read speedup in zfsonlinux References: <20130712112117.Horde.JMB3fhsU1VfZRxdV4prj5Q6@mail.vx.sk> <51FE1E1E.8080502@FreeBSD.org> <51FE8353.3070609@FreeBSD.org> In-Reply-To: <51FE8353.3070609@FreeBSD.org> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 8bit Cc: zfs-devel@freebsd.org, Xin Li X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 04 Aug 2013 16:40:08 -0000 On 04.08.2013 19:37, Alexander Motin wrote: > Hi. > > I like the idea of load-aware balancing. I went that way in both gmirror > and graid and seen substantial performance improvements. But pure > load-based balance may be ineffective in case of spinning HDDs and > sequential I/Os. Doing that way will mitigate effect of the device > read-ahead. If sequential read operations will be handled on triple > mirror as ABCABC, we may get situation when all three devices are busy, > but giving only read speed of one. vdev_mirror_shift used in the > original balance code has its reason -- it will make all I/Os to the > consequential offsets (within 2MB) go to the same disk. You may find > G_RAID_SUBDISK_TRACK_SIZE constant in graid and TRACK_SIZE in gmirror > doing exactly the same. > > I am not have doubt in effectiveness of proposed time-based magic. If Sorry, I mean I DO have doubt ... > there is only one request issued at a time, this algorithm indeed may > give benefit by more effective using of the device read-ahead. But if we > have more then one request active, the result will become purely random > since first request sent will affect later ones and (d % lowest_nr) will > give precedence to completely random device. > > Also I would like to note that getting precise time still can be > expensive operation on some hardware to do it on every I/O. I think in > many cases there will be only one device with lowest load, and reading > of time could be skipped. > > On 04.08.2013 12:25, Martin Matuska wrote: >> Attached is a FreeBSD version of this patch for testing and comments, >> including sysctl tunable: >> http://people.freebsd.org/~mm/patches/zfs/vdev_mirror.c.patch >> >> On 2013-07-12 11:21, Martin Matuka wrote: >>> zfsonlinux has implemented a change in the N-way mirror device selection >>> algorithm by selecting the device with the least pending I/O instead of >>> random selection. They measured an increased read bandwidth increase >>> up to >>> 50% and IOPS increase up to 10%. >>> >>> this might be useful for common ZFS code and we might consider porting >>> this >>> to illumos and FreeBSD: >>> https://github.com/zfsonlinux/zfs/issues/1461 >>> https://github.com/zfsonlinux/zfs/commit/556011dbec2d10579819078559a77630fc559112 >>> > -- Alexander Motin From owner-zfs-devel@FreeBSD.ORG Sun Aug 4 18:17:46 2013 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id E6E99FC0; Sun, 4 Aug 2013 18:17:45 +0000 (UTC) (envelope-from prvs=1928f37ee5=killing@multiplay.co.uk) Received: from mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) (using TLSv1 with cipher RC4-MD5 (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 3CE5A227B; Sun, 4 Aug 2013 18:17:44 +0000 (UTC) Received: from r2d2 ([82.69.141.170]) by mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) (MDaemon PRO v10.0.4) with ESMTP id md50005328437.msg; Sun, 04 Aug 2013 19:17:43 +0100 X-Spam-Processed: mail1.multiplay.co.uk, Sun, 04 Aug 2013 19:17:43 +0100 (not processed: message from valid local sender) X-MDDKIM-Result: neutral (mail1.multiplay.co.uk) X-MDRemoteIP: 82.69.141.170 X-Return-Path: prvs=1928f37ee5=killing@multiplay.co.uk X-Envelope-From: killing@multiplay.co.uk Message-ID: <5C28F2D739D94E6B928ACD55D207665A@multiplay.co.uk> From: "Steven Hartland" To: "Martin Matuska" , References: <20130712112117.Horde.JMB3fhsU1VfZRxdV4prj5Q6@mail.vx.sk> <51FE1E1E.8080502@FreeBSD.org> Subject: Re: N-way mirror read speedup in zfsonlinux Date: Sun, 4 Aug 2013 19:18:21 +0100 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="Windows-1252"; reply-type=original Content-Transfer-Encoding: 8bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.5931 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.6157 Cc: Xin Li X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 04 Aug 2013 18:17:46 -0000 Interesting stuff. I created a little test scenario here today to run this through its passes. Its very basic, running 10 x dd's from 5 * 5GB tests to /dev/null on a pool made up of a 4 SSD's and 1 HDD in a mirror: pool: tpool state: ONLINE scan: resilvered 38.5K in 0h0m with 0 errors on Sun Aug 4 18:13:59 2013 config: NAME STATE READ WRITE CKSUM tpool ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 ada2 ONLINE 0 0 0 ada3 ONLINE 0 0 0 ada4 ONLINE 0 0 0 ada5 ONLINE 0 0 0 ada1 ONLINE 0 0 0 The results are quite telling:- == Without Patch == === SSDs & HD === Read of 51200MB using bs 1048576 took 51 seconds @ 1003 MB/s Read of 51200 MB using bs 4096 took 51 seconds @ 1003 MB/s Read of 51200 MB using bs 512 took 191 seconds @ 268 MB/s === SSDs Only === Read of 51200MB using bs 1048576 took 40 seconds @ 1280 MB/s Read of 51200MB using bs 4096 took 41 seconds @ 1248 MB/s Read of 51200MB using bs 512 took 188 seconds @ 272 MB/s == With Patch == === SSDs & HD === Read of 51200MB using bs 1048576 took 32 seconds @ 1600 MB/s Read of 51200MB using bs 4096 took 31 seconds @ 1651 MB/s Read of 51200MB using bs 512 took 184 seconds @ 278 MB/s === SSDs Only === Read of 51200MB using bs 1048576 took 28 seconds @ 1828 MB/s Read of 51200MB using bs 4096 took 29 seconds @ 1765 MB/s Read of 51200MB using bs 512 took 185 seconds @ 276 MB/s Even with only the SSD's the patched version performs noticeably better. I suspect this is down to the fact the SSD's are various makes so have slightly different IO characteristics. N.B. The bs 512 tests can be mostly discounted as it was CPU limited in dd on the 8 core test machine. Regards Steve ----- Original Message ----- From: "Martin Matuska" To: Cc: "Xin Li" ; "Steven Hartland" Sent: Sunday, August 04, 2013 10:25 AM Subject: Re: N-way mirror read speedup in zfsonlinux > Attached is a FreeBSD version of this patch for testing and comments, > including sysctl tunable: > http://people.freebsd.org/~mm/patches/zfs/vdev_mirror.c.patch > > On 2013-07-12 11:21, Martin Matuka wrote: >> Hi everyone, >> >> zfsonlinux has implemented a change in the N-way mirror device selection >> algorithm by selecting the device with the least pending I/O instead of >> random selection. They measured an increased read bandwidth increase >> up to >> 50% and IOPS increase up to 10%. >> >> this might be useful for common ZFS code and we might consider porting >> this >> to illumos and FreeBSD: >> https://github.com/zfsonlinux/zfs/issues/1461 >> https://github.com/zfsonlinux/zfs/commit/556011dbec2d10579819078559a77630fc559112 >> > > ================================================ This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmaster@multiplay.co.uk. From owner-zfs-devel@FreeBSD.ORG Sun Aug 4 18:22:50 2013 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id D104D3C5; Sun, 4 Aug 2013 18:22:50 +0000 (UTC) (envelope-from mavbsd@gmail.com) Received: from mail-ea0-x231.google.com (mail-ea0-x231.google.com [IPv6:2a00:1450:4013:c01::231]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 12C4922B4; Sun, 4 Aug 2013 18:22:49 +0000 (UTC) Received: by mail-ea0-f177.google.com with SMTP id f15so1202228eak.36 for ; Sun, 04 Aug 2013 11:22:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:content-type:content-transfer-encoding; bh=eEVnhcbwutLuDifCibT5pBORgAg4v4yw3Xnb3j+4K3w=; b=ZxxLL+uIJVvlxYwl+wDbBFkd+RAMWRTJozq3LYheJ91/sghvSWURoAlMEJcKR//bfs RRAi38e5Yx3Dg9NlYtLmaAl6Z98hdD1uDnHLG6Wjf6lzI6rEvDD5a5hh7MLZpW9jp2lw XlcBz07Pp/uaLuXqZWxvDLT68k0LCCa49WF+kw7fj3vRGJXD0y235+PB3X7Vtx2kvaYA 7vTnrsOwq0CdqN2WEFMyHFS3hpObIFs5N5WGDcSK8Zz9kcXAkHhYlLT+W98vu2nK2d4G EIlAYaRRcCEsdnjsghU1SsSpurV9y7lOjWkZCxuCNqCXtrLeoj/dPW67B1WItlTjadD7 DFuw== X-Received: by 10.15.109.135 with SMTP id cf7mr13982190eeb.0.1375640568262; Sun, 04 Aug 2013 11:22:48 -0700 (PDT) Received: from mavbook.mavhome.dp.ua ([37.229.21.195]) by mx.google.com with ESMTPSA id e1sm15943437eev.8.2013.08.04.11.22.46 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Sun, 04 Aug 2013 11:22:47 -0700 (PDT) Sender: Alexander Motin Message-ID: <51FE9BF5.2000301@FreeBSD.org> Date: Sun, 04 Aug 2013 21:22:45 +0300 From: Alexander Motin User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:17.0) Gecko/20130616 Thunderbird/17.0.6 MIME-Version: 1.0 To: Steven Hartland Subject: Re: N-way mirror read speedup in zfsonlinux References: <20130712112117.Horde.JMB3fhsU1VfZRxdV4prj5Q6@mail.vx.sk> <51FE1E1E.8080502@FreeBSD.org> <5C28F2D739D94E6B928ACD55D207665A@multiplay.co.uk> In-Reply-To: <5C28F2D739D94E6B928ACD55D207665A@multiplay.co.uk> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 8bit Cc: zfs-devel@freebsd.org, Xin Li X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 04 Aug 2013 18:22:50 -0000 On 04.08.2013 21:18, Steven Hartland wrote: > Interesting stuff. > > I created a little test scenario here today to run this through its passes. > > Its very basic, running 10 x dd's from 5 * 5GB tests to /dev/null on a > pool made up of a 4 SSD's and 1 HDD in a mirror: > > pool: tpool > state: ONLINE > scan: resilvered 38.5K in 0h0m with 0 errors on Sun Aug 4 18:13:59 2013 > config: > > NAME STATE READ WRITE CKSUM > tpool ONLINE 0 0 0 > mirror-0 ONLINE 0 0 0 > ada2 ONLINE 0 0 0 > ada3 ONLINE 0 0 0 > ada4 ONLINE 0 0 0 > ada5 ONLINE 0 0 0 > ada1 ONLINE 0 0 0 > > The results are quite telling:- > > == Without Patch == > === SSDs & HD === > Read of 51200MB using bs 1048576 took 51 seconds @ 1003 MB/s > Read of 51200 MB using bs 4096 took 51 seconds @ 1003 MB/s > Read of 51200 MB using bs 512 took 191 seconds @ 268 MB/s > > === SSDs Only === > Read of 51200MB using bs 1048576 took 40 seconds @ 1280 MB/s > Read of 51200MB using bs 4096 took 41 seconds @ 1248 MB/s > Read of 51200MB using bs 512 took 188 seconds @ 272 MB/s > > == With Patch == > === SSDs & HD === > Read of 51200MB using bs 1048576 took 32 seconds @ 1600 MB/s > Read of 51200MB using bs 4096 took 31 seconds @ 1651 MB/s > Read of 51200MB using bs 512 took 184 seconds @ 278 MB/s > > === SSDs Only === > Read of 51200MB using bs 1048576 took 28 seconds @ 1828 MB/s > Read of 51200MB using bs 4096 took 29 seconds @ 1765 MB/s > Read of 51200MB using bs 512 took 185 seconds @ 276 MB/s > > Even with only the SSD's the patched version performs > noticeably better. I suspect this is down to the fact > the SSD's are various makes so have slightly different IO > characteristics. > > N.B. The bs 512 tests can be mostly discounted as it was CPU > limited in dd on the 8 core test machine. Could you also run test with HDDs only and with different (lower) number of dd's? SSDs are much more forgiving due to lack of seek time. > ----- Original Message ----- From: "Martin Matuska" > To: > Cc: "Xin Li" ; "Steven Hartland" > Sent: Sunday, August 04, 2013 10:25 AM > Subject: Re: N-way mirror read speedup in zfsonlinux > > >> Attached is a FreeBSD version of this patch for testing and comments, >> including sysctl tunable: >> http://people.freebsd.org/~mm/patches/zfs/vdev_mirror.c.patch >> >> On 2013-07-12 11:21, Martin Matuka wrote: >>> Hi everyone, >>> >>> zfsonlinux has implemented a change in the N-way mirror device selection >>> algorithm by selecting the device with the least pending I/O instead of >>> random selection. They measured an increased read bandwidth increase >>> up to >>> 50% and IOPS increase up to 10%. >>> >>> this might be useful for common ZFS code and we might consider porting >>> this >>> to illumos and FreeBSD: >>> https://github.com/zfsonlinux/zfs/issues/1461 >>> https://github.com/zfsonlinux/zfs/commit/556011dbec2d10579819078559a77630fc559112 -- Alexander Motin From owner-zfs-devel@FreeBSD.ORG Sun Aug 4 19:02:14 2013 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 3C3AFC72; Sun, 4 Aug 2013 19:02:14 +0000 (UTC) (envelope-from mavbsd@gmail.com) Received: from mail-ee0-x229.google.com (mail-ee0-x229.google.com [IPv6:2a00:1450:4013:c00::229]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 7207F244A; Sun, 4 Aug 2013 19:02:13 +0000 (UTC) Received: by mail-ee0-f41.google.com with SMTP id d17so1222499eek.14 for ; Sun, 04 Aug 2013 12:02:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:content-type:content-transfer-encoding; bh=TsAf43vjxaZpY4Sb4aNpylGici2HPTcHABWkXeTi56s=; b=CKjSoIYok1ZL12Uh8/FAr1iBfM9oyJHaPN7BoT+xxbpIfrg5wMcAozTMmVYzFOjMuv B4OXsXzXhnG2e+m+CRuq7QbmgjHgrgfdVCQRcrV51Iq6pvf+V80XdqfoLrp6+ZRQGKnX APmEcRU7p8h3ZZemvGqPE95MafshcWgqMsEVlz9UYopOJyrOGE1cg4GNPdkr2sEK/fiw pCRcEaFZykHAy8NVkb/Klwp7p4J/KzwSG6jfXBDIBxVEpSR9x3sYN7n7qjUBlYEFxBan FnZLIBdJ7/IirUaZeLHyyIXZeIT5Bc2FyukD9kfyAUxmJu8e6JSTa6q89inG8qKiCnzc X5Sw== X-Received: by 10.14.213.4 with SMTP id z4mr14006399eeo.51.1375642931474; Sun, 04 Aug 2013 12:02:11 -0700 (PDT) Received: from mavbook.mavhome.dp.ua ([37.229.21.195]) by mx.google.com with ESMTPSA id a1sm1719033eem.1.2013.08.04.12.02.09 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Sun, 04 Aug 2013 12:02:10 -0700 (PDT) Sender: Alexander Motin Message-ID: <51FEA530.2090008@FreeBSD.org> Date: Sun, 04 Aug 2013 22:02:08 +0300 From: Alexander Motin User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:17.0) Gecko/20130616 Thunderbird/17.0.6 MIME-Version: 1.0 To: Steven Hartland Subject: Re: N-way mirror read speedup in zfsonlinux References: <20130712112117.Horde.JMB3fhsU1VfZRxdV4prj5Q6@mail.vx.sk> <51FE1E1E.8080502@FreeBSD.org> <5C28F2D739D94E6B928ACD55D207665A@multiplay.co.uk> <51FE9BF5.2000301@FreeBSD.org> In-Reply-To: <51FE9BF5.2000301@FreeBSD.org> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 8bit Cc: zfs-devel@freebsd.org, Xin Li X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 04 Aug 2013 19:02:14 -0000 On 04.08.2013 21:22, Alexander Motin wrote: > On 04.08.2013 21:18, Steven Hartland wrote: >> Interesting stuff. >> >> I created a little test scenario here today to run this through its >> passes. >> >> Its very basic, running 10 x dd's from 5 * 5GB tests to /dev/null on a >> pool made up of a 4 SSD's and 1 HDD in a mirror: >> >> pool: tpool >> state: ONLINE >> scan: resilvered 38.5K in 0h0m with 0 errors on Sun Aug 4 18:13:59 >> 2013 >> config: >> >> NAME STATE READ WRITE CKSUM >> tpool ONLINE 0 0 0 >> mirror-0 ONLINE 0 0 0 >> ada2 ONLINE 0 0 0 >> ada3 ONLINE 0 0 0 >> ada4 ONLINE 0 0 0 >> ada5 ONLINE 0 0 0 >> ada1 ONLINE 0 0 0 >> >> The results are quite telling:- >> >> == Without Patch == >> === SSDs & HD === >> Read of 51200MB using bs 1048576 took 51 seconds @ 1003 MB/s >> Read of 51200 MB using bs 4096 took 51 seconds @ 1003 MB/s >> Read of 51200 MB using bs 512 took 191 seconds @ 268 MB/s >> >> === SSDs Only === >> Read of 51200MB using bs 1048576 took 40 seconds @ 1280 MB/s >> Read of 51200MB using bs 4096 took 41 seconds @ 1248 MB/s >> Read of 51200MB using bs 512 took 188 seconds @ 272 MB/s >> >> == With Patch == >> === SSDs & HD === >> Read of 51200MB using bs 1048576 took 32 seconds @ 1600 MB/s >> Read of 51200MB using bs 4096 took 31 seconds @ 1651 MB/s >> Read of 51200MB using bs 512 took 184 seconds @ 278 MB/s >> >> === SSDs Only === >> Read of 51200MB using bs 1048576 took 28 seconds @ 1828 MB/s >> Read of 51200MB using bs 4096 took 29 seconds @ 1765 MB/s >> Read of 51200MB using bs 512 took 185 seconds @ 276 MB/s >> >> Even with only the SSD's the patched version performs >> noticeably better. I suspect this is down to the fact >> the SSD's are various makes so have slightly different IO >> characteristics. >> >> N.B. The bs 512 tests can be mostly discounted as it was CPU >> limited in dd on the 8 core test machine. > > Could you also run test with HDDs only and with different (lower) number > of dd's? SSDs are much more forgiving due to lack of seek time. I couldn't wait and did it myself with 4xHDDs in mirror: Without patch: 1xdd 360MB/s 2xdd 434MB/s 4xdd 448MB/s With patch: 1xdd 167MB/s 2xdd 310MB/s 4xdd 455MB/s So yes, while it helps with multi-threaded read, sequential low-threaded read is heavily harmed. I would not call it a win. >> ----- Original Message ----- From: "Martin Matuska" >> To: >> Cc: "Xin Li" ; "Steven Hartland" >> Sent: Sunday, August 04, 2013 10:25 AM >> Subject: Re: N-way mirror read speedup in zfsonlinux >> >> >>> Attached is a FreeBSD version of this patch for testing and comments, >>> including sysctl tunable: >>> http://people.freebsd.org/~mm/patches/zfs/vdev_mirror.c.patch >>> >>> On 2013-07-12 11:21, Martin Matuka wrote: >>>> Hi everyone, >>>> >>>> zfsonlinux has implemented a change in the N-way mirror device >>>> selection >>>> algorithm by selecting the device with the least pending I/O instead of >>>> random selection. They measured an increased read bandwidth increase >>>> up to >>>> 50% and IOPS increase up to 10%. >>>> >>>> this might be useful for common ZFS code and we might consider porting >>>> this >>>> to illumos and FreeBSD: >>>> https://github.com/zfsonlinux/zfs/issues/1461 >>>> https://github.com/zfsonlinux/zfs/commit/556011dbec2d10579819078559a77630fc559112 >>>> > > -- Alexander Motin From owner-zfs-devel@FreeBSD.ORG Mon Aug 5 11:08:18 2013 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 86545456 for ; Mon, 5 Aug 2013 11:08:18 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 5912826C0 for ; Mon, 5 Aug 2013 11:08:18 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.7/8.14.7) with ESMTP id r75B8IY4037956 for ; Mon, 5 Aug 2013 11:08:18 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.7/8.14.7/Submit) id r75B8HqZ037954 for zfs-devel@FreeBSD.org; Mon, 5 Aug 2013 11:08:17 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 5 Aug 2013 11:08:17 GMT Message-Id: <201308051108.r75B8HqZ037954@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: zfs-devel@FreeBSD.org Subject: Current problem reports assigned to zfs-devel@FreeBSD.org X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 05 Aug 2013 11:08:18 -0000 Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- s kern/178467 zfs-devel [zfs] [request] Optimized Checksum Code for ZFS o kern/178387 zfs-devel [zfs] [patch] sparse files performance improvements 2 problems total. From owner-zfs-devel@FreeBSD.ORG Tue Aug 6 15:37:48 2013 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 0CE0A2AE; Tue, 6 Aug 2013 15:37:48 +0000 (UTC) (envelope-from prvs=1930867aa0=killing@multiplay.co.uk) Received: from mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) (using TLSv1 with cipher RC4-MD5 (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 5750124B7; Tue, 6 Aug 2013 15:37:46 +0000 (UTC) Received: from r2d2 ([82.69.141.170]) by mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) (MDaemon PRO v10.0.4) with ESMTP id md50005362398.msg; Tue, 06 Aug 2013 16:37:44 +0100 X-Spam-Processed: mail1.multiplay.co.uk, Tue, 06 Aug 2013 16:37:44 +0100 (not processed: message from valid local sender) X-MDDKIM-Result: neutral (mail1.multiplay.co.uk) X-MDRemoteIP: 82.69.141.170 X-Return-Path: prvs=1930867aa0=killing@multiplay.co.uk X-Envelope-From: killing@multiplay.co.uk Message-ID: <4B747F3B634B4A859068FC8D782A7E1D@multiplay.co.uk> From: "Steven Hartland" To: "Alexander Motin" References: <20130712112117.Horde.JMB3fhsU1VfZRxdV4prj5Q6@mail.vx.sk> <51FE1E1E.8080502@FreeBSD.org> <5C28F2D739D94E6B928ACD55D207665A@multiplay.co.uk> <51FE9BF5.2000301@FreeBSD.org> <51FEA530.2090008@FreeBSD.org> Subject: Re: N-way mirror read speedup in zfsonlinux Date: Tue, 6 Aug 2013 16:37:52 +0100 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="Windows-1252"; reply-type=response Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.5931 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.6157 Cc: zfs-devel@freebsd.org, Xin Li X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Aug 2013 15:37:48 -0000 >----- Original Message ----- >From: "Alexander Motin" >On 04.08.2013 21:22, Alexander Motin wrote: >> On 04.08.2013 21:18, Steven Hartland wrote: >>> Interesting stuff. >>> >>> I created a little test scenario here today to run this through its >>> passes. >>> >>> Its very basic, running 10 x dd's from 5 * 5GB tests to /dev/null on a >>> pool made up of a 4 SSD's and 1 HDD in a mirror: >>> >>> pool: tpool >>> state: ONLINE >>> scan: resilvered 38.5K in 0h0m with 0 errors on Sun Aug 4 18:13:59 >>> 2013 >>> config: >>> >>> NAME STATE READ WRITE CKSUM >>> tpool ONLINE 0 0 0 >>> mirror-0 ONLINE 0 0 0 >>> ada2 ONLINE 0 0 0 >>> ada3 ONLINE 0 0 0 >>> ada4 ONLINE 0 0 0 >>> ada5 ONLINE 0 0 0 >>> ada1 ONLINE 0 0 0 >>> >>> The results are quite telling:- >>> >>> == Without Patch == >>> === SSDs & HD === >>> Read of 51200MB using bs 1048576 took 51 seconds @ 1003 MB/s >>> Read of 51200 MB using bs 4096 took 51 seconds @ 1003 MB/s >>> Read of 51200 MB using bs 512 took 191 seconds @ 268 MB/s >>> >>> === SSDs Only === >>> Read of 51200MB using bs 1048576 took 40 seconds @ 1280 MB/s >>> Read of 51200MB using bs 4096 took 41 seconds @ 1248 MB/s >>> Read of 51200MB using bs 512 took 188 seconds @ 272 MB/s >>> >>> == With Patch == >>> === SSDs & HD === >>> Read of 51200MB using bs 1048576 took 32 seconds @ 1600 MB/s >>> Read of 51200MB using bs 4096 took 31 seconds @ 1651 MB/s >>> Read of 51200MB using bs 512 took 184 seconds @ 278 MB/s >>> >>> === SSDs Only === >>> Read of 51200MB using bs 1048576 took 28 seconds @ 1828 MB/s >>> Read of 51200MB using bs 4096 took 29 seconds @ 1765 MB/s >>> Read of 51200MB using bs 512 took 185 seconds @ 276 MB/s >>> >>> Even with only the SSD's the patched version performs >>> noticeably better. I suspect this is down to the fact >>> the SSD's are various makes so have slightly different IO >>> characteristics. >>> >>> N.B. The bs 512 tests can be mostly discounted as it was CPU >>> limited in dd on the 8 core test machine. >> >> Could you also run test with HDDs only and with different (lower) number >> of dd's? SSDs are much more forgiving due to lack of seek time. > > I couldn't wait and did it myself with 4xHDDs in mirror: > Without patch: > 1xdd 360MB/s > 2xdd 434MB/s > 4xdd 448MB/s > > With patch: > 1xdd 167MB/s > 2xdd 310MB/s > 4xdd 455MB/s > > So yes, while it helps with multi-threaded read, sequential low-threaded > read is heavily harmed. I would not call it a win. > Hmmm, my results for two HDD's disagree with your results:- == 2 disks - Without Patch (max_pending=10) == Read of 5120MB using bs: 1048576, readers: 1, took 34 seconds @ 150 MB/s Read of 10240MB using bs: 1048576, readers: 2, took 71 seconds @ 144 MB/s Read of 25600MB using bs: 1048576, readers: 5, took 188 seconds @ 136 MB/s Read of 51200MB using bs: 1048576, readers: 10, took 188 seconds @ 272 MB/s == 2 disks - With Patch (max_pending=10) == Read of 5120MB using bs: 1048576, readers: 1, took 24 seconds @ 213 MB/s Read of 10240MB using bs: 1048576, readers: 2, took 59 seconds @ 173 MB/s Read of 25600MB using bs: 1048576, readers: 5, took 114 seconds @ 224 MB/s Read of 51200MB using bs: 1048576, readers: 10, took 168 seconds @ 304 MB/s == 2 disks - Without Patch (max_pending=100) == Read of 5120MB using bs: 1048576, readers: 1, took 37 seconds @ 138 MB/s Read of 10240MB using bs: 1048576, readers: 2, took 80 seconds @ 128 MB/s Read of 25600MB using bs: 1048576, readers: 5, took 206 seconds @ 124 MB/s Read of 51200MB using bs: 1048576, readers: 10, took 208 seconds @ 246 MB/s == 2 disks - With Patch (max_pending=100) == Read of 5120MB using bs: 1048576, readers: 1, took 24 seconds @ 213 MB/s Read of 10240MB using bs: 1048576, readers: 2, took 49 seconds @ 208 MB/s Read of 25600MB using bs: 1048576, readers: 5, took 113 seconds @ 226 MB/s Read of 51200MB using bs: 1048576, readers: 10, took 116 seconds @ 441 MB/s Visually watching gstat output shows the first disk ada1 @ 50% busy and the second disk (ada5) @ 100% most of the time without the patch, where as with they both take an even amount of the load. Note: I believe any values over ~240MB/s are due to ARC replay. Regards Steve ================================================ This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmaster@multiplay.co.uk. From owner-zfs-devel@FreeBSD.ORG Tue Aug 6 16:01:48 2013 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 213B5BDC; Tue, 6 Aug 2013 16:01:48 +0000 (UTC) (envelope-from mavbsd@gmail.com) Received: from mail-ea0-x22c.google.com (mail-ea0-x22c.google.com [IPv6:2a00:1450:4013:c01::22c]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 6DC5325B0; Tue, 6 Aug 2013 16:01:47 +0000 (UTC) Received: by mail-ea0-f172.google.com with SMTP id r16so314337ead.31 for ; Tue, 06 Aug 2013 09:01:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:content-type:content-transfer-encoding; bh=nYOHYwVxpXRqVcsgNeFHCrzotP9yOIoaMUJtGXPciAc=; b=iHukv2bAO0XFPNBALZa02e07B3z5A+NNFi5YTrtSzjqxzkmsGpy9FsRtB+8BA/Sw34 XbglKnWbO93zKHl4IKD0AeBuVaG9VBQKytzJiqS7eYprwb0jFhkUK5Ilxxo0m3yJ2ELD 8F+ndvAFI7tW5oDZVINgnk5BCIC8hhs12dIhVdA93hdlrv3FbbogFoK10QDAmccSSL5e aMrm8zEXzEE0ChkgwT8uKryGWsbSXSZwAaND5O8dQiy+fdhLhBn6vuCLYi3RxxJPaPgS 8/Rmc29ZphIS6xfRzCj1reZGgM1wDDm0jpFkejIdVMwrDaD6olhzoRPtk+w0L6jnJZL6 HwUA== X-Received: by 10.15.22.138 with SMTP id f10mr1729625eeu.126.1375804905591; Tue, 06 Aug 2013 09:01:45 -0700 (PDT) Received: from mavbook.mavhome.dp.ua ([37.229.21.195]) by mx.google.com with ESMTPSA id k7sm3087313eeg.13.2013.08.06.09.01.43 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 06 Aug 2013 09:01:44 -0700 (PDT) Sender: Alexander Motin Message-ID: <52011DE6.8090708@FreeBSD.org> Date: Tue, 06 Aug 2013 19:01:42 +0300 From: Alexander Motin User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:17.0) Gecko/20130616 Thunderbird/17.0.6 MIME-Version: 1.0 To: Steven Hartland Subject: Re: N-way mirror read speedup in zfsonlinux References: <20130712112117.Horde.JMB3fhsU1VfZRxdV4prj5Q6@mail.vx.sk> <51FE1E1E.8080502@FreeBSD.org> <5C28F2D739D94E6B928ACD55D207665A@multiplay.co.uk> <51FE9BF5.2000301@FreeBSD.org> <51FEA530.2090008@FreeBSD.org> <4B747F3B634B4A859068FC8D782A7E1D@multiplay.co.uk> In-Reply-To: <4B747F3B634B4A859068FC8D782A7E1D@multiplay.co.uk> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit Cc: zfs-devel@freebsd.org, Xin Li X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Aug 2013 16:01:48 -0000 On 06.08.2013 18:37, Steven Hartland wrote: >> ----- Original Message ----- From: "Alexander Motin" >> On 04.08.2013 21:22, Alexander Motin wrote: >>> On 04.08.2013 21:18, Steven Hartland wrote: >>>> Interesting stuff. >>>> >>>> I created a little test scenario here today to run this through its >>>> passes. >>>> >>>> Its very basic, running 10 x dd's from 5 * 5GB tests to /dev/null on a >>>> pool made up of a 4 SSD's and 1 HDD in a mirror: >>>> >>>> pool: tpool >>>> state: ONLINE >>>> scan: resilvered 38.5K in 0h0m with 0 errors on Sun Aug 4 18:13:59 >>>> 2013 >>>> config: >>>> >>>> NAME STATE READ WRITE CKSUM >>>> tpool ONLINE 0 0 0 >>>> mirror-0 ONLINE 0 0 0 >>>> ada2 ONLINE 0 0 0 >>>> ada3 ONLINE 0 0 0 >>>> ada4 ONLINE 0 0 0 >>>> ada5 ONLINE 0 0 0 >>>> ada1 ONLINE 0 0 0 >>>> >>>> The results are quite telling:- >>>> >>>> == Without Patch == >>>> === SSDs & HD === >>>> Read of 51200MB using bs 1048576 took 51 seconds @ 1003 MB/s >>>> Read of 51200 MB using bs 4096 took 51 seconds @ 1003 MB/s >>>> Read of 51200 MB using bs 512 took 191 seconds @ 268 MB/s >>>> >>>> === SSDs Only === >>>> Read of 51200MB using bs 1048576 took 40 seconds @ 1280 MB/s >>>> Read of 51200MB using bs 4096 took 41 seconds @ 1248 MB/s >>>> Read of 51200MB using bs 512 took 188 seconds @ 272 MB/s >>>> >>>> == With Patch == >>>> === SSDs & HD === >>>> Read of 51200MB using bs 1048576 took 32 seconds @ 1600 MB/s >>>> Read of 51200MB using bs 4096 took 31 seconds @ 1651 MB/s >>>> Read of 51200MB using bs 512 took 184 seconds @ 278 MB/s >>>> >>>> === SSDs Only === >>>> Read of 51200MB using bs 1048576 took 28 seconds @ 1828 MB/s >>>> Read of 51200MB using bs 4096 took 29 seconds @ 1765 MB/s >>>> Read of 51200MB using bs 512 took 185 seconds @ 276 MB/s >>>> >>>> Even with only the SSD's the patched version performs >>>> noticeably better. I suspect this is down to the fact >>>> the SSD's are various makes so have slightly different IO >>>> characteristics. >>>> >>>> N.B. The bs 512 tests can be mostly discounted as it was CPU >>>> limited in dd on the 8 core test machine. >>> >>> Could you also run test with HDDs only and with different (lower) number >>> of dd's? SSDs are much more forgiving due to lack of seek time. >> >> I couldn't wait and did it myself with 4xHDDs in mirror: >> Without patch: >> 1xdd 360MB/s >> 2xdd 434MB/s >> 4xdd 448MB/s >> >> With patch: >> 1xdd 167MB/s >> 2xdd 310MB/s >> 4xdd 455MB/s >> >> So yes, while it helps with multi-threaded read, sequential >> low-threaded read is heavily harmed. I would not call it a win. >> > > Hmmm, my results for two HDD's disagree with your results:- > > == 2 disks - Without Patch (max_pending=10) == > Read of 5120MB using bs: 1048576, readers: 1, took 34 seconds @ 150 MB/s > Read of 10240MB using bs: 1048576, readers: 2, took 71 seconds @ 144 MB/s > Read of 25600MB using bs: 1048576, readers: 5, took 188 seconds @ 136 MB/s > Read of 51200MB using bs: 1048576, readers: 10, took 188 seconds @ 272 MB/s > > == 2 disks - With Patch (max_pending=10) == > Read of 5120MB using bs: 1048576, readers: 1, took 24 seconds @ 213 MB/s > Read of 10240MB using bs: 1048576, readers: 2, took 59 seconds @ 173 MB/s > Read of 25600MB using bs: 1048576, readers: 5, took 114 seconds @ 224 MB/s > Read of 51200MB using bs: 1048576, readers: 10, took 168 seconds @ 304 MB/s > > == 2 disks - Without Patch (max_pending=100) == > Read of 5120MB using bs: 1048576, readers: 1, took 37 seconds @ 138 MB/s > Read of 10240MB using bs: 1048576, readers: 2, took 80 seconds @ 128 MB/s > Read of 25600MB using bs: 1048576, readers: 5, took 206 seconds @ 124 MB/s > Read of 51200MB using bs: 1048576, readers: 10, took 208 seconds @ 246 MB/s > > == 2 disks - With Patch (max_pending=100) == > Read of 5120MB using bs: 1048576, readers: 1, took 24 seconds @ 213 MB/s > Read of 10240MB using bs: 1048576, readers: 2, took 49 seconds @ 208 MB/s > Read of 25600MB using bs: 1048576, readers: 5, took 113 seconds @ 226 MB/s > Read of 51200MB using bs: 1048576, readers: 10, took 116 seconds @ 441 MB/s > > Visually watching gstat output shows the first disk ada1 @ 50% busy and > the second disk (ada5) @ 100% most of the time without the patch, where > as with > they both take an even amount of the load. > > Note: I believe any values over ~240MB/s are due to ARC replay. Please show your test script so I could try to reproduce it. Experiment with more then 2 disks could be made if we are talking about N. Also aside of practical results it would be good to get logical explanation with some answer to my counterarguments. -- Alexander Motin From owner-zfs-devel@FreeBSD.ORG Mon Aug 12 11:08:20 2013 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id C92E47D for ; Mon, 12 Aug 2013 11:08:20 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 9C5892262 for ; Mon, 12 Aug 2013 11:08:20 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.7/8.14.7) with ESMTP id r7CB8Krd086944 for ; Mon, 12 Aug 2013 11:08:20 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.7/8.14.7/Submit) id r7CB8Knu086942 for zfs-devel@FreeBSD.org; Mon, 12 Aug 2013 11:08:20 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 12 Aug 2013 11:08:20 GMT Message-Id: <201308121108.r7CB8Knu086942@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: zfs-devel@FreeBSD.org Subject: Current problem reports assigned to zfs-devel@FreeBSD.org X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Aug 2013 11:08:20 -0000 Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- s kern/178467 zfs-devel [zfs] [request] Optimized Checksum Code for ZFS o kern/178387 zfs-devel [zfs] [patch] sparse files performance improvements 2 problems total. From owner-zfs-devel@FreeBSD.ORG Wed Aug 14 13:50:10 2013 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id AE927B2E; Wed, 14 Aug 2013 13:50:10 +0000 (UTC) (envelope-from prvs=19381d5a45=killing@multiplay.co.uk) Received: from mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) (using TLSv1 with cipher RC4-MD5 (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id B87D22C2B; Wed, 14 Aug 2013 13:50:09 +0000 (UTC) Received: from r2d2 ([82.69.141.170]) by mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) (MDaemon PRO v10.0.4) with ESMTP id md50005491846.msg; Wed, 14 Aug 2013 14:50:05 +0100 X-Spam-Processed: mail1.multiplay.co.uk, Wed, 14 Aug 2013 14:50:05 +0100 (not processed: message from valid local sender) X-MDDKIM-Result: neutral (mail1.multiplay.co.uk) X-MDRemoteIP: 82.69.141.170 X-Return-Path: prvs=19381d5a45=killing@multiplay.co.uk X-Envelope-From: killing@multiplay.co.uk Message-ID: <4DA6B66E61374BE68C895C8B9A93BCB7@multiplay.co.uk> From: "Steven Hartland" To: "Alexander Motin" References: <20130712112117.Horde.JMB3fhsU1VfZRxdV4prj5Q6@mail.vx.sk> <51FE1E1E.8080502@FreeBSD.org> <5C28F2D739D94E6B928ACD55D207665A@multiplay.co.uk> <51FE9BF5.2000301@FreeBSD.org> <51FEA530.2090008@FreeBSD.org> <4B747F3B634B4A859068FC8D782A7E1D@multiplay.co.uk> <52011DE6.8090708@FreeBSD.org> Subject: Re: N-way mirror read speedup in zfsonlinux Date: Wed, 14 Aug 2013 14:50:22 +0100 MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="----=_NextPart_000_0D53_01CE98FD.9AA18EB0" X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.5931 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.6157 Cc: zfs-devel@freebsd.org, Xin Li X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Aug 2013 13:50:10 -0000 This is a multi-part message in MIME format. ------=_NextPart_000_0D53_01CE98FD.9AA18EB0 Content-Type: text/plain; format=flowed; charset="iso-8859-1"; reply-type=response Content-Transfer-Encoding: 7bit So based on mav's comments I've created a new version of the load balanced patch which takes into account the last IO's locality on the disk. I've added detection of non-rotating media, that also triggers the locality bonus, which significantly improves mixed HDD and SSD mirrors. I've also removed the time based switching as this was flawed for N-Way mirrors where N != 2, and didn't see to provide any benefit either. One question I did have; is it necessary to use protect the call to avl_numnodes with the vq_lock mutex, as this seems like it may not actually be needed given its not iterating or modifying the avl? My tests show this version provides a significant performance increases on both from the original strip balancing as well as the zfsonlinux load balancing version; resulting in up to 3 x the read rates on an 3 way mirror with 2 x HDD's and 1 x SSD. The patch is attached, comments? Here is my full set of test results: == Setup == 3 Way Mirror with 2 x HD's and 1 x SSD === Prefetch Disabled == ==== Load Balanced (locality) ==== Read 15360MB using bs: 1048576, readers: 3, took 54 seconds @ 284 MB/s Read 30720MB using bs: 1048576, readers: 6, took 77 seconds @ 398 MB/s Read 46080MB using bs: 1048576, readers: 9, took 89 seconds @ 517 MB/s ==== Stripe Balanced (default) ==== Read 15360MB using bs: 1048576, readers: 3, took 161 seconds @ 95 MB/s ==== Load Balanced (zfslinux) ==== Read 15360MB using bs: 1048576, readers: 3, took 297 seconds @ 51 MB/s === Prefetch Enabled === ==== Load Balanced (locality) ==== Read 15360MB using bs: 1048576, readers: 3, took 48 seconds @ 320 MB/s ==== Stripe Balanced (default) ==== Read 15360MB using bs: 1048576, readers: 3, took 91 seconds @ 168 MB/s ==== Load Balanced (zfslinux) ==== Read 15360MB using bs: 1048576, readers: 3, took 108 seconds @ 142 MB/s == Setup == 2 Way Mirror with 2 x HD's === Prefetch Disabled == ==== Load Balanced (locality) ==== Read 10240MB using bs: 1048576, readers: 2, took 131 seconds @ 78 MB/s ==== Stripe Balanced (default) ==== Read 10240MB using bs: 1048576, readers: 2, took 160 seconds @ 64 MB/s ==== Load Balanced (zfslinux) ==== Read 10240MB using bs: 1048576, readers: 2, took 207 seconds @ 49 MB/s === Prefetch Enabled === ==== Load Balanced (locality) ==== Read 10240MB using bs: 1048576, readers: 2, took 85 seconds @ 120 MB/s ==== Stripe Balanced (default) ==== Read 10240MB using bs: 1048576, readers: 2, took 109 seconds @ 93 MB/s ==== Load Balanced (zfslinux) ==== Read 10240MB using bs: 1048576, readers: 2, took 94 seconds @ 108 MB/s Regards Steve ================================================ This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmaster@multiplay.co.uk. ------=_NextPart_000_0D53_01CE98FD.9AA18EB0 Content-Type: application/octet-stream; name="zfs-mirror-load.patch" Content-Transfer-Encoding: quoted-printable Content-Disposition: attachment; filename="zfs-mirror-load.patch" Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/vdev_impl.h=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/vdev_impl.h = (revision 253996)=0A= +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/vdev_impl.h = (working copy)=0A= @@ -106,6 +106,7 @@=0A= avl_tree_t vq_pending_tree;=0A= hrtime_t vq_io_complete_ts;=0A= kmutex_t vq_lock;=0A= + uint64_t vq_lastoffset;=0A= };=0A= =0A= /*=0A= @@ -199,7 +200,8 @@=0A= spa_aux_vdev_t *vdev_aux; /* for l2cache vdevs */=0A= zio_t *vdev_probe_zio; /* root of current probe */=0A= vdev_aux_t vdev_label_aux; /* on-disk aux state */=0A= - struct trim_map *vdev_trimmap;=0A= + struct trim_map *vdev_trimmap; /* map on outstanding trims */ =0A= + boolean_t vdev_rotating; /* backed by rotating media */=0A= =0A= /*=0A= * For DTrace to work in userland (libzpool) context, these fields must=0A= Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_geom.c=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_geom.c (revision = 253996)=0A= +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_geom.c (working = copy)=0A= @@ -42,9 +42,11 @@=0A= * Virtual device vector for GEOM.=0A= */=0A= =0A= +static g_attrchanged_t vdev_geom_attrchanged;=0A= struct g_class zfs_vdev_class =3D {=0A= .name =3D "ZFS::VDEV",=0A= .version =3D G_VERSION,=0A= + .attrchanged =3D vdev_geom_attrchanged,=0A= };=0A= =0A= DECLARE_GEOM_CLASS(zfs_vdev_class, zfs_vdev);=0A= @@ -62,6 +64,33 @@=0A= &vdev_geom_bio_delete_disable, 0, "Disable BIO_DELETE");=0A= =0A= static void=0A= +vdev_geom_set_rotating(vdev_t *vd, struct g_consumer *cp)=0A= +{ =0A= + int error, rotating;=0A= +=0A= + error =3D g_getattr("GEOM::rotating", cp, &rotating);=0A= + if (error =3D=3D 0)=0A= + vd->vdev_rotating =3D (rotating =3D=3D 1);=0A= + else=0A= + vd->vdev_rotating =3D B_TRUE;=0A= +}=0A= +=0A= +static void=0A= +vdev_geom_attrchanged(struct g_consumer *cp, const char *attr)=0A= +{=0A= + vdev_t *vd;=0A= +=0A= + vd =3D cp->private;=0A= + if (vd =3D=3D NULL)=0A= + return;=0A= +=0A= + if (strcmp(attr, "GEOM::rotating") !=3D 0)=0A= + return;=0A= +=0A= + vdev_geom_set_rotating(vd, cp);=0A= +}=0A= +=0A= +static void=0A= vdev_geom_orphan(struct g_consumer *cp)=0A= {=0A= vdev_t *vd;=0A= @@ -678,6 +707,8 @@=0A= vd->vdev_physpath =3D kmem_alloc(bufsize, KM_SLEEP);=0A= snprintf(vd->vdev_physpath, bufsize, "/dev/%s", pp->name);=0A= =0A= + vdev_geom_set_rotating(vd, cp);=0A= +=0A= return (0);=0A= }=0A= =0A= Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_mirror.c=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_mirror.c = (revision 253996)=0A= +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_mirror.c = (working copy)=0A= @@ -33,6 +33,8 @@=0A= #include =0A= #include =0A= =0A= +SYSCTL_DECL(_vfs_zfs_vdev);=0A= +=0A= /*=0A= * Virtual device vector for mirroring.=0A= */=0A= @@ -41,6 +43,7 @@=0A= vdev_t *mc_vd;=0A= uint64_t mc_offset;=0A= int mc_error;=0A= + int mc_load;=0A= uint8_t mc_tried;=0A= uint8_t mc_skipped;=0A= uint8_t mc_speculative;=0A= @@ -54,7 +57,26 @@=0A= mirror_child_t mm_child[1];=0A= } mirror_map_t;=0A= =0A= +int zfs_vdev_mirror_locality_track_size =3D 1 * 1024 * 1024;=0A= +TUNABLE_INT("vfs.zfs.vdev.mirror_locality_track_size",=0A= + &zfs_vdev_mirror_locality_track_size);=0A= +SYSCTL_INT(_vfs_zfs_vdev, OID_AUTO, mirror_locality_track_size, = CTLFLAG_RW,=0A= + &zfs_vdev_mirror_locality_track_size, 0, "Number of bytes that a = matching "=0A= + "IO pending for a vdev will trigger preference for that vdev");=0A= +=0A= +int zfs_vdev_mirror_locality_bonus =3D 5;=0A= +TUNABLE_INT("vfs.zfs.vdev.mirror_locality_bonus",=0A= + &zfs_vdev_mirror_locality_bonus);=0A= +SYSCTL_INT(_vfs_zfs_vdev, OID_AUTO, mirror_locality_bonus, CTLFLAG_RW,=0A= + &zfs_vdev_mirror_locality_bonus, 0,=0A= + "Load bonus driver applied to IO's based on locality and to all = IO's"=0A= + "for vdevs based on non-rotating media");=0A= +=0A= int vdev_mirror_shift =3D 21;=0A= +int zfs_mirror_by_load =3D 1;=0A= +TUNABLE_INT("vfs.zfs.vdev.mirror_by_load", &zfs_mirror_by_load);=0A= +SYSCTL_INT(_vfs_zfs_vdev, OID_AUTO, mirror_by_load, CTLFLAG_RW,=0A= + &zfs_mirror_by_load, 0, "Balance mirror reads based vdev load");=0A= =0A= static void=0A= vdev_mirror_map_free(zio_t *zio)=0A= @@ -69,6 +91,41 @@=0A= zio_vsd_default_cksum_report=0A= };=0A= =0A= +static int=0A= +vdev_mirror_load(vdev_t *vd, uint64_t zio_offset)=0A= +{=0A= + vdev_queue_t *vq;=0A= + int load;=0A= +=0A= + if (!vdev_readable(vd))=0A= + return (INT_MAX);=0A= +=0A= + vq =3D &vd->vdev_queue;=0A= +=0A= + /* Baseline load based on pending queue length */=0A= + mutex_enter(&vq->vq_lock);=0A= + load =3D avl_numnodes(&vq->vq_pending_tree);=0A= + mutex_exit(&vq->vq_lock);=0A= +=0A= + /*=0A= + * Apply the full locality bonus for IO's which are=0A= + * sequential to the last IO queued or if the media is=0A= + * non-rotating as there is locality penalty (seek time).=0A= + */ =0A= + if (vq->vq_lastoffset =3D=3D zio_offset || !vd->vdev_rotating)=0A= + return (load - zfs_vdev_mirror_locality_bonus);=0A= +=0A= + /*=0A= + * Apply half the locality bonus for IO's within track size=0A= + * bytes of the last queued IO.=0A= + */=0A= + if (ABS(vq->vq_lastoffset - zio_offset) <=0A= + zfs_vdev_mirror_locality_track_size)=0A= + return (load - (zfs_vdev_mirror_locality_bonus / 2));=0A= +=0A= + return (load);=0A= +}=0A= +=0A= static mirror_map_t *=0A= vdev_mirror_map_alloc(zio_t *zio)=0A= {=0A= @@ -108,6 +165,10 @@=0A= mc->mc_offset =3D DVA_GET_OFFSET(&dva[c]);=0A= }=0A= } else {=0A= + int lowest_load =3D INT_MAX;=0A= + int lowest_child =3D 0;=0A= + uint64_t zio_offset =3D zio->io_offset;=0A= +=0A= c =3D vd->vdev_children;=0A= =0A= mm =3D kmem_zalloc(offsetof(mirror_map_t, mm_child[c]), KM_SLEEP);=0A= @@ -121,8 +182,23 @@=0A= for (c =3D 0; c < mm->mm_children; c++) {=0A= mc =3D &mm->mm_child[c];=0A= mc->mc_vd =3D vd->vdev_child[c];=0A= - mc->mc_offset =3D zio->io_offset;=0A= + mc->mc_offset =3D zio_offset;=0A= +=0A= + if (zfs_mirror_by_load =3D=3D 0 || mm->mm_replacing)=0A= + continue;=0A= +=0A= + mc->mc_load =3D vdev_mirror_load(mc->mc_vd, zio_offset);=0A= + if (mc->mc_load < lowest_load) {=0A= + lowest_load =3D mc->mc_load;=0A= + lowest_child =3D c;=0A= + }=0A= }=0A= +=0A= + if (zfs_mirror_by_load !=3D 0) {=0A= + mm->mm_preferred =3D lowest_child;=0A= + vd->vdev_child[lowest_child]->vdev_queue.vq_lastoffset =3D=0A= + zio_offset + zio->io_size;=0A= + }=0A= }=0A= =0A= zio->io_vsd =3D mm;=0A= Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c = (revision 253996)=0A= +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c (working = copy)=0A= @@ -155,6 +155,8 @@=0A= =0A= avl_create(&vq->vq_pending_tree, vdev_queue_offset_compare,=0A= sizeof (zio_t), offsetof(struct zio, io_offset_node));=0A= +=0A= + vq->vq_lastoffset =3D 0;=0A= }=0A= =0A= void=0A= Index: sys/geom/geom_disk.c=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/geom/geom_disk.c (revision 253996)=0A= +++ sys/geom/geom_disk.c (working copy)=0A= @@ -392,6 +392,9 @@=0A= g_disk_kerneldump(bp, dp);=0A= else if (!strcmp(bp->bio_attribute, "GEOM::setstate"))=0A= g_disk_setstate(bp, sc);=0A= + else if (g_handleattr_int(bp, "GEOM::rotating",=0A= + (dp->d_nonrotating =3D=3D 0)))=0A= + break;=0A= else =0A= error =3D ENOIOCTL;=0A= break;=0A= Index: sys/geom/geom_disk.h=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/geom/geom_disk.h (revision 253996)=0A= +++ sys/geom/geom_disk.h (working copy)=0A= @@ -97,6 +97,7 @@=0A= uint16_t d_hba_device;=0A= uint16_t d_hba_subvendor;=0A= uint16_t d_hba_subdevice;=0A= + int d_nonrotating;=0A= =0A= /* Fields private to the driver */=0A= void *d_drv1;=0A= @@ -121,7 +122,8 @@=0A= #define DISK_VERSION_01 0x5856105a=0A= #define DISK_VERSION_02 0x5856105b=0A= #define DISK_VERSION_03 0x5856105c=0A= -#define DISK_VERSION DISK_VERSION_03=0A= +#define DISK_VERSION_04 0x5856105d=0A= +#define DISK_VERSION DISK_VERSION_04=0A= =0A= #endif /* _KERNEL */=0A= #endif /* _GEOM_GEOM_DISK_H_ */=0A= Index: sys/cam/ata/ata_da.c=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/cam/ata/ata_da.c (revision 253996)=0A= +++ sys/cam/ata/ata_da.c (working copy)=0A= @@ -1200,13 +1224,14 @@=0A= snprintf(announce_buf, sizeof(announce_buf),=0A= "kern.cam.ada.%d.write_cache", periph->unit_number);=0A= TUNABLE_INT_FETCH(announce_buf, &softc->write_cache);=0A= + adagetparams(periph, cgd);=0A= + softc->disk =3D disk_alloc();=0A= /* Disable queue sorting for non-rotational media by default. */=0A= - if (cgd->ident_data.media_rotation_rate =3D=3D 1)=0A= + if (cgd->ident_data.media_rotation_rate =3D=3D 1) {=0A= softc->sort_io_queue =3D 0;=0A= - else=0A= + softc->disk->d_nonrotating =3D 1;=0A= + } else=0A= softc->sort_io_queue =3D -1;=0A= - adagetparams(periph, cgd);=0A= - softc->disk =3D disk_alloc();=0A= softc->disk->d_devstat =3D devstat_new_entry(periph->periph_name,=0A= periph->unit_number, softc->params.secsize,=0A= DEVSTAT_ALL_SUPPORTED,=0A= Index: sys/cam/scsi/scsi_da.c=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/cam/scsi/scsi_da.c (revision 253996)=0A= +++ sys/cam/scsi/scsi_da.c (working copy)=0A= @@ -3362,8 +3386,12 @@=0A= * by default.=0A= */=0A= if (scsi_2btoul(bdc->medium_rotation_rate) =3D=3D=0A= - SVPD_BDC_RATE_NONE_ROTATING)=0A= + SVPD_BDC_RATE_NONE_ROTATING) {=0A= softc->sort_io_queue =3D 0;=0A= + softc->disk->d_nonrotating =3D 1;=0A= + disk_attr_changed(softc->disk,=0A= + "GEOM::rotating", M_NOWAIT);=0A= + }=0A= } else {=0A= int error;=0A= error =3D daerror(done_ccb, CAM_RETRY_SELTO,=0A= @@ -3412,8 +3440,12 @@=0A= * Disable queue sorting for non-rotational media=0A= * by default.=0A= */=0A= - if (ata_params->media_rotation_rate =3D=3D 1)=0A= + if (ata_params->media_rotation_rate =3D=3D 1) {=0A= softc->sort_io_queue =3D 0;=0A= + softc->disk->d_nonrotating =3D 1;=0A= + disk_attr_changed(softc->disk,=0A= + "GEOM::rotating", M_NOWAIT);=0A= + }=0A= } else {=0A= int error;=0A= error =3D daerror(done_ccb, CAM_RETRY_SELTO,=0A= ------=_NextPart_000_0D53_01CE98FD.9AA18EB0-- From owner-zfs-devel@FreeBSD.ORG Wed Aug 14 23:02:37 2013 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 9D099BF8 for ; Wed, 14 Aug 2013 23:02:37 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: from mail-pb0-x22e.google.com (mail-pb0-x22e.google.com [IPv6:2607:f8b0:400e:c01::22e]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 6E4B52D5B for ; Wed, 14 Aug 2013 23:02:37 +0000 (UTC) Received: by mail-pb0-f46.google.com with SMTP id rq2so74609pbb.33 for ; Wed, 14 Aug 2013 16:02:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=delphix.com; s=google; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=8jL0S1+j6XJr1+DcSo0BvJg+epRrAq9O3Z5UJAe6EZc=; b=QdmitExsoQfkt8rK/D4Mb798aoMWufDwQlDVgJaumE8OknH8sCpfFQw0Doy4eYDqgS 4k2KC6wlKvgRqGoXafv0bpsBa9l814+TVcM2f+oLswKVor0CNavOvcCQYz9QZBDQYqt3 pnVsAVEfOvlyl4syzDQ7hg9QiTyX0dBptEtSU= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=8jL0S1+j6XJr1+DcSo0BvJg+epRrAq9O3Z5UJAe6EZc=; b=ELfXNldnh2+m7XBowr2GGlDEDiHah7mceefD/sYlAc2Ccl7Zvi6wAVtfeHSOpisFWL /hZ7YoQUPPa17t/Kar641w/9tIg2YmNMrMiGZNMvKj3RZisiKI5mc4uY8SaYEeg00beA 5jxt/vt9DIa1H74osbthbecQaRHIhn6PoLuYUPnWebgg5vKK7rzbOGiO6A4nxYs0Dg8+ zzQ6VfTRWicqj7cFgOyN4J9Sl0nEkz/c4Nfy5PWRvkUxc6aWGPfLg0flJ1MFF3JOv4ur aFwWSeRENP83TH+UpMuQshB1A/5nd2nXXepXhZKEWvfboPWHbA49tkknB/lbrvvag6jT DqQA== X-Gm-Message-State: ALoCoQmqMUiY7ukeOyWPWLLk7FnPkwDdznpLMJdq2BrkmoUbiPRKU+RDmaH6r59iFNt/0ICgqmTd MIME-Version: 1.0 X-Received: by 10.66.2.164 with SMTP id 4mr12522685pav.55.1376521356908; Wed, 14 Aug 2013 16:02:36 -0700 (PDT) Received: by 10.70.132.66 with HTTP; Wed, 14 Aug 2013 16:02:36 -0700 (PDT) In-Reply-To: <4DA6B66E61374BE68C895C8B9A93BCB7@multiplay.co.uk> References: <20130712112117.Horde.JMB3fhsU1VfZRxdV4prj5Q6@mail.vx.sk> <51FE1E1E.8080502@FreeBSD.org> <5C28F2D739D94E6B928ACD55D207665A@multiplay.co.uk> <51FE9BF5.2000301@FreeBSD.org> <51FEA530.2090008@FreeBSD.org> <4B747F3B634B4A859068FC8D782A7E1D@multiplay.co.uk> <52011DE6.8090708@FreeBSD.org> <4DA6B66E61374BE68C895C8B9A93BCB7@multiplay.co.uk> Date: Wed, 14 Aug 2013 16:02:36 -0700 Message-ID: Subject: Re: N-way mirror read speedup in zfsonlinux From: Matthew Ahrens To: Steven Hartland Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: Xin Li , zfs-devel@freebsd.org X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Aug 2013 23:02:37 -0000 This patch looks reasonable to me. I shared Alexander's skepticism of the time-based metric. Just a couple of specific notes: I believe that mm_preferred is only used for reads. You might not bother with the load balancing logic if this is not a read. You might add accessor functions for avl_numnodes(&vq->vq_pending_tree) and vq_lastoffset, rather than having vdev_disk.c reach into the queue implementation. It looks like technically you could get away without vq_lock for avl_numnodes(), as it's just loading a "long". But that is really relying on the implementation details. More significantly, you have no locking for vq_lastoffset. At a minimum, we should have some comments explaining that this is OK. It seems like it could get the wrong result on 32-bit systems (where load/store of 64 bit values is not atomic), but I guess that's OK since it will only cause a tiny performance impact. zfs_mirror_by_load -- could this be removed in favor of setting zfs_vdev_mirror_locality_bonus = 0 to turn off the bonus? If not, seems like it should at least be a boolean_t. vdev_mirror_load() -- if the vdev is not rotating, you always apply the locality bonus. It was slightly difficult to follow the comment and logic around this. Would a "seek penalty" which is applied if (rotating && lastoffset != offset) be easier to understand? --matt On Wed, Aug 14, 2013 at 6:50 AM, Steven Hartland wrote: > So based on mav's comments I've created a new version of the load balanced > patch which takes into account the last IO's locality on the disk. > > I've added detection of non-rotating media, that also triggers the locality > bonus, which significantly improves mixed HDD and SSD mirrors. > > I've also removed the time based switching as this was flawed for N-Way > mirrors where N != 2, and didn't see to provide any benefit either. > > One question I did have; is it necessary to use protect the call to > avl_numnodes with the vq_lock mutex, as this seems like it may not > actually be needed given its not iterating or modifying the avl? > > My tests show this version provides a significant performance increases > on both from the original strip balancing as well as the zfsonlinux load > balancing version; resulting in up to 3 x the read rates on an 3 way > mirror with 2 x HDD's and 1 x SSD. > > The patch is attached, comments? > > Here is my full set of test results: > > == Setup == > 3 Way Mirror with 2 x HD's and 1 x SSD > > === Prefetch Disabled == > ==== Load Balanced (locality) ==== > Read 15360MB using bs: 1048576, readers: 3, took 54 seconds @ 284 MB/s > Read 30720MB using bs: 1048576, readers: 6, took 77 seconds @ 398 MB/s > Read 46080MB using bs: 1048576, readers: 9, took 89 seconds @ 517 MB/s > > ==== Stripe Balanced (default) ==== > Read 15360MB using bs: 1048576, readers: 3, took 161 seconds @ 95 MB/s > > ==== Load Balanced (zfslinux) ==== > Read 15360MB using bs: 1048576, readers: 3, took 297 seconds @ 51 MB/s > > === Prefetch Enabled === > ==== Load Balanced (locality) ==== > Read 15360MB using bs: 1048576, readers: 3, took 48 seconds @ 320 MB/s > > ==== Stripe Balanced (default) ==== > Read 15360MB using bs: 1048576, readers: 3, took 91 seconds @ 168 MB/s > > ==== Load Balanced (zfslinux) ==== > Read 15360MB using bs: 1048576, readers: 3, took 108 seconds @ 142 MB/s > > == Setup == > 2 Way Mirror with 2 x HD's > > === Prefetch Disabled == > ==== Load Balanced (locality) ==== > Read 10240MB using bs: 1048576, readers: 2, took 131 seconds @ 78 MB/s > > ==== Stripe Balanced (default) ==== > Read 10240MB using bs: 1048576, readers: 2, took 160 seconds @ 64 MB/s > > ==== Load Balanced (zfslinux) ==== > Read 10240MB using bs: 1048576, readers: 2, took 207 seconds @ 49 MB/s > > === Prefetch Enabled === > ==== Load Balanced (locality) ==== > Read 10240MB using bs: 1048576, readers: 2, took 85 seconds @ 120 MB/s > > ==== Stripe Balanced (default) ==== > Read 10240MB using bs: 1048576, readers: 2, took 109 seconds @ 93 MB/s > > ==== Load Balanced (zfslinux) ==== > Read 10240MB using bs: 1048576, readers: 2, took 94 seconds @ 108 MB/s > > > Regards > Steve > > ==============================**================== > This e.mail is private and confidential between Multiplay (UK) Ltd. and > the person or entity to whom it is addressed. In the event of misdirection, > the recipient is prohibited from using, copying, printing or otherwise > disseminating it or any information contained in it. > In the event of misdirection, illegible or incomplete transmission please > telephone +44 845 868 1337 > or return the E.mail to postmaster@multiplay.co.uk. > > _______________________________________________ > zfs-devel@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/zfs-devel > To unsubscribe, send any mail to "zfs-devel-unsubscribe@freebsd.org" > From owner-zfs-devel@FreeBSD.ORG Thu Aug 15 02:47:57 2013 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 5DC2C31B; Thu, 15 Aug 2013 02:47:57 +0000 (UTC) (envelope-from prvs=19393a69e5=killing@multiplay.co.uk) Received: from mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) (using TLSv1 with cipher RC4-MD5 (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 61F8B2C20; Thu, 15 Aug 2013 02:47:55 +0000 (UTC) Received: from r2d2 ([82.69.141.170]) by mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) (MDaemon PRO v10.0.4) with ESMTP id md50005502982.msg; Thu, 15 Aug 2013 03:47:51 +0100 X-Spam-Processed: mail1.multiplay.co.uk, Thu, 15 Aug 2013 03:47:51 +0100 (not processed: message from valid local sender) X-MDDKIM-Result: neutral (mail1.multiplay.co.uk) X-MDRemoteIP: 82.69.141.170 X-Return-Path: prvs=19393a69e5=killing@multiplay.co.uk X-Envelope-From: killing@multiplay.co.uk Message-ID: From: "Steven Hartland" To: "Matthew Ahrens" References: <20130712112117.Horde.JMB3fhsU1VfZRxdV4prj5Q6@mail.vx.sk><51FE1E1E.8080502@FreeBSD.org><5C28F2D739D94E6B928ACD55D207665A@multiplay.co.uk><51FE9BF5.2000301@FreeBSD.org><51FEA530.2090008@FreeBSD.org><4B747F3B634B4A859068FC8D782A7E1D@multiplay.co.uk><52011DE6.8090708@FreeBSD.org><4DA6B66E61374BE68C895C8B9A93BCB7@multiplay.co.uk> Subject: Re: N-way mirror read speedup in zfsonlinux Date: Thu, 15 Aug 2013 03:48:06 +0100 MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="----=_NextPart_000_031B_01CE996A.40624C10" X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.5931 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.6157 Cc: Xin Li , zfs-devel@freebsd.org X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 15 Aug 2013 02:47:57 -0000 This is a multi-part message in MIME format. ------=_NextPart_000_031B_01CE996A.40624C10 Content-Type: text/plain; format=flowed; charset="iso-8859-1"; reply-type=original Content-Transfer-Encoding: 7bit ----- Original Message ----- From: "Matthew Ahrens" > This patch looks reasonable to me. I shared Alexander's skepticism of the > time-based metric. Just a couple of specific notes: > > I believe that mm_preferred is only used for reads. You might not bother > with the load balancing logic if this is not a read. Good idea, done. > You might add accessor functions for avl_numnodes(&vq->vq_pending_tree) > and vq_lastoffset, rather than having vdev_disk.c reach into the queue > implementation. It looks like technically you could get away without > vq_lock for avl_numnodes(), as it's just loading a "long". But that is > really relying on the implementation details. More significantly, you have > no locking for vq_lastoffset. At a minimum, we should have some comments > explaining that this is OK. It seems like it could get the wrong result on > 32-bit systems (where load/store of 64 bit values is not atomic), but I > guess that's OK since it will only cause a tiny performance impact. Accessors added to vdev_queue including comment about lack of locking. > zfs_mirror_by_load -- could this be removed in favor of setting > zfs_vdev_mirror_locality_bonus = 0 to turn off the bonus? If not, seems > like it should at least be a boolean_t. zfs_mirror_by_load changes the entire read distribution method not just the locality bonus with 0 = current striped balancer, 1 = new load balancer (default) so I don't think thats appropriate. It was also added for testing convenience as much as anything, however all being well it could be removed unless there's objections to that? > vdev_mirror_load() -- if the vdev is not rotating, you always apply the > locality bonus. It was slightly difficult to follow the comment and logic > around this. Would a "seek penalty" which is applied if (rotating && > lastoffset != offset) be easier to understand? It might do but that would make the common case slightly more expensive what do others think? Regards Steve ================================================ This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmaster@multiplay.co.uk. ------=_NextPart_000_031B_01CE996A.40624C10 Content-Type: application/octet-stream; name="zfs-mirror-load.patch" Content-Transfer-Encoding: quoted-printable Content-Disposition: attachment; filename="zfs-mirror-load.patch" Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/vdev.h=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/vdev.h (revision = 254328)=0A= +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/vdev.h (working = copy)=0A= @@ -119,6 +119,9 @@=0A= extern void vdev_queue_fini(vdev_t *vd);=0A= extern zio_t *vdev_queue_io(zio_t *zio);=0A= extern void vdev_queue_io_done(zio_t *zio);=0A= +extern int vdev_queue_length(vdev_t *vd);=0A= +extern uint64_t vdev_queue_lastoffset(vdev_t *vd);=0A= +extern void vdev_queue_register_lastoffset(vdev_t *vd, zio_t *zio);=0A= =0A= extern void vdev_config_dirty(vdev_t *vd);=0A= extern void vdev_config_clean(vdev_t *vd);=0A= Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/vdev_impl.h=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/vdev_impl.h = (revision 254328)=0A= +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/vdev_impl.h = (working copy)=0A= @@ -106,6 +106,7 @@=0A= avl_tree_t vq_pending_tree;=0A= hrtime_t vq_io_complete_ts;=0A= kmutex_t vq_lock;=0A= + uint64_t vq_lastoffset;=0A= };=0A= =0A= /*=0A= @@ -199,7 +200,8 @@=0A= spa_aux_vdev_t *vdev_aux; /* for l2cache vdevs */=0A= zio_t *vdev_probe_zio; /* root of current probe */=0A= vdev_aux_t vdev_label_aux; /* on-disk aux state */=0A= - struct trim_map *vdev_trimmap;=0A= + struct trim_map *vdev_trimmap; /* map on outstanding trims */ =0A= + boolean_t vdev_rotating; /* backed by rotating media */=0A= =0A= /*=0A= * For DTrace to work in userland (libzpool) context, these fields must=0A= Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_geom.c=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_geom.c (revision = 254328)=0A= +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_geom.c (working = copy)=0A= @@ -42,9 +42,11 @@=0A= * Virtual device vector for GEOM.=0A= */=0A= =0A= +static g_attrchanged_t vdev_geom_attrchanged;=0A= struct g_class zfs_vdev_class =3D {=0A= .name =3D "ZFS::VDEV",=0A= .version =3D G_VERSION,=0A= + .attrchanged =3D vdev_geom_attrchanged,=0A= };=0A= =0A= DECLARE_GEOM_CLASS(zfs_vdev_class, zfs_vdev);=0A= @@ -62,6 +64,33 @@=0A= &vdev_geom_bio_delete_disable, 0, "Disable BIO_DELETE");=0A= =0A= static void=0A= +vdev_geom_set_rotating(vdev_t *vd, struct g_consumer *cp)=0A= +{ =0A= + int error, rotating;=0A= +=0A= + error =3D g_getattr("GEOM::rotating", cp, &rotating);=0A= + if (error =3D=3D 0)=0A= + vd->vdev_rotating =3D (rotating =3D=3D 1);=0A= + else=0A= + vd->vdev_rotating =3D B_TRUE;=0A= +}=0A= +=0A= +static void=0A= +vdev_geom_attrchanged(struct g_consumer *cp, const char *attr)=0A= +{=0A= + vdev_t *vd;=0A= +=0A= + vd =3D cp->private;=0A= + if (vd =3D=3D NULL)=0A= + return;=0A= +=0A= + if (strcmp(attr, "GEOM::rotating") !=3D 0)=0A= + return;=0A= +=0A= + vdev_geom_set_rotating(vd, cp);=0A= +}=0A= +=0A= +static void=0A= vdev_geom_orphan(struct g_consumer *cp)=0A= {=0A= vdev_t *vd;=0A= @@ -678,6 +707,8 @@=0A= vd->vdev_physpath =3D kmem_alloc(bufsize, KM_SLEEP);=0A= snprintf(vd->vdev_physpath, bufsize, "/dev/%s", pp->name);=0A= =0A= + vdev_geom_set_rotating(vd, cp);=0A= +=0A= return (0);=0A= }=0A= =0A= Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_mirror.c=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_mirror.c = (revision 254328)=0A= +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_mirror.c = (working copy)=0A= @@ -33,6 +33,8 @@=0A= #include =0A= #include =0A= =0A= +SYSCTL_DECL(_vfs_zfs_vdev);=0A= +=0A= /*=0A= * Virtual device vector for mirroring.=0A= */=0A= @@ -41,6 +43,7 @@=0A= vdev_t *mc_vd;=0A= uint64_t mc_offset;=0A= int mc_error;=0A= + int mc_load;=0A= uint8_t mc_tried;=0A= uint8_t mc_skipped;=0A= uint8_t mc_speculative;=0A= @@ -54,6 +57,26 @@=0A= mirror_child_t mm_child[1];=0A= } mirror_map_t;=0A= =0A= +int zfs_vdev_mirror_locality_track_size =3D 1 * 1024 * 1024;=0A= +TUNABLE_INT("vfs.zfs.vdev.mirror_locality_track_size",=0A= + &zfs_vdev_mirror_locality_track_size);=0A= +SYSCTL_INT(_vfs_zfs_vdev, OID_AUTO, mirror_locality_track_size, = CTLFLAG_RW,=0A= + &zfs_vdev_mirror_locality_track_size, 0, "Number of bytes that a = matching "=0A= + "IO pending for a vdev will trigger preference for that vdev");=0A= +=0A= +int zfs_vdev_mirror_locality_bonus =3D 5;=0A= +TUNABLE_INT("vfs.zfs.vdev.mirror_locality_bonus",=0A= + &zfs_vdev_mirror_locality_bonus);=0A= +SYSCTL_INT(_vfs_zfs_vdev, OID_AUTO, mirror_locality_bonus, CTLFLAG_RW,=0A= + &zfs_vdev_mirror_locality_bonus, 0,=0A= + "Load bonus driver applied to IO's based on locality and to all = IO's"=0A= + "for vdevs based on non-rotating media");=0A= +=0A= +int zfs_mirror_by_load =3D 1;=0A= +TUNABLE_INT("vfs.zfs.vdev.mirror_by_load", &zfs_mirror_by_load);=0A= +SYSCTL_INT(_vfs_zfs_vdev, OID_AUTO, mirror_by_load, CTLFLAG_RW,=0A= + &zfs_mirror_by_load, 0, "Balance mirror reads based vdev load");=0A= +=0A= int vdev_mirror_shift =3D 21;=0A= =0A= static void=0A= @@ -69,6 +92,39 @@=0A= zio_vsd_default_cksum_report=0A= };=0A= =0A= +static int=0A= +vdev_mirror_load(vdev_t *vd, uint64_t zio_offset)=0A= +{=0A= + uint64_t lastoffset;=0A= + int load;=0A= +=0A= + /* If the vdev isn't readable use the maximum load. */=0A= + if (!vdev_readable(vd))=0A= + return (INT_MAX);=0A= +=0A= + /* Baseline load based on pending queue length */=0A= + load =3D vdev_queue_length(vd);=0A= + lastoffset =3D vdev_queue_lastoffset(vd);=0A= +=0A= + /*=0A= + * Apply the full locality bonus for IO's which are=0A= + * sequential to the last IO queued or if the media is=0A= + * non-rotating as there is locality penalty (seek time).=0A= + */ =0A= + if (lastoffset =3D=3D zio_offset || !vd->vdev_rotating)=0A= + return (load - zfs_vdev_mirror_locality_bonus);=0A= +=0A= + /*=0A= + * Apply half the locality bonus for IO's within track size=0A= + * bytes of the last queued IO.=0A= + */=0A= + if (ABS(lastoffset - zio_offset) <=0A= + zfs_vdev_mirror_locality_track_size)=0A= + return (load - (zfs_vdev_mirror_locality_bonus / 2));=0A= +=0A= + return (load);=0A= +}=0A= +=0A= static mirror_map_t *=0A= vdev_mirror_map_alloc(zio_t *zio)=0A= {=0A= @@ -108,21 +164,47 @@=0A= mc->mc_offset =3D DVA_GET_OFFSET(&dva[c]);=0A= }=0A= } else {=0A= + int lowest_load =3D INT_MAX;=0A= + int lowest_child =3D 0;=0A= + boolean_t use_load =3D B_FALSE;=0A= + uint64_t zio_offset =3D zio->io_offset;=0A= +=0A= c =3D vd->vdev_children;=0A= =0A= mm =3D kmem_zalloc(offsetof(mirror_map_t, mm_child[c]), KM_SLEEP);=0A= mm->mm_children =3D c;=0A= mm->mm_replacing =3D (vd->vdev_ops =3D=3D &vdev_replacing_ops ||=0A= vd->vdev_ops =3D=3D &vdev_spare_ops);=0A= - mm->mm_preferred =3D mm->mm_replacing ? 0 :=0A= - (zio->io_offset >> vdev_mirror_shift) % c;=0A= mm->mm_root =3D B_FALSE;=0A= + if (mm->mm_replacing || zio->io_type !=3D ZIO_TYPE_READ) {=0A= + mm->mm_preferred =3D 0;=0A= + } else if (zfs_mirror_by_load =3D=3D 0) {=0A= + mm->mm_preferred =3D (zio->io_offset >>=0A= + vdev_mirror_shift) % c;=0A= + } else {=0A= + use_load =3D B_TRUE;=0A= + }=0A= =0A= for (c =3D 0; c < mm->mm_children; c++) {=0A= mc =3D &mm->mm_child[c];=0A= mc->mc_vd =3D vd->vdev_child[c];=0A= - mc->mc_offset =3D zio->io_offset;=0A= + mc->mc_offset =3D zio_offset;=0A= +=0A= + if (use_load) {=0A= + mc->mc_load =3D vdev_mirror_load(mc->mc_vd,=0A= + zio_offset);=0A= + if (mc->mc_load < lowest_load) {=0A= + lowest_load =3D mc->mc_load;=0A= + lowest_child =3D c; =0A= + }=0A= + }=0A= }=0A= +=0A= + if (use_load) {=0A= + mm->mm_preferred =3D lowest_child;=0A= + vdev_queue_register_lastoffset(=0A= + vd->vdev_child[lowest_child], zio);=0A= + }=0A= }=0A= =0A= zio->io_vsd =3D mm;=0A= Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c = (revision 254328)=0A= +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c (working = copy)=0A= @@ -155,6 +155,8 @@=0A= =0A= avl_create(&vq->vq_pending_tree, vdev_queue_offset_compare,=0A= sizeof (zio_t), offsetof(struct zio, io_offset_node));=0A= +=0A= + vq->vq_lastoffset =3D 0;=0A= }=0A= =0A= void=0A= @@ -446,3 +448,26 @@=0A= =0A= mutex_exit(&vq->vq_lock);=0A= }=0A= +=0A= +/*=0A= + * As these three methods are only used for load calculations we're not = precious=0A= + * if we get an incorrect value on 32bit platforms due to lack of = vq_lock mutex=0A= + * uses here. Instead we prefer to keep it lock free for the = performance.=0A= + */ =0A= +int=0A= +vdev_queue_length(vdev_t *vd)=0A= +{=0A= + return (avl_numnodes(&vd->vdev_queue.vq_pending_tree));=0A= +}=0A= +=0A= +uint64_t=0A= +vdev_queue_lastoffset(vdev_t *vd)=0A= +{=0A= + return (vd->vdev_queue.vq_lastoffset);=0A= +}=0A= +=0A= +void=0A= +vdev_queue_register_lastoffset(vdev_t *vd, zio_t *zio)=0A= +{=0A= + vd->vdev_queue.vq_lastoffset =3D zio->io_offset + zio->io_size;=0A= +}=0A= Index: sys/geom/geom_disk.c=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/geom/geom_disk.c (revision 254328)=0A= +++ sys/geom/geom_disk.c (working copy)=0A= @@ -392,6 +392,9 @@=0A= g_disk_kerneldump(bp, dp);=0A= else if (!strcmp(bp->bio_attribute, "GEOM::setstate"))=0A= g_disk_setstate(bp, sc);=0A= + else if (g_handleattr_int(bp, "GEOM::rotating",=0A= + (dp->d_nonrotating =3D=3D 0)))=0A= + break;=0A= else =0A= error =3D ENOIOCTL;=0A= break;=0A= Index: sys/geom/geom_disk.h=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/geom/geom_disk.h (revision 254328)=0A= +++ sys/geom/geom_disk.h (working copy)=0A= @@ -97,6 +97,7 @@=0A= uint16_t d_hba_device;=0A= uint16_t d_hba_subvendor;=0A= uint16_t d_hba_subdevice;=0A= + int d_nonrotating;=0A= =0A= /* Fields private to the driver */=0A= void *d_drv1;=0A= @@ -121,7 +122,8 @@=0A= #define DISK_VERSION_01 0x5856105a=0A= #define DISK_VERSION_02 0x5856105b=0A= #define DISK_VERSION_03 0x5856105c=0A= -#define DISK_VERSION DISK_VERSION_03=0A= +#define DISK_VERSION_04 0x5856105d=0A= +#define DISK_VERSION DISK_VERSION_04=0A= =0A= #endif /* _KERNEL */=0A= #endif /* _GEOM_GEOM_DISK_H_ */=0A= Index: sys/cam/ata/ata_da.c=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/cam/ata/ata_da.c (revision 254329)=0A= +++ sys/cam/ata/ata_da.c (working copy)=0A= @@ -1224,13 +1224,14 @@=0A= snprintf(announce_buf, sizeof(announce_buf),=0A= "kern.cam.ada.%d.write_cache", periph->unit_number);=0A= TUNABLE_INT_FETCH(announce_buf, &softc->write_cache);=0A= + adagetparams(periph, cgd);=0A= + softc->disk =3D disk_alloc();=0A= /* Disable queue sorting for non-rotational media by default. */=0A= - if (cgd->ident_data.media_rotation_rate =3D=3D 1)=0A= + if (cgd->ident_data.media_rotation_rate =3D=3D 1) {=0A= softc->sort_io_queue =3D 0;=0A= - else=0A= + softc->disk->d_nonrotating =3D 1;=0A= + } else=0A= softc->sort_io_queue =3D -1;=0A= - adagetparams(periph, cgd);=0A= - softc->disk =3D disk_alloc();=0A= softc->disk->d_devstat =3D devstat_new_entry(periph->periph_name,=0A= periph->unit_number, softc->params.secsize,=0A= DEVSTAT_ALL_SUPPORTED,=0A= Index: sys/cam/scsi/scsi_da.c=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/cam/scsi/scsi_da.c (revision 254329)=0A= +++ sys/cam/scsi/scsi_da.c (working copy)=0A= @@ -3387,8 +3387,12 @@=0A= * by default.=0A= */=0A= if (scsi_2btoul(bdc->medium_rotation_rate) =3D=3D=0A= - SVPD_BDC_RATE_NONE_ROTATING)=0A= + SVPD_BDC_RATE_NONE_ROTATING) {=0A= softc->sort_io_queue =3D 0;=0A= + softc->disk->d_nonrotating =3D 1;=0A= + disk_attr_changed(softc->disk,=0A= + "GEOM::rotating", M_NOWAIT);=0A= + }=0A= } else {=0A= int error;=0A= error =3D daerror(done_ccb, CAM_RETRY_SELTO,=0A= @@ -3437,8 +3441,12 @@=0A= * Disable queue sorting for non-rotational media=0A= * by default.=0A= */=0A= - if (ata_params->media_rotation_rate =3D=3D 1)=0A= + if (ata_params->media_rotation_rate =3D=3D 1) {=0A= softc->sort_io_queue =3D 0;=0A= + softc->disk->d_nonrotating =3D 1;=0A= + disk_attr_changed(softc->disk,=0A= + "GEOM::rotating", M_NOWAIT);=0A= + }=0A= } else {=0A= int error;=0A= error =3D daerror(done_ccb, CAM_RETRY_SELTO,=0A= ------=_NextPart_000_031B_01CE996A.40624C10-- From owner-zfs-devel@FreeBSD.ORG Thu Aug 15 03:43:52 2013 Return-Path: Delivered-To: zfs-devel@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 4D29B8BE; Thu, 15 Aug 2013 03:43:52 +0000 (UTC) (envelope-from linimon@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 20A2C2F52; Thu, 15 Aug 2013 03:43:52 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.7/8.14.7) with ESMTP id r7F3hpvc087112; Thu, 15 Aug 2013 03:43:51 GMT (envelope-from linimon@freefall.freebsd.org) Received: (from linimon@localhost) by freefall.freebsd.org (8.14.7/8.14.7/Submit) id r7F3hpwx087111; Thu, 15 Aug 2013 03:43:51 GMT (envelope-from linimon) Date: Thu, 15 Aug 2013 03:43:51 GMT Message-Id: <201308150343.r7F3hpwx087111@freefall.freebsd.org> To: linimon@FreeBSD.org, zfs-devel@FreeBSD.org, freebsd-fs@FreeBSD.org From: linimon@FreeBSD.org Subject: Re: kern/178387: [zfs] [patch] sparse files performance improvements X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 15 Aug 2013 03:43:52 -0000 Synopsis: [zfs] [patch] sparse files performance improvements Responsible-Changed-From-To: zfs-devel->freebsd-fs Responsible-Changed-By: linimon Responsible-Changed-When: Thu Aug 15 03:43:25 UTC 2013 Responsible-Changed-Why: http://www.freebsd.org/cgi/query-pr.cgi?pr=178387 From owner-zfs-devel@FreeBSD.ORG Thu Aug 15 03:44:03 2013 Return-Path: Delivered-To: zfs-devel@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 47C5D8C0; Thu, 15 Aug 2013 03:44:03 +0000 (UTC) (envelope-from linimon@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 1D5CA2F54; Thu, 15 Aug 2013 03:44:03 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.7/8.14.7) with ESMTP id r7F3i2Ii087158; Thu, 15 Aug 2013 03:44:02 GMT (envelope-from linimon@freefall.freebsd.org) Received: (from linimon@localhost) by freefall.freebsd.org (8.14.7/8.14.7/Submit) id r7F3i2Z7087157; Thu, 15 Aug 2013 03:44:02 GMT (envelope-from linimon) Date: Thu, 15 Aug 2013 03:44:02 GMT Message-Id: <201308150344.r7F3i2Z7087157@freefall.freebsd.org> To: linimon@FreeBSD.org, zfs-devel@FreeBSD.org, freebsd-fs@FreeBSD.org From: linimon@FreeBSD.org Subject: Re: kern/178467: [zfs] [request] Optimized Checksum Code for ZFS X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 15 Aug 2013 03:44:03 -0000 Synopsis: [zfs] [request] Optimized Checksum Code for ZFS Responsible-Changed-From-To: zfs-devel->freebsd-fs Responsible-Changed-By: linimon Responsible-Changed-When: Thu Aug 15 03:43:25 UTC 2013 Responsible-Changed-Why: apparently there is no such alias. reassign to freebsd-fs. http://www.freebsd.org/cgi/query-pr.cgi?pr=178467 From owner-zfs-devel@FreeBSD.ORG Thu Aug 15 08:23:36 2013 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 0489FA3D; Thu, 15 Aug 2013 08:23:36 +0000 (UTC) (envelope-from mavbsd@gmail.com) Received: from mail-bk0-x22c.google.com (mail-bk0-x22c.google.com [IPv6:2a00:1450:4008:c01::22c]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 5CAA12F00; Thu, 15 Aug 2013 08:23:35 +0000 (UTC) Received: by mail-bk0-f44.google.com with SMTP id mz10so127071bkb.17 for ; Thu, 15 Aug 2013 01:23:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:content-type:content-transfer-encoding; bh=bg9adBDa+wSNSBUZiRGAE7UQ17UBndYSRLpiI6GUIXs=; b=AkgaQYx2zNCwyknam9BXQTz96calGs0Eu/mWytUhhpu46dMe28cAsbjcdONc+P+X/S NIJGX/Gi8XEY3wituNxCHvQ3yM2QD0M5rl7t3/aiSyvnYRtdkj86xR/E1Kz1YylAxgvH qRiUNqRn1vcnyO+ei6eAXjY/BGtC9SvRqCsGQcLlYlXFdGvV8YqrXrF9sVpiyxlmaeyI am1Faw5wcZT8ZDnfuQCMZt+TQqjnbpg3kdl7XXx2uB+ockuwcxCGSNLrxEvxsm4+jUfb iGqTvoZ3MVsVVjBCivSUr13mKy46LaWLw62l8ere7UtPqE2KfMIsJ2BqL6R3Ol9H5ziN 7+qA== X-Received: by 10.205.115.17 with SMTP id fc17mr292956bkc.31.1376555013462; Thu, 15 Aug 2013 01:23:33 -0700 (PDT) Received: from mavbook.mavhome.dp.ua ([37.229.21.195]) by mx.google.com with ESMTPSA id m6sm8125035bki.7.2013.08.15.01.23.31 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Thu, 15 Aug 2013 01:23:32 -0700 (PDT) Sender: Alexander Motin Message-ID: <520C9001.2040406@FreeBSD.org> Date: Thu, 15 Aug 2013 11:23:29 +0300 From: Alexander Motin User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:17.0) Gecko/20130616 Thunderbird/17.0.6 MIME-Version: 1.0 To: Steven Hartland Subject: Re: N-way mirror read speedup in zfsonlinux References: <20130712112117.Horde.JMB3fhsU1VfZRxdV4prj5Q6@mail.vx.sk> <51FE1E1E.8080502@FreeBSD.org> <5C28F2D739D94E6B928ACD55D207665A@multiplay.co.uk> <51FE9BF5.2000301@FreeBSD.org> <51FEA530.2090008@FreeBSD.org> <4B747F3B634B4A859068FC8D782A7E1D@multiplay.co.uk> <52011DE6.8090708@FreeBSD.org> <4DA6B66E61374BE68C895C8B9A93BCB7@multiplay.co.uk> In-Reply-To: <4DA6B66E61374BE68C895C8B9A93BCB7@multiplay.co.uk> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: zfs-devel@freebsd.org, Xin Li X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 15 Aug 2013 08:23:36 -0000 On 14.08.2013 16:50, Steven Hartland wrote: > So based on mav's comments I've created a new version of the load balanced > patch which takes into account the last IO's locality on the disk. I like it. Thank you! Just a few comments: 1) I guess your change to scsi_da.c will send "GEOM::rotating" attribute change every time disk is probed (including each open) even when nothing really changed. It is not fatal, but absolutely unneeded overhead. 2) You are always giving full locality bonus to SSDs. It means that HDDs quite likely won't be used even for sequential read operations until SSDs are really busy, while that is the only point where HDDs still could if not compete but at least be useful. Surely it very much depend on specific devices characteristics, but I would try to give SSD half (or may be 3/4), so that precisely positioned HDD could still do something it can. 3) bde@ would argue about too long sysctl comments, while for me at least first is quite tangled. 4) I don't very like idea of expanding struct disk every time (especially not at the end). While that is probably easier then alternatives, now we have d_getattr method specifically to avoid that. -- Alexander Motin From owner-zfs-devel@FreeBSD.ORG Thu Aug 15 09:10:11 2013 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 85B536AA; Thu, 15 Aug 2013 09:10:11 +0000 (UTC) (envelope-from mavbsd@gmail.com) Received: from mail-bk0-x229.google.com (mail-bk0-x229.google.com [IPv6:2a00:1450:4008:c01::229]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id DC862214A; Thu, 15 Aug 2013 09:10:10 +0000 (UTC) Received: by mail-bk0-f41.google.com with SMTP id na10so149787bkb.0 for ; Thu, 15 Aug 2013 02:10:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:content-type:content-transfer-encoding; bh=x3XzKPksnbgJqAVczARuRmFQk/iobLcrD9hH5EIEKGg=; b=QRdUmidW2rpmmgT2LGsFL3aHz41oqMIY3x8zF/as8/lgx8GOUvcCdbm+GwuyyIUr1B 4aLPx6RZJUaEaHARAUvS3IoLIdjQiBm9Hr4t9yilNzUEpm9RZURaBvZMaQMdFbGMm5Zz V9Sx7ZQMfSA34l38EgfBJtY5TwEfIkx8RNFkL4m9AsxMUliIZ49/CjSY7VboAkXVSu98 /S+/sQLAN9KdvSfuAXWgBpgAOpAJz3UgHRFaimz7phZa4tV1swb2iL9AavbGK6MaB5l3 zLVBisAVYi7NXfztkr/iKw9ORFy89j3nmmkI/XTSLHySFvInconaDTPZ7WVtaJr+aqQF UVKw== X-Received: by 10.204.121.201 with SMTP id i9mr10155863bkr.13.1376557808854; Thu, 15 Aug 2013 02:10:08 -0700 (PDT) Received: from mavbook.mavhome.dp.ua ([37.229.21.195]) by mx.google.com with ESMTPSA id hn4sm8171588bkc.2.2013.08.15.02.10.06 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Thu, 15 Aug 2013 02:10:07 -0700 (PDT) Sender: Alexander Motin Message-ID: <520C9AED.9020303@FreeBSD.org> Date: Thu, 15 Aug 2013 12:10:05 +0300 From: Alexander Motin User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:17.0) Gecko/20130616 Thunderbird/17.0.6 MIME-Version: 1.0 To: Steven Hartland Subject: Re: N-way mirror read speedup in zfsonlinux References: <20130712112117.Horde.JMB3fhsU1VfZRxdV4prj5Q6@mail.vx.sk> <51FE1E1E.8080502@FreeBSD.org> <5C28F2D739D94E6B928ACD55D207665A@multiplay.co.uk> <51FE9BF5.2000301@FreeBSD.org> <51FEA530.2090008@FreeBSD.org> <4B747F3B634B4A859068FC8D782A7E1D@multiplay.co.uk> <52011DE6.8090708@FreeBSD.org> <4DA6B66E61374BE68C895C8B9A93BCB7@multiplay.co.uk> <520C9001.2040406@FreeBSD.org> In-Reply-To: <520C9001.2040406@FreeBSD.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: zfs-devel@freebsd.org, Xin Li X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 15 Aug 2013 09:10:11 -0000 On 15.08.2013 11:23, Alexander Motin wrote: > On 14.08.2013 16:50, Steven Hartland wrote: >> So based on mav's comments I've created a new version of the load >> balanced >> patch which takes into account the last IO's locality on the disk. > > I like it. Thank you! > > Just a few comments: > 1) I guess your change to scsi_da.c will send "GEOM::rotating" attribute > change every time disk is probed (including each open) even when nothing > really changed. It is not fatal, but absolutely unneeded overhead. > 2) You are always giving full locality bonus to SSDs. It means that HDDs > quite likely won't be used even for sequential read operations until > SSDs are really busy, while that is the only point where HDDs still > could if not compete but at least be useful. Surely it very much depend > on specific devices characteristics, but I would try to give SSD half > (or may be 3/4), so that precisely positioned HDD could still do > something it can. > 3) bde@ would argue about too long sysctl comments, while for me at > least first is quite tangled. > 4) I don't very like idea of expanding struct disk every time > (especially not at the end). While that is probably easier then > alternatives, now we have d_getattr method specifically to avoid that. One more minor thought: I guess with the proposed code all requests without locality bonus in case of equal load will go to the first disk. Probably not a big problem, but may be adding small (less then load of 1) random factor (like one from offset in original code) could not be bad from drives wearing point. -- Alexander Motin From owner-zfs-devel@FreeBSD.ORG Thu Aug 15 16:45:17 2013 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id A2E779F9 for ; Thu, 15 Aug 2013 16:45:17 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: from mail-pb0-x236.google.com (mail-pb0-x236.google.com [IPv6:2607:f8b0:400e:c01::236]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 73F352BF5 for ; Thu, 15 Aug 2013 16:45:17 +0000 (UTC) Received: by mail-pb0-f54.google.com with SMTP id ro12so945633pbb.41 for ; Thu, 15 Aug 2013 09:45:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=delphix.com; s=google; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=IRHkotrSkf0iEe7DBVb7kB8eo047EmTMljlWU1STYtg=; b=U8Sbj29Z2c47WjSdxcdw52zKfxiy9HzoV2auc+CmphEHxgRhfFwgHBDIRSQTlgl4fo bYKxmwRjHnFhBSTtqUKj6EXHh98ioE2nomG0FvQWS07wQu9y+ld/SNjqKl3OcB/wcFMv AbCFhgJmsbHTMrvMIAfeQCGSNPHpMKsSwryvE= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=IRHkotrSkf0iEe7DBVb7kB8eo047EmTMljlWU1STYtg=; b=YJvkbNeefQ2/ugw379i9VzKwo83wbN7evBL4n2v1OzY0bypeYQvdhmxD4fhSTwagDR QYnWf59xhJFG0ewX4o/jxlxHQ71eGYdS2SSVG7CfWsqEBkoXrvjZScC5j4sHtZbCma1I 5R6/qPiZdN06VfRmWvBapIqtCHGRq876a60zGrO4H6NakgOuR8qEJj/MWntQTR6yUCKw wasc9HR2ag243qctCzr2bPgMfklB1kxUMeyADd5UcAvXgU3/SeHaJXC6QOAgS8ilcAU+ jTfKLfymIHUuRQ6INnX2IQZ/z/29NpqNcONVdnnoB8nG1AoQMmgjiWvMWwZjkdW46/K/ YAng== X-Gm-Message-State: ALoCoQkkDSCZ6HuyxOGLf/P+UGrgKJYRofhQgMcq6TN+eN6AQRokf9LRDz9eALDZ2qv0AGD0OAt/ MIME-Version: 1.0 X-Received: by 10.68.179.65 with SMTP id de1mr16675776pbc.73.1376585117051; Thu, 15 Aug 2013 09:45:17 -0700 (PDT) Received: by 10.70.132.66 with HTTP; Thu, 15 Aug 2013 09:45:16 -0700 (PDT) In-Reply-To: References: <20130712112117.Horde.JMB3fhsU1VfZRxdV4prj5Q6@mail.vx.sk> <51FE1E1E.8080502@FreeBSD.org> <5C28F2D739D94E6B928ACD55D207665A@multiplay.co.uk> <51FE9BF5.2000301@FreeBSD.org> <51FEA530.2090008@FreeBSD.org> <4B747F3B634B4A859068FC8D782A7E1D@multiplay.co.uk> <52011DE6.8090708@FreeBSD.org> <4DA6B66E61374BE68C895C8B9A93BCB7@multiplay.co.uk> Date: Thu, 15 Aug 2013 09:45:16 -0700 Message-ID: Subject: Re: N-way mirror read speedup in zfsonlinux From: Matthew Ahrens To: Steven Hartland Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: Xin Li , zfs-devel@freebsd.org X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 15 Aug 2013 16:45:17 -0000 On Wed, Aug 14, 2013 at 7:48 PM, Steven Hartland wrote: > ----- Original Message ----- From: "Matthew Ahrens" > > This patch looks reasonable to me. I shared Alexander's skepticism of the >> time-based metric. Just a couple of specific notes: >> >> I believe that mm_preferred is only used for reads. You might not bother >> with the load balancing logic if this is not a read. >> > > Good idea, done. > > > You might add accessor functions for avl_numnodes(&vq->vq_pending_**tree) >> and vq_lastoffset, rather than having vdev_disk.c reach into the queue >> implementation. It looks like technically you could get away without >> vq_lock for avl_numnodes(), as it's just loading a "long". But that is >> really relying on the implementation details. More significantly, you >> have >> no locking for vq_lastoffset. At a minimum, we should have some comments >> explaining that this is OK. It seems like it could get the wrong result >> on >> 32-bit systems (where load/store of 64 bit values is not atomic), but I >> guess that's OK since it will only cause a tiny performance impact. >> > > Accessors added to vdev_queue including comment about lack of locking. > > > zfs_mirror_by_load -- could this be removed in favor of setting >> zfs_vdev_mirror_locality_bonus = 0 to turn off the bonus? If not, seems >> like it should at least be a boolean_t. >> > > zfs_mirror_by_load changes the entire read distribution method not just > the locality bonus with 0 = current striped balancer, 1 = new load balancer > (default) so I don't think thats appropriate. > Ah, yes, I was misreading that code. Even with locality_bonus=0, we'd take into account the load. > > It was also added for testing convenience as much as anything, however all > being well it could be removed unless there's objections to that? I'm fine with removing the old algorithm and the switch (zfs_mirror_by_load). > > > vdev_mirror_load() -- if the vdev is not rotating, you always apply the >> locality bonus. It was slightly difficult to follow the comment and logic >> around this. Would a "seek penalty" which is applied if (rotating && >> lastoffset != offset) be easier to understand? >> > > It might do but that would make the common case slightly more expensive > what do others think? How would it be more expensive? --matt > > > Regards > Steve > > ==============================**================== > This e.mail is private and confidential between Multiplay (UK) Ltd. and > the person or entity to whom it is addressed. In the event of misdirection, > the recipient is prohibited from using, copying, printing or otherwise > disseminating it or any information contained in it. > In the event of misdirection, illegible or incomplete transmission please > telephone +44 845 868 1337 > or return the E.mail to postmaster@multiplay.co.uk. > From owner-zfs-devel@FreeBSD.ORG Thu Aug 15 19:59:59 2013 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 4337873A; Thu, 15 Aug 2013 19:59:59 +0000 (UTC) (envelope-from prvs=19393a69e5=killing@multiplay.co.uk) Received: from mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) (using TLSv1 with cipher RC4-MD5 (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 56C812651; Thu, 15 Aug 2013 19:59:58 +0000 (UTC) Received: from r2d2 ([82.69.141.170]) by mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) (MDaemon PRO v10.0.4) with ESMTP id md50005514596.msg; Thu, 15 Aug 2013 20:59:47 +0100 X-Spam-Processed: mail1.multiplay.co.uk, Thu, 15 Aug 2013 20:59:47 +0100 (not processed: message from valid local sender) X-MDDKIM-Result: neutral (mail1.multiplay.co.uk) X-MDRemoteIP: 82.69.141.170 X-Return-Path: prvs=19393a69e5=killing@multiplay.co.uk X-Envelope-From: killing@multiplay.co.uk Message-ID: From: "Steven Hartland" To: "Alexander Motin" , "Matthew Ahrens" References: <20130712112117.Horde.JMB3fhsU1VfZRxdV4prj5Q6@mail.vx.sk> <51FE1E1E.8080502@FreeBSD.org> <5C28F2D739D94E6B928ACD55D207665A@multiplay.co.uk> <51FE9BF5.2000301@FreeBSD.org> <51FEA530.2090008@FreeBSD.org> <4B747F3B634B4A859068FC8D782A7E1D@multiplay.co.uk> <52011DE6.8090708@FreeBSD.org> <4DA6B66E61374BE68C895C8B9A93BCB7@multiplay.co.uk> <520C9001.2040406@FreeBSD.org> Subject: Re: N-way mirror read speedup in zfsonlinux Date: Thu, 15 Aug 2013 21:00:05 +0100 MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="----=_NextPart_000_0679_01CE99FA.6B2DF210" X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.5931 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.6157 Cc: zfs-devel@freebsd.org, Xin Li X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 15 Aug 2013 19:59:59 -0000 This is a multi-part message in MIME format. ------=_NextPart_000_0679_01CE99FA.6B2DF210 Content-Type: text/plain; format=flowed; charset="iso-8859-1"; reply-type=response Content-Transfer-Encoding: 7bit Replies to everyone's individual feedback below:- ----- Original Message ----- From: "Alexander Motin" > On 14.08.2013 16:50, Steven Hartland wrote: >> So based on mav's comments I've created a new version of the load balanced >> patch which takes into account the last IO's locality on the disk. > > I like it. Thank you! > > Just a few comments: > 1) I guess your change to scsi_da.c will send "GEOM::rotating" attribute > change every time disk is probed (including each open) even when nothing > really changed. It is not fatal, but absolutely unneeded overhead. No problem, change notifications are now guarded by a current value check. > 2) You are always giving full locality bonus to SSDs. It means that HDDs > quite likely won't be used even for sequential read operations until > SSDs are really busy, while that is the only point where HDDs still > could if not compete but at least be useful. Surely it very much depend > on specific devices characteristics, but I would try to give SSD half > (or may be 3/4), so that precisely positioned HDD could still do > something it can. I tested this case with the 2 x HDD's and 1 x SDD case and it results in very good balancing with all disks @ ~95% load with 3 threads e.g. dT: 1.001s w: 1.000s filter: ada[123]p1 L(q) ops/s r/s kBps ms/r w/s kBps ms/w %busy Name 4 1730 1730 219936 2.5 0 0 0.0 92.5| ada1p1 3 274 274 34659 10.5 0 0 0.0 91.1| ada2p1 5 266 266 33758 10.9 0 0 0.0 94.4| ada3p1 Reducing this to 1 thread does put all load to the SSD in my setup but this still results in signficantly higher performance than the current configuration but asll case where the HD's take a small portion of the load. That said I've reworked the way loads are calculating adding in more configuration options so it can be turned to how the user wants. > 3) bde@ would argue about too long sysctl comments, while for me at > least first is quite tangled. Matt already suggested changing it to a seek / latency penalty which having played around I also believe is much clearer :) @matt: Is this the sort of thing you had in mind with regards penalty / increment instead of bonus? > 4) I don't very like idea of expanding struct disk every time > (especially not at the end). While that is probably easier then > alternatives, now we have d_getattr method specifically to avoid that. I see what you mean but it needs to be stored somewhere otherwise we're going to have to query the device every time or did you have something else specific in mind? > One more minor thought: I guess with the proposed code all requests > without locality bonus in case of equal load will go to the first disk. > Probably not a big problem, but may be adding small (less then load of > 1) random factor (like one from offset in original code) could not be > bad from drives wearing point. I've re-implemented and optimised version from zfsonlinux based on zio_offset instead of time, which should ensure this doesn't happen. From: "Matthew Ahrens" > > It was also added for testing convenience as much as anything, however all > > being well it could be removed unless there's objections to that? > > I'm fine with removing the old algorithm and the switch > (zfs_mirror_by_load). Ok I've removed this, which does simply the code some what :) Updated patch attached. Regards Steve ================================================ This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmaster@multiplay.co.uk. ------=_NextPart_000_0679_01CE99FA.6B2DF210 Content-Type: application/octet-stream; name="zfs-mirror-load.patch" Content-Transfer-Encoding: quoted-printable Content-Disposition: attachment; filename="zfs-mirror-load.patch" Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/vdev.h=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/vdev.h (revision = 254328)=0A= +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/vdev.h (working = copy)=0A= @@ -119,6 +119,9 @@=0A= extern void vdev_queue_fini(vdev_t *vd);=0A= extern zio_t *vdev_queue_io(zio_t *zio);=0A= extern void vdev_queue_io_done(zio_t *zio);=0A= +extern int vdev_queue_length(vdev_t *vd);=0A= +extern uint64_t vdev_queue_lastoffset(vdev_t *vd);=0A= +extern void vdev_queue_register_lastoffset(vdev_t *vd, zio_t *zio);=0A= =0A= extern void vdev_config_dirty(vdev_t *vd);=0A= extern void vdev_config_clean(vdev_t *vd);=0A= Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/vdev_impl.h=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/vdev_impl.h = (revision 254328)=0A= +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/vdev_impl.h = (working copy)=0A= @@ -106,6 +106,7 @@=0A= avl_tree_t vq_pending_tree;=0A= hrtime_t vq_io_complete_ts;=0A= kmutex_t vq_lock;=0A= + uint64_t vq_lastoffset;=0A= };=0A= =0A= /*=0A= @@ -199,7 +200,8 @@=0A= spa_aux_vdev_t *vdev_aux; /* for l2cache vdevs */=0A= zio_t *vdev_probe_zio; /* root of current probe */=0A= vdev_aux_t vdev_label_aux; /* on-disk aux state */=0A= - struct trim_map *vdev_trimmap;=0A= + struct trim_map *vdev_trimmap; /* map on outstanding trims */ =0A= + boolean_t vdev_rotating; /* backed by rotating media */=0A= =0A= /*=0A= * For DTrace to work in userland (libzpool) context, these fields must=0A= Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_geom.c=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_geom.c (revision = 254328)=0A= +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_geom.c (working = copy)=0A= @@ -42,9 +42,11 @@=0A= * Virtual device vector for GEOM.=0A= */=0A= =0A= +static g_attrchanged_t vdev_geom_attrchanged;=0A= struct g_class zfs_vdev_class =3D {=0A= .name =3D "ZFS::VDEV",=0A= .version =3D G_VERSION,=0A= + .attrchanged =3D vdev_geom_attrchanged,=0A= };=0A= =0A= DECLARE_GEOM_CLASS(zfs_vdev_class, zfs_vdev);=0A= @@ -62,6 +64,33 @@=0A= &vdev_geom_bio_delete_disable, 0, "Disable BIO_DELETE");=0A= =0A= static void=0A= +vdev_geom_set_rotating(vdev_t *vd, struct g_consumer *cp)=0A= +{ =0A= + int error, rotating;=0A= +=0A= + error =3D g_getattr("GEOM::rotating", cp, &rotating);=0A= + if (error =3D=3D 0)=0A= + vd->vdev_rotating =3D (rotating =3D=3D 1);=0A= + else=0A= + vd->vdev_rotating =3D B_TRUE;=0A= +}=0A= +=0A= +static void=0A= +vdev_geom_attrchanged(struct g_consumer *cp, const char *attr)=0A= +{=0A= + vdev_t *vd;=0A= +=0A= + vd =3D cp->private;=0A= + if (vd =3D=3D NULL)=0A= + return;=0A= +=0A= + if (strcmp(attr, "GEOM::rotating") !=3D 0)=0A= + return;=0A= +=0A= + vdev_geom_set_rotating(vd, cp);=0A= +}=0A= +=0A= +static void=0A= vdev_geom_orphan(struct g_consumer *cp)=0A= {=0A= vdev_t *vd;=0A= @@ -678,6 +707,8 @@=0A= vd->vdev_physpath =3D kmem_alloc(bufsize, KM_SLEEP);=0A= snprintf(vd->vdev_physpath, bufsize, "/dev/%s", pp->name);=0A= =0A= + vdev_geom_set_rotating(vd, cp);=0A= +=0A= return (0);=0A= }=0A= =0A= Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_mirror.c=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_mirror.c = (revision 254328)=0A= +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_mirror.c = (working copy)=0A= @@ -41,6 +41,7 @@=0A= vdev_t *mc_vd;=0A= uint64_t mc_offset;=0A= int mc_error;=0A= + int mc_load;=0A= uint8_t mc_tried;=0A= uint8_t mc_skipped;=0A= uint8_t mc_speculative;=0A= @@ -54,8 +55,35 @@=0A= mirror_child_t mm_child[1];=0A= } mirror_map_t;=0A= =0A= -int vdev_mirror_shift =3D 21;=0A= +SYSCTL_DECL(_vfs_zfs_vdev);=0A= +static SYSCTL_NODE(_vfs_zfs_vdev, OID_AUTO, mirror, CTLFLAG_RD, 0,=0A= + "ZFS VDEV Mirror");=0A= =0A= +/* Rotating media load calculation configuration. */=0A= +static int rotating_inc =3D 0;=0A= +TUNABLE_INT("vfs.zfs.vdev.mirror.rotating_inc", &rotating_inc);=0A= +SYSCTL_INT(_vfs_zfs_vdev_mirror, OID_AUTO, rotating_inc, CTLFLAG_RW,=0A= + &rotating_inc, 0, "Rotating media load increment for non-seeking = IO's");=0A= +=0A= +static int rotating_seek_inc =3D 5;=0A= +TUNABLE_INT("vfs.zfs.vdev.mirror.rotating_seek_inc", = &rotating_seek_inc);=0A= +SYSCTL_INT(_vfs_zfs_vdev_mirror, OID_AUTO, rotating_seek_inc, = CTLFLAG_RW,=0A= + &rotating_seek_inc, 0, "Rotating media load increment for seeking = IO's");=0A= +=0A= +static int rotating_seek_offset =3D 1 * 1024 * 1024;=0A= +TUNABLE_INT("vfs.zfs.vdev.mirror.rotating_seek_offset", = &rotating_seek_offset);=0A= +SYSCTL_INT(_vfs_zfs_vdev_mirror, OID_AUTO, rotating_seek_offset, = CTLFLAG_RW,=0A= + &rotating_seek_offset, 0, "Offset in bytes from the last IO which "=0A= + "triggers a reduced rotating media seek increment");=0A= +=0A= +/* Non-rotating media load calculation configuration. */=0A= +static int non_rotating_inc =3D 0;=0A= +TUNABLE_INT("vfs.zfs.vdev.mirror.non_rotating_inc", &non_rotating_inc);=0A= +SYSCTL_INT(_vfs_zfs_vdev_mirror, OID_AUTO, non_rotating_inc, CTLFLAG_RW,=0A= + &non_rotating_inc, 0, "Non-rotating media load increment");=0A= +=0A= +static int vdev_mirror_shift =3D 21;=0A= +=0A= static void=0A= vdev_mirror_map_free(zio_t *zio)=0A= {=0A= @@ -69,6 +97,40 @@=0A= zio_vsd_default_cksum_report=0A= };=0A= =0A= +static int=0A= +vdev_mirror_load(vdev_t *vd, uint64_t zio_offset)=0A= +{=0A= + uint64_t lastoffset;=0A= + int load;=0A= +=0A= + /* If the vdev isn't readable use the maximum load. */=0A= + if (!vdev_readable(vd))=0A= + return (INT_MAX);=0A= +=0A= + /* Standard load based on pending queue length. */=0A= + load =3D vdev_queue_length(vd);=0A= + lastoffset =3D vdev_queue_lastoffset(vd);=0A= +=0A= + /* Non-rotating media. */=0A= + if (!vd->vdev_rotating)=0A= + return (load + non_rotating_inc);=0A= +=0A= + /* IO's which directly follow the last IO. */=0A= + if (lastoffset =3D=3D zio_offset)=0A= + return (load + rotating_inc);=0A= +=0A= + /*=0A= + * Apply half the seek increment to IO's within seek offset=0A= + * of the last IO queued to this vdev as they should incure less=0A= + * of a seek increment.=0A= + */=0A= + if (ABS(lastoffset - zio_offset) < rotating_seek_offset)=0A= + return (load + (rotating_seek_inc / 2));=0A= +=0A= + /* Apply the full seek increment to all other IO's. */=0A= + return (load + rotating_seek_inc);=0A= +}=0A= +=0A= static mirror_map_t *=0A= vdev_mirror_map_alloc(zio_t *zio)=0A= {=0A= @@ -108,20 +170,70 @@=0A= mc->mc_offset =3D DVA_GET_OFFSET(&dva[c]);=0A= }=0A= } else {=0A= + int lowest_load, lowest_child, lowest_nr;=0A= + uint64_t zio_offset;=0A= + boolean_t use_load;=0A= +=0A= c =3D vd->vdev_children;=0A= =0A= mm =3D kmem_zalloc(offsetof(mirror_map_t, mm_child[c]), KM_SLEEP);=0A= - mm->mm_children =3D c;=0A= + mm->mm_children =3D c--;=0A= mm->mm_replacing =3D (vd->vdev_ops =3D=3D &vdev_replacing_ops ||=0A= vd->vdev_ops =3D=3D &vdev_spare_ops);=0A= - mm->mm_preferred =3D mm->mm_replacing ? 0 :=0A= - (zio->io_offset >> vdev_mirror_shift) % c;=0A= mm->mm_root =3D B_FALSE;=0A= =0A= - for (c =3D 0; c < mm->mm_children; c++) {=0A= + use_load =3D (zio->io_type =3D=3D ZIO_TYPE_READ && !mm->mm_replacing);=0A= + zio_offset =3D zio->io_offset;=0A= + lowest_load =3D INT_MAX;=0A= +=0A= + do {=0A= mc =3D &mm->mm_child[c];=0A= mc->mc_vd =3D vd->vdev_child[c];=0A= - mc->mc_offset =3D zio->io_offset;=0A= + mc->mc_offset =3D zio_offset;=0A= +=0A= + if (use_load) {=0A= + mc->mc_load =3D vdev_mirror_load(mc->mc_vd,=0A= + zio_offset);=0A= + if (mc->mc_load < lowest_load) {=0A= + lowest_load =3D mc->mc_load;=0A= + lowest_child =3D c; =0A= + lowest_nr =3D 1;=0A= + } else if (mc->mc_load =3D=3D lowest_load) {=0A= + lowest_nr++;=0A= + }=0A= + }=0A= + } while (--c >=3D 0);=0A= +=0A= + if (use_load) {=0A= + if (lowest_nr !=3D 1) {=0A= + int d;=0A= +=0A= + /*=0A= + * To ensure we don't always favour the first=0A= + * matching vdev, which could lead to wear=0A= + * leveling issues on SSD's, we use the IO=0A= + * offset as a sudo random seed into the vdevs=0A= + * which have the lowest load.=0A= + */=0A= + d =3D (zio_offset >> vdev_mirror_shift)=0A= + % lowest_nr;=0A= + if (d !=3D 0) {=0A= + c =3D lowest_child - 1;=0A= + do {=0A= + if (mm->mm_child[c].mc_load =3D=3D=0A= + lowest_load && --d =3D=3D 0) {=0A= + lowest_child =3D c;=0A= + break;=0A= + }=0A= + c--;=0A= + } while (B_TRUE);=0A= + }=0A= + }=0A= + mm->mm_preferred =3D lowest_child;=0A= + vdev_queue_register_lastoffset(=0A= + vd->vdev_child[lowest_child], zio);=0A= + } else {=0A= + mm->mm_preferred =3D 0;=0A= }=0A= }=0A= =0A= Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c = (revision 254328)=0A= +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c (working = copy)=0A= @@ -155,6 +155,8 @@=0A= =0A= avl_create(&vq->vq_pending_tree, vdev_queue_offset_compare,=0A= sizeof (zio_t), offsetof(struct zio, io_offset_node));=0A= +=0A= + vq->vq_lastoffset =3D 0;=0A= }=0A= =0A= void=0A= @@ -446,3 +448,26 @@=0A= =0A= mutex_exit(&vq->vq_lock);=0A= }=0A= +=0A= +/*=0A= + * As these three methods are only used for load calculations we're not = precious=0A= + * if we get an incorrect value on 32bit platforms due to lack of = vq_lock mutex=0A= + * uses here. Instead we prefer to keep it lock free for the = performance.=0A= + */ =0A= +int=0A= +vdev_queue_length(vdev_t *vd)=0A= +{=0A= + return (avl_numnodes(&vd->vdev_queue.vq_pending_tree));=0A= +}=0A= +=0A= +uint64_t=0A= +vdev_queue_lastoffset(vdev_t *vd)=0A= +{=0A= + return (vd->vdev_queue.vq_lastoffset);=0A= +}=0A= +=0A= +void=0A= +vdev_queue_register_lastoffset(vdev_t *vd, zio_t *zio)=0A= +{=0A= + vd->vdev_queue.vq_lastoffset =3D zio->io_offset + zio->io_size;=0A= +}=0A= Index: sys/geom/geom_disk.c=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/geom/geom_disk.c (revision 254328)=0A= +++ sys/geom/geom_disk.c (working copy)=0A= @@ -392,6 +392,9 @@=0A= g_disk_kerneldump(bp, dp);=0A= else if (!strcmp(bp->bio_attribute, "GEOM::setstate"))=0A= g_disk_setstate(bp, sc);=0A= + else if (g_handleattr_int(bp, "GEOM::rotating",=0A= + (dp->d_nonrotating =3D=3D 0)))=0A= + break;=0A= else =0A= error =3D ENOIOCTL;=0A= break;=0A= Index: sys/geom/geom_disk.h=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/geom/geom_disk.h (revision 254328)=0A= +++ sys/geom/geom_disk.h (working copy)=0A= @@ -97,6 +97,7 @@=0A= uint16_t d_hba_device;=0A= uint16_t d_hba_subvendor;=0A= uint16_t d_hba_subdevice;=0A= + int d_nonrotating;=0A= =0A= /* Fields private to the driver */=0A= void *d_drv1;=0A= @@ -121,7 +122,8 @@=0A= #define DISK_VERSION_01 0x5856105a=0A= #define DISK_VERSION_02 0x5856105b=0A= #define DISK_VERSION_03 0x5856105c=0A= -#define DISK_VERSION DISK_VERSION_03=0A= +#define DISK_VERSION_04 0x5856105d=0A= +#define DISK_VERSION DISK_VERSION_04=0A= =0A= #endif /* _KERNEL */=0A= #endif /* _GEOM_GEOM_DISK_H_ */=0A= Index: sys/cam/ata/ata_da.c=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/cam/ata/ata_da.c (revision 254329)=0A= +++ sys/cam/ata/ata_da.c (working copy)=0A= @@ -1224,13 +1224,14 @@=0A= snprintf(announce_buf, sizeof(announce_buf),=0A= "kern.cam.ada.%d.write_cache", periph->unit_number);=0A= TUNABLE_INT_FETCH(announce_buf, &softc->write_cache);=0A= + adagetparams(periph, cgd);=0A= + softc->disk =3D disk_alloc();=0A= /* Disable queue sorting for non-rotational media by default. */=0A= - if (cgd->ident_data.media_rotation_rate =3D=3D 1)=0A= + if (cgd->ident_data.media_rotation_rate =3D=3D 1) {=0A= softc->sort_io_queue =3D 0;=0A= - else=0A= + softc->disk->d_nonrotating =3D 1;=0A= + } else=0A= softc->sort_io_queue =3D -1;=0A= - adagetparams(periph, cgd);=0A= - softc->disk =3D disk_alloc();=0A= softc->disk->d_devstat =3D devstat_new_entry(periph->periph_name,=0A= periph->unit_number, softc->params.secsize,=0A= DEVSTAT_ALL_SUPPORTED,=0A= Index: sys/cam/scsi/scsi_all.h=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/cam/scsi/scsi_all.h (revision 254328)=0A= +++ sys/cam/scsi/scsi_all.h (working copy)=0A= @@ -1451,7 +1451,7 @@=0A= u_int8_t page_length[2];=0A= u_int8_t medium_rotation_rate[2];=0A= #define SVPD_BDC_RATE_NOT_REPORTED 0x00=0A= -#define SVPD_BDC_RATE_NONE_ROTATING 0x01=0A= +#define SVPD_BDC_RATE_NON_ROTATING 0x01=0A= u_int8_t reserved1;=0A= u_int8_t nominal_form_factor;=0A= #define SVPD_BDC_FORM_NOT_REPORTED 0x00=0A= Index: sys/cam/scsi/scsi_da.c=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/cam/scsi/scsi_da.c (revision 254329)=0A= +++ sys/cam/scsi/scsi_da.c (working copy)=0A= @@ -3386,9 +3386,18 @@=0A= * Disable queue sorting for non-rotational media=0A= * by default.=0A= */=0A= + int old_nonrotating =3D softc->disk->d_nonrotating;=0A= if (scsi_2btoul(bdc->medium_rotation_rate) =3D=3D=0A= - SVPD_BDC_RATE_NONE_ROTATING)=0A= + SVPD_BDC_RATE_NON_ROTATING) {=0A= softc->sort_io_queue =3D 0;=0A= + softc->disk->d_nonrotating =3D 1;=0A= + } else {=0A= + softc->disk->d_nonrotating =3D 0;=0A= + }=0A= + if (softc->disk->d_nonrotating !=3D old_nonrotating) {=0A= + disk_attr_changed(softc->disk,=0A= + "GEOM::rotating", M_NOWAIT);=0A= + }=0A= } else {=0A= int error;=0A= error =3D daerror(done_ccb, CAM_RETRY_SELTO,=0A= @@ -3423,6 +3432,8 @@=0A= ptr =3D (uint16_t *)ata_params;=0A= =0A= if ((csio->ccb_h.status & CAM_STATUS_MASK) =3D=3D CAM_REQ_CMP) {=0A= + int old_nonrotating;=0A= +=0A= for (i =3D 0; i < sizeof(*ata_params) / 2; i++)=0A= ptr[i] =3D le16toh(ptr[i]);=0A= if (ata_params->support_dsm & ATA_SUPPORT_DSM_TRIM) {=0A= @@ -3437,8 +3448,18 @@=0A= * Disable queue sorting for non-rotational media=0A= * by default.=0A= */=0A= - if (ata_params->media_rotation_rate =3D=3D 1)=0A= + old_nonrotating =3D softc->disk->d_nonrotating;=0A= + if (ata_params->media_rotation_rate =3D=3D 1) {=0A= softc->sort_io_queue =3D 0;=0A= + softc->disk->d_nonrotating =3D 1;=0A= + } else {=0A= + softc->disk->d_nonrotating =3D 0;=0A= + }=0A= +=0A= + if (softc->disk->d_nonrotating !=3D old_nonrotating) {=0A= + disk_attr_changed(softc->disk,=0A= + "GEOM::rotating", M_NOWAIT);=0A= + }=0A= } else {=0A= int error;=0A= error =3D daerror(done_ccb, CAM_RETRY_SELTO,=0A= ------=_NextPart_000_0679_01CE99FA.6B2DF210-- From owner-zfs-devel@FreeBSD.ORG Thu Aug 15 20:19:11 2013 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 321F8C02; Thu, 15 Aug 2013 20:19:11 +0000 (UTC) (envelope-from mavbsd@gmail.com) Received: from mail-ee0-x22f.google.com (mail-ee0-x22f.google.com [IPv6:2a00:1450:4013:c00::22f]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 9532E2746; Thu, 15 Aug 2013 20:19:10 +0000 (UTC) Received: by mail-ee0-f47.google.com with SMTP id d49so576746eek.20 for ; Thu, 15 Aug 2013 13:19:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:content-type:content-transfer-encoding; bh=aEUpcKDZaajUS995T06bHVSh/N5BwmKKfSWvklSilD8=; b=GJsx3hpU35bfvVQHly5RtBoLRKYqYKQDHZbtZVcduH5TBZL9Ng06jGrtATT1pk7/7K p3VE+CmSm03TWOeebBHpUvhcxtfVsgdQ4ox0UuDWtKCbx+yGijp2KZpEyi6HpxbuOZFK JYWq5Gu+HWAq7Jwqfk0s5GZyWfPDbeMqyEx4qLh+Vj2KqnN1ravLEJJa3MQy05/sokj9 hjeDULIObmijzOMeVNZ9ZyMoRU6O6BXUfOywuOMRVKYoAYNMGmhOePXNva07gv43vGu1 kNFsqEhH8sCwORCtSTnqLRPItGNXzv/0Uo5PrdeWVjIVLAgnXCkT29lumYPbdshCQUjX ODRw== X-Received: by 10.14.115.133 with SMTP id e5mr25077659eeh.27.1376597948763; Thu, 15 Aug 2013 13:19:08 -0700 (PDT) Received: from mavbook.mavhome.dp.ua ([37.229.21.195]) by mx.google.com with ESMTPSA id bp43sm1400635eeb.4.2013.08.15.13.19.06 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Thu, 15 Aug 2013 13:19:07 -0700 (PDT) Sender: Alexander Motin Message-ID: <520D37B8.40706@FreeBSD.org> Date: Thu, 15 Aug 2013 23:19:04 +0300 From: Alexander Motin User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:17.0) Gecko/20130616 Thunderbird/17.0.6 MIME-Version: 1.0 To: Steven Hartland Subject: Re: N-way mirror read speedup in zfsonlinux References: <20130712112117.Horde.JMB3fhsU1VfZRxdV4prj5Q6@mail.vx.sk> <51FE1E1E.8080502@FreeBSD.org> <5C28F2D739D94E6B928ACD55D207665A@multiplay.co.uk> <51FE9BF5.2000301@FreeBSD.org> <51FEA530.2090008@FreeBSD.org> <4B747F3B634B4A859068FC8D782A7E1D@multiplay.co.uk> <52011DE6.8090708@FreeBSD.org> <4DA6B66E61374BE68C895C8B9A93BCB7@multiplay.co.uk> <520C9001.2040406@FreeBSD.org> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Matthew Ahrens , Xin Li , zfs-devel@freebsd.org X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 15 Aug 2013 20:19:11 -0000 On 15.08.2013 23:00, Steven Hartland wrote: > ----- Original Message ----- From: "Alexander Motin" >> 4) I don't very like idea of expanding struct disk every time >> (especially not at the end). While that is probably easier then >> alternatives, now we have d_getattr method specifically to avoid that. > > I see what you mean but it needs to be stored somewhere otherwise > we're going to have to query the device every time or did you have > something else specific in mind? I was thinking about storing it in da/ada periph softc's and modifying dagetattr()/adagetattr(). -- Alexander Motin From owner-zfs-devel@FreeBSD.ORG Thu Aug 15 20:36:13 2013 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id B15F669A for ; Thu, 15 Aug 2013 20:36:13 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: from mail-pd0-x22c.google.com (mail-pd0-x22c.google.com [IPv6:2607:f8b0:400e:c02::22c]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 8291C2839 for ; Thu, 15 Aug 2013 20:36:13 +0000 (UTC) Received: by mail-pd0-f172.google.com with SMTP id z10so1248027pdj.17 for ; Thu, 15 Aug 2013 13:36:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=delphix.com; s=google; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=PSLjkkscsHirXXI4YHQDMUlotTsVBbjz8c5Il8ymtXU=; b=bsnZsKOvy8oP3mdZmhff4Xz3z8Q46/mWKAYDU12UPz6iznSq3Y94Yuzd5g+e2GKJAW AkHy0vZXWB3Bra7+EewECLnLbL2NCIpdghMTJk9Vb3QQ8NnLPdWY1Oanpwm0SD/TNDxk AuOHlvkraRs1EOFyKAJkeux/lqVZj/iEEX1z0= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=PSLjkkscsHirXXI4YHQDMUlotTsVBbjz8c5Il8ymtXU=; b=fB9Io6SYsrkwBeXcadtB1oyvCoD4bTvUR0QVnkvdmU62q5rtLPxvM4YVIduOaM9nV3 biSM6lsmEIq+g4UbDDdLPxdUfQV2jWxlsqFtbPWTM7UnNUOQvhUoKAN7cLShY+1LJ5n/ sCTZLqEw5zD0g301g+4I3jtiecf2PhtLADMHmtXyy1pfkMypUe1D5tOBYEvyn8cKB+ZX gNdBIghE+dzB1RLi36ZsL/07QTcbS7/vocx4VUfyIc+CAnGAl0vSCpAPv5PPDag1GPZr c5Xxy7Yfhu38rdzEhsbY0aAHL5ZT9WkfoVdaxTtmeNiEyYl6ILms7iaSV4mhgt7dc54C bEmA== X-Gm-Message-State: ALoCoQmPNbIkVHQotA+MP3rNVYCeIZDNz0X+ppbQOwJRn7NC5E4x5qrTvo3MlEeH8Lt/SmLReZ1c MIME-Version: 1.0 X-Received: by 10.68.211.233 with SMTP id nf9mr17682946pbc.26.1376598973084; Thu, 15 Aug 2013 13:36:13 -0700 (PDT) Received: by 10.70.132.66 with HTTP; Thu, 15 Aug 2013 13:36:12 -0700 (PDT) In-Reply-To: References: <20130712112117.Horde.JMB3fhsU1VfZRxdV4prj5Q6@mail.vx.sk> <51FE1E1E.8080502@FreeBSD.org> <5C28F2D739D94E6B928ACD55D207665A@multiplay.co.uk> <51FE9BF5.2000301@FreeBSD.org> <51FEA530.2090008@FreeBSD.org> <4B747F3B634B4A859068FC8D782A7E1D@multiplay.co.uk> <52011DE6.8090708@FreeBSD.org> <4DA6B66E61374BE68C895C8B9A93BCB7@multiplay.co.uk> <520C9001.2040406@FreeBSD.org> Date: Thu, 15 Aug 2013 13:36:12 -0700 Message-ID: Subject: Re: N-way mirror read speedup in zfsonlinux From: Matthew Ahrens To: Steven Hartland Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: Xin Li , zfs-devel@freebsd.org X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 15 Aug 2013 20:36:13 -0000 The platform-independent code in the patch looks good to me with one exceptions: mm->mm_children = c--; It's non-obvious why c should be decremented *here*, as opposed to as part of the loop logic. If there's some trick going on with the loop then it's lost on me. Does it matter that we go through the children in reverse order? Is there something wrong with the existing loop logic: for (c = 0; c < mm->mm_children; c++) { --matt On Thu, Aug 15, 2013 at 1:00 PM, Steven Hartland wrote: > Replies to everyone's individual feedback below:- > > > ----- Original Message ----- From: "Alexander Motin" > > > On 14.08.2013 16:50, Steven Hartland wrote: >> >>> So based on mav's comments I've created a new version of the load >>> balanced >>> patch which takes into account the last IO's locality on the disk. >>> >> >> I like it. Thank you! >> >> Just a few comments: >> 1) I guess your change to scsi_da.c will send "GEOM::rotating" attribute >> change every time disk is probed (including each open) even when nothing >> really changed. It is not fatal, but absolutely unneeded overhead. >> > > No problem, change notifications are now guarded by a current value check. > > > 2) You are always giving full locality bonus to SSDs. It means that HDDs >> quite likely won't be used even for sequential read operations until SSDs >> are really busy, while that is the only point where HDDs still could if not >> compete but at least be useful. Surely it very much depend on specific >> devices characteristics, but I would try to give SSD half (or may be 3/4), >> so that precisely positioned HDD could still do something it can. >> > > I tested this case with the 2 x HDD's and 1 x SDD case and it results in > very good balancing with all disks @ ~95% load with 3 threads e.g. > > dT: 1.001s w: 1.000s filter: ada[123]p1 > L(q) ops/s r/s kBps ms/r w/s kBps ms/w %busy Name > 4 1730 1730 219936 2.5 0 0 0.0 92.5| ada1p1 > 3 274 274 34659 10.5 0 0 0.0 91.1| ada2p1 > 5 266 266 33758 10.9 0 0 0.0 94.4| ada3p1 > > Reducing this to 1 thread does put all load to the SSD in my setup > but this still results in signficantly higher performance than the > current configuration but asll case where the HD's take a small portion > of the load. > > That said I've reworked the way loads are calculating adding in more > configuration options so it can be turned to how the user wants. > > > 3) bde@ would argue about too long sysctl comments, while for me at >> least first is quite tangled. >> > > Matt already suggested changing it to a seek / latency penalty which > having played around I also believe is much clearer :) > > @matt: Is this the sort of thing you had in mind with regards penalty / > increment instead of bonus? > > > 4) I don't very like idea of expanding struct disk every time (especially >> not at the end). While that is probably easier then alternatives, now we >> have d_getattr method specifically to avoid that. >> > > I see what you mean but it needs to be stored somewhere otherwise > we're going to have to query the device every time or did you have > something else specific in mind? > > > One more minor thought: I guess with the proposed code all requests >> without locality bonus in case of equal load will go to the first disk. >> Probably not a big problem, but may be adding small (less then load of 1) >> random factor (like one from offset in original code) could not be bad from >> drives wearing point. >> > > I've re-implemented and optimised version from zfsonlinux based on > zio_offset instead of time, which should ensure this doesn't happen. > > From: "Matthew Ahrens" > > > It was also added for testing convenience as much as anything, however >> all >> > being well it could be removed unless there's objections to that? >> >> I'm fine with removing the old algorithm and the switch >> (zfs_mirror_by_load). >> > > Ok I've removed this, which does simply the code some what :) > > Updated patch attached. > > > Regards > Steve > > ==============================**================== > This e.mail is private and confidential between Multiplay (UK) Ltd. and > the person or entity to whom it is addressed. In the event of misdirection, > the recipient is prohibited from using, copying, printing or otherwise > disseminating it or any information contained in it. > In the event of misdirection, illegible or incomplete transmission please > telephone +44 845 868 1337 > or return the E.mail to postmaster@multiplay.co.uk. > From owner-zfs-devel@FreeBSD.ORG Thu Aug 15 21:36:04 2013 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 35DEB818 for ; Thu, 15 Aug 2013 21:36:04 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from mail.vx.sk (mail.vx.sk [176.9.45.25]) by mx1.freebsd.org (Postfix) with ESMTP id 402262B37 for ; Thu, 15 Aug 2013 21:36:03 +0000 (UTC) Received: from core.vx.sk (localhost [127.0.0.2]) by mail.vx.sk (Postfix) with ESMTP id 8168B4419C for ; Thu, 15 Aug 2013 23:35:56 +0200 (CEST) X-Virus-Scanned: amavisd-new at mail.vx.sk Received: from mail.vx.sk by core.vx.sk (amavisd-new, unix socket) with LMTP id 4qFXqGzMjEfA for ; Thu, 15 Aug 2013 23:35:55 +0200 (CEST) Received: from [192.168.2.103] (dslb-188-102-024-080.pools.arcor-ip.net [188.102.24.80]) by mail.vx.sk (Postfix) with ESMTPSA id A99AF4418B for ; Thu, 15 Aug 2013 23:35:53 +0200 (CEST) Message-ID: <520D49B8.6020108@FreeBSD.org> Date: Thu, 15 Aug 2013 23:35:52 +0200 From: Martin Matuska User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:17.0) Gecko/20130803 Thunderbird/17.0.8 MIME-Version: 1.0 To: zfs-devel@freebsd.org Subject: Fwd: review: 4045 zfs write throttle & i/o scheduler performance work References: In-Reply-To: X-Forwarded-Message-Id: Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.14 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list Reply-To: zfs-devel@freebsd.org, illumos-zfs List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 15 Aug 2013 21:36:04 -0000 -------- Original Message -------- Subject: review: 4045 zfs write throttle & i/o scheduler performance work Date: Thu, 15 Aug 2013 11:35:00 -0700 From: Matthew Ahrens To: illumos-zfs , Brian Behlendorf , Martin Matuska , Will Andrews , "Justin T. Gibbs" , Richard , Jorgen Lundman http://cr.illumos.org/~webrev/csiden/illumos-throttle/ This review mainly consists of two related performance improvements: 1. The ZFS i/o scheduler (vdev_queue.c) now divides i/os into 5 classes: sync read, sync write, async read, async write, and scrub/resilver. The scheduler issues a number of concurrent i/os from each class to the device. Once a class has been selected, an i/o is selected from this class using either an elevator algorithem (async, scrub classes) or FIFO (sync classes). The number of concurrent async write i/os is tuned dynamically based on i/o load, to achieve good sync i/o latency when there is not a high load of writes, and good write throughput when there is. See the block comment in vdev_queue.c (reproduced below) for more details. 2. The write throttle (dsl_pool_tempreserve_space() and txg_constrain_throughput()) is rewritten to produce much more consistent delays when under constant load. The new write throttle is based on the amount of dirty data, rather than guesses about future performance of the system. When there is a lot of dirty data, each transaction (e.g. write() syscall) will be delayed by the same small amount. This eliminates the "brick wall of wait" that the old write throttle could hit, causing all transactions to wait several seconds until the next txg opens. One of the keys to the new write throttle is decrementing the amount of dirty data as i/o completes, rather than at the end of spa_sync(). Note that the write throttle is only applied once the i/o scheduler is issuing the maximum number of outstanding async writes. See the block comments in dsl_pool.c and above dmu_tx_delay() (reproduced below) for more details. This diff has several other effects, including: * the commonly-tuned global variable zfs_vdev_max_pending has been removed; use per-class zfs_vdev_*_max_active values or zfs_vdev_max_active instead. * the size of each txg (meaning the amount of dirty data written, and thus the time it takes to write out) is now controlled differently. There is no longer an explicit time goal; the primary determinant is amount of dirty data. Systems that are under light or medium load will now often see that a txg is always syncing, but the impact to performance (e.g. read latency) is minimal. Tune zfs_dirty_data_max and zfs_dirty_data_sync to control this. * zio_taskq_batch_pct = 75 -- Only use 75% of all CPUs for compression, checksum, etc. This improves latency by not allowing these CPU-intensive tasks to consume all CPU (on machines with at least 4 CPU's; the percentage is rounded up). --matt APPENDIX: problems with the current i/o scheduler The current ZFS i/o scheduler (vdev_queue.c) is deadline based. The problem with this is that if there are always i/os pending, then certain classes of i/os can see very long delays. For example, if there are always synchronous reads outstanding, then no async writes will be serviced until they become "past due". One symptom of this situation is that each pass of the txg sync takes at least several seconds (typically 3 seconds). If many i/os become "past due" (their deadline is in the past), then we must service all of these overdue i/os before any new i/os. This happens when we enqueue a batch of async writes for the txg sync, with deadlines 2.5 seconds in the future. If we can't complete all the i/os in 2.5 seconds (e.g. because there were always reads pending), then these i/os will become past due. Now we must service all the "async" writes (which could be hundreds of megabytes) before we service any reads, introducing considerable latency to synchronous i/os (reads or ZIL writes). REFERENCE: block comments mentioned above: /* * ZFS Write Throttle * ------------------ * * ZFS must limit the rate of incoming writes to the rate at which it is able * to sync data modifications to the backend storage. Throttling by too much * creates an artificial limit; throttling by too little can only be sustained * for short periods and would lead to highly lumpy performance. On a per-pool * basis, ZFS tracks the amount of modified (dirty) data. As operations change * data, the amount of dirty data increases; as ZFS syncs out data, the amount * of dirty data decreases. When the amount of dirty data exceeds a * predetermined threshold further modifications are blocked until the amount * of dirty data decreases (as data is synced out). * * The limit on dirty data is tunable, and should be adjusted according to * both the IO capacity and available memory of the system. The larger the * window, the more ZFS is able to aggregate and amortize metadata (and data) * changes. However, memory is a limited resource, and allowing for more dirty * data comes at the cost of keeping other useful data in memory (for example * ZFS data cached by the ARC). * * Implementation * * As buffers are modified dsl_pool_willuse_space() increments both the per- * txg (dp_dirty_pertxg[]) and poolwide (dp_dirty_total) accounting of * dirty space used; dsl_pool_dirty_space() decrements those values as data * is synced out from dsl_pool_sync(). While only the poolwide value is * relevant, the per-txg value is useful for debugging. The tunable * zfs_dirty_data_max determines the dirty space limit. Once that value is * exceeded, new writes are halted until space frees up. * * The zfs_dirty_data_sync tunable dictates the threshold at which we * ensure that there is a txg syncing (see the comment in txg.c for a full * description of transaction group stages). * * The IO scheduler uses both the dirty space limit and current amount of * dirty data as inputs. Those values affect the number of concurrent IOs ZFS * issues. See the comment in vdev_queue.c for details of the IO scheduler. * * The delay is also calculated based on the amount of dirty data. See the * comment above dmu_tx_delay() for details. */ /* * We delay transactions when we've determined that the backend storage * isn't able to accommodate the rate of incoming writes. * * If there is already a transaction waiting, we delay relative to when * that transaction finishes waiting. This way the calculated min_time * is independent of the number of threads concurrently executing * transactions. * * If we are the only waiter, wait relative to when the transaction * started, rather than the current time. This credits the transaction for * "time already served", e.g. reading indirect blocks. * * The minimum time for a transaction to take is calculated as: * min_time = scale * (dirty - min) / (max - dirty) * min_time is then capped at zfs_delay_max_ns. * * The delay has two degrees of freedom that can be adjusted via tunables. * The percentage of dirty data at which we start to delay is defined by * zfs_delay_min_dirty_percent. This should typically be at or above * zfs_vdev_async_write_active_max_dirty_percent so that we only start to * delay after writing at full speed has failed to keep up with the incoming * write rate. The scale of the curve is defined by zfs_delay_scale. Roughly * speaking, this variable determines the amount of delay at the midpoint of * the curve. * * delay * 10ms +-------------------------------------------------------------*+ * | *| * 9ms + *+ * | *| * 8ms + *+ * | * | * 7ms + * + * | * | * 6ms + * + * | * | * 5ms + * + * | * | * 4ms + * + * | * | * 3ms + * + * | * | * 2ms + (midpoint) * + * | | ** | * 1ms + v *** + * | zfs_delay_scale ----------> ******** | * 0 +-------------------------------------*********----------------+ * 0% <- zfs_dirty_data_max -> 100% * * Note that since the delay is added to the outstanding time remaining on the * most recent transaction, the delay is effectively the inverse of IOPS. * Here the midpoint of 500us translates to 2000 IOPS. The shape of the curve * was chosen such that small changes in the amount of accumulated dirty data * in the first 3/4 of the curve yield relatively small differences in the * amount of delay. * * The effects can be easier to understand when the amount of delay is * represented on a log scale: * * delay * 100ms +-------------------------------------------------------------++ * + + * | | * + *+ * 10ms + *+ * + ** + * | (midpoint) ** | * + | ** + * 1ms + v **** + * + zfs_delay_scale ----------> ***** + * | **** | * + **** + * 100us + ** + * + * + * | * | * + * + * 10us + * + * + + * | | * + + * +--------------------------------------------------------------+ * 0% <- zfs_dirty_data_max -> 100% * * Note here that only as the amount of dirty data approaches its limit does * the delay start to increase rapidly. The goal of a properly tuned system * should be to keep the amount of dirty data out of that range by first * ensuring that the appropriate limits are set the I/O scheduler to reach * optimal throughput on the backend storage, and then by changing the value * of zfs_delay_scale to increase the steepness of the curve. */ /* * ZFS I/O Scheduler * --------------- * * ZFS issues I/O operations to leaf vdevs to satisfy and complete zios. The * I/O scheduler determines when and in what order those operations are * issued. The I/O scheduler divides operations into five I/O classes * prioritized in the following order: sync read, sync write, async read, * async write, and scrub/resilver. Each queue defines the minimum and * maximum number of concurrent operations that may be issued to the device. * In addition, the device has an aggregate maximum. Note that the sum of the * per-queue minimums must not exceed the aggregate maximum, and if the * aggregate maximum is equal to or greater than the sum of the per-queue * maximums, the per-queue minimum has no effect. * * For many physical devices, throughput increases with the number of * concurrent operations, but latency typically suffers. Further, physical * devices typically have a limit at which more concurrent operations have no * effect on throughput or can actually cause it to decrease. * * The scheduler selects the next operation to issue by first looking for an * I/O class whose minimum has not been satisfied. Once all are satisfied and * the aggregate maximum has not been hit, the scheduler looks for classes * whose maximum has not been satisfied. Iteration through the I/O classes is * done in the order specified above. No further operations are issued if the * aggregate maximum number of concurrent operations has been hit or if there * are no operations queued for an I/O class that has not hit its maximum. * Every time an i/o is queued or an operation completes, the I/O scheduler * looks for new operations to issue. * * All I/O classes have a fixed maximum number of outstanding operations * except for the async write class. Asynchronous writes represent the data * that is committed to stable storage during the syncing stage for * transaction groups (see txg.c). Transaction groups enter the syncing state * periodically so the number of queued async writes will quickly burst up and * then bleed down to zero. Rather than servicing them as quickly as possible, * the I/O scheduler changes the maximum number of active async write i/os * according to the amount of dirty data in the pool (see dsl_pool.c). Since * both throughput and latency typically increase with the number of * concurrent operations issued to physical devices, reducing the burstiness * in the number of concurrent operations also stabilizes the response time of * operations from other -- and in particular synchronous -- queues. In broad * strokes, the I/O scheduler will issue more concurrent operations from the * async write queue as there's more dirty data in the pool. * * Async Writes * * The number of concurrent operations issued for the async write I/O class * follows a piece-wise linear function defined by a few adjustable points. * * | o---------| <-- zfs_vdev_async_write_max_active * ^ | /^ | * | | / | | * active | / | | * I/O | / | | * count | / | | * | / | | * |------------o | | <-- zfs_vdev_async_write_min_active * 0|____________^______|_________| * 0% | | 100% of zfs_dirty_data_max * | | * | `-- zfs_vdev_async_write_active_max_dirty_percent * `--------- zfs_vdev_async_write_active_min_dirty_percent * * Until the amount of dirty data exceeds a minimum percentage of the dirty * data allowed in the pool, the I/O scheduler will limit the number of * concurrent operations to the minimum. As that threshold is crossed, the * number of concurrent operations issued increases linearly to the maximum at * the specified maximum percentage of the dirty data allowed in the pool. * * Ideally, the amount of dirty data on a busy pool will stay in the sloped * part of the function between zfs_vdev_async_write_active_min_dirty_percent * and zfs_vdev_async_write_active_max_dirty_percent. If it exceeds the * maximum percentage, this indicates that the rate of incoming data is * greater than the rate that the backend storage can handle. In this case, we * must further throttle incoming writes (see dmu_tx_delay() for details). */ From owner-zfs-devel@FreeBSD.ORG Thu Aug 15 21:39:04 2013 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 1201E93F; Thu, 15 Aug 2013 21:39:04 +0000 (UTC) (envelope-from prvs=19393a69e5=killing@multiplay.co.uk) Received: from mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) (using TLSv1 with cipher RC4-MD5 (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 506632B53; Thu, 15 Aug 2013 21:39:02 +0000 (UTC) Received: from r2d2 ([82.69.141.170]) by mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) (MDaemon PRO v10.0.4) with ESMTP id md50005515885.msg; Thu, 15 Aug 2013 22:39:00 +0100 X-Spam-Processed: mail1.multiplay.co.uk, Thu, 15 Aug 2013 22:39:00 +0100 (not processed: message from valid local sender) X-MDDKIM-Result: neutral (mail1.multiplay.co.uk) X-MDRemoteIP: 82.69.141.170 X-Return-Path: prvs=19393a69e5=killing@multiplay.co.uk X-Envelope-From: killing@multiplay.co.uk Message-ID: From: "Steven Hartland" To: "Matthew Ahrens" References: <20130712112117.Horde.JMB3fhsU1VfZRxdV4prj5Q6@mail.vx.sk><51FE1E1E.8080502@FreeBSD.org><5C28F2D739D94E6B928ACD55D207665A@multiplay.co.uk><51FE9BF5.2000301@FreeBSD.org><51FEA530.2090008@FreeBSD.org><4B747F3B634B4A859068FC8D782A7E1D@multiplay.co.uk><52011DE6.8090708@FreeBSD.org><4DA6B66E61374BE68C895C8B9A93BCB7@multiplay.co.uk><520C9001.2040406@FreeBSD.org> Subject: Re: N-way mirror read speedup in zfsonlinux Date: Thu, 15 Aug 2013 22:39:16 +0100 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="iso-8859-1"; reply-type=original Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.5931 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.6157 Cc: Xin Li , zfs-devel@freebsd.org X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 15 Aug 2013 21:39:04 -0000 ----- Original Message ----- From: "Matthew Ahrens" > The platform-independent code in the patch looks good to me with one > exceptions: > > mm->mm_children = c--; > > It's non-obvious why c should be decremented *here*, as opposed to as part > of the loop logic. If there's some trick going on with the loop then it's > lost on me. This just configures 'c' so the loop starts with the last child, could split to a c--; on its own line if people prefer? > Does it matter that we go through the children in reverse > order? Is there something wrong with the existing loop logic: > > for (c = 0; c < mm->mm_children; c++) { Order is not important, but using a do{} while(); provides a small optimisation which avoids the first loop test totally and then on subisquent iterations compare against a constant not through a struct pointer. Regards Steve ================================================ This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmaster@multiplay.co.uk. From owner-zfs-devel@FreeBSD.ORG Thu Aug 15 21:41:18 2013 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id C752DCF5; Thu, 15 Aug 2013 21:41:18 +0000 (UTC) (envelope-from prvs=19393a69e5=killing@multiplay.co.uk) Received: from mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) (using TLSv1 with cipher RC4-MD5 (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 1123A2BC5; Thu, 15 Aug 2013 21:41:17 +0000 (UTC) Received: from r2d2 ([82.69.141.170]) by mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) (MDaemon PRO v10.0.4) with ESMTP id md50005515938.msg; Thu, 15 Aug 2013 22:41:15 +0100 X-Spam-Processed: mail1.multiplay.co.uk, Thu, 15 Aug 2013 22:41:15 +0100 (not processed: message from valid local sender) X-MDDKIM-Result: neutral (mail1.multiplay.co.uk) X-MDRemoteIP: 82.69.141.170 X-Return-Path: prvs=19393a69e5=killing@multiplay.co.uk X-Envelope-From: killing@multiplay.co.uk Message-ID: From: "Steven Hartland" To: "Alexander Motin" References: <20130712112117.Horde.JMB3fhsU1VfZRxdV4prj5Q6@mail.vx.sk> <51FE1E1E.8080502@FreeBSD.org> <5C28F2D739D94E6B928ACD55D207665A@multiplay.co.uk> <51FE9BF5.2000301@FreeBSD.org> <51FEA530.2090008@FreeBSD.org> <4B747F3B634B4A859068FC8D782A7E1D@multiplay.co.uk> <52011DE6.8090708@FreeBSD.org> <4DA6B66E61374BE68C895C8B9A93BCB7@multiplay.co.uk> <520C9001.2040406@FreeBSD.org> <520D37B8.40706@FreeBSD.org> Subject: Re: N-way mirror read speedup in zfsonlinux Date: Thu, 15 Aug 2013 22:41:35 +0100 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="iso-8859-1"; reply-type=response Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.5931 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.6157 Cc: Matthew Ahrens , Xin Li , zfs-devel@freebsd.org X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 15 Aug 2013 21:41:18 -0000 ----- Original Message ----- From: "Alexander Motin" > On 15.08.2013 23:00, Steven Hartland wrote: >> ----- Original Message ----- From: "Alexander Motin" >>> 4) I don't very like idea of expanding struct disk every time >>> (especially not at the end). While that is probably easier then >>> alternatives, now we have d_getattr method specifically to avoid that. >> >> I see what you mean but it needs to be stored somewhere otherwise >> we're going to have to query the device every time or did you have >> something else specific in mind? > > I was thinking about storing it in da/ada periph softc's and modifying > dagetattr()/adagetattr(). That would make it more work to add non-cam based support such as mfi. Regards Steve ================================================ This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmaster@multiplay.co.uk. From owner-zfs-devel@FreeBSD.ORG Thu Aug 15 21:50:13 2013 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 9D999E67; Thu, 15 Aug 2013 21:50:13 +0000 (UTC) (envelope-from mavbsd@gmail.com) Received: from mail-ee0-x22f.google.com (mail-ee0-x22f.google.com [IPv6:2a00:1450:4013:c00::22f]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 0E5712BFB; Thu, 15 Aug 2013 21:50:12 +0000 (UTC) Received: by mail-ee0-f47.google.com with SMTP id d49so613307eek.20 for ; Thu, 15 Aug 2013 14:50:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:content-type:content-transfer-encoding; bh=jh8XLrwzgsgBS4wUoN/J3jap5Omzs0HWjmGuH0vWGI4=; b=ZKn+RoQ8AeExXqR7VmeNyRdn15t8SXTjOA+W/Ba7Vdh6tQPTFu8BXJ48ynilPx6zhh 1P+KvrcFo5xfk9HrXE1SLYCm+I+dyJbewDy5fYB4wTTONp3IIZfhJJ6x9YGRX1SeCrLG lLhQm8SLLhA1oSgKozPeybmS/v986QJv2EB7GAAXElGBy1CTrvj0p99SlgjnGeO3jPcm FigyaZJmfFPbVK5ZSamrYpTFI7sFvFr3iEdJLwCSGuVYnavIVtXwK0CGbxQBWK8HUOHv URPAh3qaVS5QeyUrkVSGu6tTXbYXe+SqmpsZFBT/SzMDSlQKEMZRHYdIcYO4jbVYOoWq yDmw== X-Received: by 10.14.3.3 with SMTP id 3mr7087626eeg.57.1376603411430; Thu, 15 Aug 2013 14:50:11 -0700 (PDT) Received: from mavbook.mavhome.dp.ua ([37.229.21.195]) by mx.google.com with ESMTPSA id n48sm2058862eeg.17.2013.08.15.14.50.09 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Thu, 15 Aug 2013 14:50:10 -0700 (PDT) Sender: Alexander Motin Message-ID: <520D4D10.6010507@FreeBSD.org> Date: Fri, 16 Aug 2013 00:50:08 +0300 From: Alexander Motin User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:17.0) Gecko/20130616 Thunderbird/17.0.6 MIME-Version: 1.0 To: Steven Hartland Subject: Re: N-way mirror read speedup in zfsonlinux References: <20130712112117.Horde.JMB3fhsU1VfZRxdV4prj5Q6@mail.vx.sk> <51FE1E1E.8080502@FreeBSD.org> <5C28F2D739D94E6B928ACD55D207665A@multiplay.co.uk> <51FE9BF5.2000301@FreeBSD.org> <51FEA530.2090008@FreeBSD.org> <4B747F3B634B4A859068FC8D782A7E1D@multiplay.co.uk> <52011DE6.8090708@FreeBSD.org> <4DA6B66E61374BE68C895C8B9A93BCB7@multiplay.co.uk> <520C9001.2040406@FreeBSD.org> <520D37B8.40706@FreeBSD.org> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Matthew Ahrens , Xin Li , zfs-devel@freebsd.org X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 15 Aug 2013 21:50:13 -0000 On 16.08.2013 00:41, Steven Hartland wrote: > ----- Original Message ----- From: "Alexander Motin" > >> On 15.08.2013 23:00, Steven Hartland wrote: >>> ----- Original Message ----- From: "Alexander Motin" >>>> 4) I don't very like idea of expanding struct disk every time >>>> (especially not at the end). While that is probably easier then >>>> alternatives, now we have d_getattr method specifically to avoid that. >>> >>> I see what you mean but it needs to be stored somewhere otherwise >>> we're going to have to query the device every time or did you have >>> something else specific in mind? >> >> I was thinking about storing it in da/ada periph softc's and modifying >> dagetattr()/adagetattr(). > > That would make it more work to add non-cam based support such as mfi. That is why I have told that indeed proposed implementation is easier then alternatives. :) BTW, does RAID controllers like mfi can provide such details? -- Alexander Motin From owner-zfs-devel@FreeBSD.ORG Thu Aug 15 22:03:35 2013 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id B27AEF08; Thu, 15 Aug 2013 22:03:35 +0000 (UTC) (envelope-from prvs=19393a69e5=killing@multiplay.co.uk) Received: from mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) (using TLSv1 with cipher RC4-MD5 (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 0461A2C90; Thu, 15 Aug 2013 22:03:34 +0000 (UTC) Received: from r2d2 ([82.69.141.170]) by mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) (MDaemon PRO v10.0.4) with ESMTP id md50005516196.msg; Thu, 15 Aug 2013 23:03:33 +0100 X-Spam-Processed: mail1.multiplay.co.uk, Thu, 15 Aug 2013 23:03:33 +0100 (not processed: message from valid local sender) X-MDDKIM-Result: neutral (mail1.multiplay.co.uk) X-MDRemoteIP: 82.69.141.170 X-Return-Path: prvs=19393a69e5=killing@multiplay.co.uk X-Envelope-From: killing@multiplay.co.uk Message-ID: From: "Steven Hartland" To: "Alexander Motin" References: <20130712112117.Horde.JMB3fhsU1VfZRxdV4prj5Q6@mail.vx.sk> <51FE1E1E.8080502@FreeBSD.org> <5C28F2D739D94E6B928ACD55D207665A@multiplay.co.uk> <51FE9BF5.2000301@FreeBSD.org> <51FEA530.2090008@FreeBSD.org> <4B747F3B634B4A859068FC8D782A7E1D@multiplay.co.uk> <52011DE6.8090708@FreeBSD.org> <4DA6B66E61374BE68C895C8B9A93BCB7@multiplay.co.uk> <520C9001.2040406@FreeBSD.org> <520D37B8.40706@FreeBSD.org> <520D4D10.6010507@FreeBSD.org> Subject: Re: N-way mirror read speedup in zfsonlinux Date: Thu, 15 Aug 2013 23:03:53 +0100 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="iso-8859-1"; reply-type=response Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.5931 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.6157 Cc: Matthew Ahrens , Xin Li , zfs-devel@freebsd.org X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 15 Aug 2013 22:03:35 -0000 ----- Original Message ----- From: "Alexander Motin" > On 16.08.2013 00:41, Steven Hartland wrote: >> ----- Original Message ----- From: "Alexander Motin" >> >>> On 15.08.2013 23:00, Steven Hartland wrote: >>>> ----- Original Message ----- From: "Alexander Motin" >>>>> 4) I don't very like idea of expanding struct disk every time >>>>> (especially not at the end). While that is probably easier then >>>>> alternatives, now we have d_getattr method specifically to avoid that. >>>> >>>> I see what you mean but it needs to be stored somewhere otherwise >>>> we're going to have to query the device every time or did you have >>>> something else specific in mind? >>> >>> I was thinking about storing it in da/ada periph softc's and modifying >>> dagetattr()/adagetattr(). >> >> That would make it more work to add non-cam based support such as mfi. > > That is why I have told that indeed proposed implementation is easier > then alternatives. :) BTW, does RAID controllers like mfi can provide > such details? When used in JBOD mode yes we should be able to access all the normal device data. Regards Steve ================================================ This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmaster@multiplay.co.uk. From owner-zfs-devel@FreeBSD.ORG Fri Aug 16 16:16:54 2013 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 674B1B31; Fri, 16 Aug 2013 16:16:54 +0000 (UTC) (envelope-from prvs=1940958969=killing@multiplay.co.uk) Received: from mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) (using TLSv1 with cipher RC4-MD5 (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id D5EB32187; Fri, 16 Aug 2013 16:16:53 +0000 (UTC) Received: from r2d2 ([82.69.141.170]) by mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) (MDaemon PRO v10.0.4) with ESMTP id md50005525416.msg; Fri, 16 Aug 2013 17:16:51 +0100 X-Spam-Processed: mail1.multiplay.co.uk, Fri, 16 Aug 2013 17:16:51 +0100 (not processed: message from valid local sender) X-MDDKIM-Result: neutral (mail1.multiplay.co.uk) X-MDRemoteIP: 82.69.141.170 X-Return-Path: prvs=1940958969=killing@multiplay.co.uk X-Envelope-From: killing@multiplay.co.uk Message-ID: From: "Steven Hartland" To: "Steven Hartland" , "Matthew Ahrens" References: <20130712112117.Horde.JMB3fhsU1VfZRxdV4prj5Q6@mail.vx.sk><51FE1E1E.8080502@FreeBSD.org><5C28F2D739D94E6B928ACD55D207665A@multiplay.co.uk><51FE9BF5.2000301@FreeBSD.org><51FEA530.2090008@FreeBSD.org><4B747F3B634B4A859068FC8D782A7E1D@multiplay.co.uk><52011DE6.8090708@FreeBSD.org><4DA6B66E61374BE68C895C8B9A93BCB7@multiplay.co.uk><520C9001.2040406@FreeBSD.org> Subject: Re: N-way mirror read speedup in zfsonlinux Date: Fri, 16 Aug 2013 17:17:10 +0100 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="iso-8859-1"; reply-type=response Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.5931 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.6157 Cc: zfs-devel@freebsd.org, Xin Li X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 Aug 2013 16:16:54 -0000 Baring any more feedback I intend to commit this soon, so please shout if there's anything outstanding ;-) ----- Original Message ----- From: "Steven Hartland" > ----- Original Message ----- > From: "Matthew Ahrens" > > >> The platform-independent code in the patch looks good to me with one >> exceptions: >> >> mm->mm_children = c--; >> >> It's non-obvious why c should be decremented *here*, as opposed to as part >> of the loop logic. If there's some trick going on with the loop then it's >> lost on me. > > This just configures 'c' so the loop starts with the last child, could split > to a c--; on its own line if people prefer? > >> Does it matter that we go through the children in reverse >> order? Is there something wrong with the existing loop logic: >> >> for (c = 0; c < mm->mm_children; c++) { > > Order is not important, but using a do{} while(); provides a small optimisation > which avoids the first loop test totally and then on subisquent iterations > compare against a constant not through a struct pointer. Regards Steve ================================================ This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmaster@multiplay.co.uk. From owner-zfs-devel@FreeBSD.ORG Fri Aug 16 17:02:13 2013 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 3A816703 for ; Fri, 16 Aug 2013 17:02:13 +0000 (UTC) (envelope-from will@firepipe.net) Received: from mail-vc0-f182.google.com (mail-vc0-f182.google.com [209.85.220.182]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id EB55F246A for ; Fri, 16 Aug 2013 17:02:12 +0000 (UTC) Received: by mail-vc0-f182.google.com with SMTP id hf12so1651924vcb.13 for ; Fri, 16 Aug 2013 10:02:06 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=pHKlQi+F2Fhy8ZuZmy/a7gTlWnz9j2snBZp8i0vKCrI=; b=i7SlhQcsxvsJVk949VPDIxbPdpzqfvmmsQ8u7GmH183bYJAcKxLIwM1fDL6cevRg1H 8EcLDRlzswlLZIPdTXbVBWSLb8gN02oHB2sBrvWK1CU+rzeTRCUERXpS/goFUSrjJiBt EUn6UWjuCEO5SYbmwf7Dfp7fMs7ZD9S39qS2ZLpoo0ZpwqnsJfwO0OJGEuziAaq6tnCu KW/awajF7AUcSYcyzED8ujjrBxLHWXOumaC81koNLkFZLCVSBsbIBVGk24fG/v9kbzyp yLhqbwBe4BmCg/5Qhwz0DLAZ/FHm74cPFo+PQ0RzI0PhXup3V+c92dK3gB0nepHyPd2y RUtg== X-Gm-Message-State: ALoCoQmCSK5KCgEfPC/O7mt2Y+4wxEcY1hfyl/ZH8QVaq68FiDpt+uh4/K+GsC7HsixLgVuVaVuK MIME-Version: 1.0 X-Received: by 10.220.10.194 with SMTP id q2mr2105057vcq.2.1376672526768; Fri, 16 Aug 2013 10:02:06 -0700 (PDT) Received: by 10.58.226.66 with HTTP; Fri, 16 Aug 2013 10:02:06 -0700 (PDT) In-Reply-To: References: <20130712112117.Horde.JMB3fhsU1VfZRxdV4prj5Q6@mail.vx.sk> <51FE1E1E.8080502@FreeBSD.org> <5C28F2D739D94E6B928ACD55D207665A@multiplay.co.uk> <51FE9BF5.2000301@FreeBSD.org> <51FEA530.2090008@FreeBSD.org> <4B747F3B634B4A859068FC8D782A7E1D@multiplay.co.uk> <52011DE6.8090708@FreeBSD.org> <4DA6B66E61374BE68C895C8B9A93BCB7@multiplay.co.uk> <520C9001.2040406@FreeBSD.org> Date: Fri, 16 Aug 2013 11:02:06 -0600 Message-ID: Subject: Re: N-way mirror read speedup in zfsonlinux From: Will Andrews To: Steven Hartland Content-Type: text/plain; charset=UTF-8 Cc: Matthew Ahrens , Xin Li , zfs-devel@freebsd.org X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 Aug 2013 17:02:13 -0000 We have a vdev_geom_attrchanged() in our tree as well, to support physical path management (an early version of which is in the zfsd project branch; commits to HEAD are still pending). So, my request would be that you change vdev_geom_attrchanged() to execute vdev_geom_set_rotating() explicitly only if the attribute matches, to ease merging. Does the change pass a STF run on a DEBUG build? Thanks! --Will. On Fri, Aug 16, 2013 at 10:17 AM, Steven Hartland wrote: > Baring any more feedback I intend to commit this soon, so please shout > if there's anything outstanding ;-) > > ----- Original Message ----- From: "Steven Hartland" > >> ----- Original Message ----- From: "Matthew Ahrens" >> >> >>> The platform-independent code in the patch looks good to me with one >>> exceptions: >>> >>> mm->mm_children = c--; >>> >>> It's non-obvious why c should be decremented *here*, as opposed to as >>> part >>> of the loop logic. If there's some trick going on with the loop then >>> it's >>> lost on me. >> >> >> This just configures 'c' so the loop starts with the last child, could >> split >> to a c--; on its own line if people prefer? >> >>> Does it matter that we go through the children in reverse >>> order? Is there something wrong with the existing loop logic: >>> >>> for (c = 0; c < mm->mm_children; c++) { >> >> >> Order is not important, but using a do{} while(); provides a small >> optimisation >> which avoids the first loop test totally and then on subisquent iterations >> compare against a constant not through a struct pointer. > > > Regards > Steve > > ================================================ > This e.mail is private and confidential between Multiplay (UK) Ltd. and the > person or entity to whom it is addressed. In the event of misdirection, the > recipient is prohibited from using, copying, printing or otherwise > disseminating it or any information contained in it. > In the event of misdirection, illegible or incomplete transmission please > telephone +44 845 868 1337 > or return the E.mail to postmaster@multiplay.co.uk. > > _______________________________________________ > zfs-devel@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/zfs-devel > To unsubscribe, send any mail to "zfs-devel-unsubscribe@freebsd.org" From owner-zfs-devel@FreeBSD.ORG Fri Aug 16 17:35:57 2013 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id EDC7E624; Fri, 16 Aug 2013 17:35:57 +0000 (UTC) (envelope-from prvs=1940958969=killing@multiplay.co.uk) Received: from mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) (using TLSv1 with cipher RC4-MD5 (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 677042658; Fri, 16 Aug 2013 17:35:56 +0000 (UTC) Received: from r2d2 ([82.69.141.170]) by mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) (MDaemon PRO v10.0.4) with ESMTP id md50005526369.msg; Fri, 16 Aug 2013 18:35:54 +0100 X-Spam-Processed: mail1.multiplay.co.uk, Fri, 16 Aug 2013 18:35:54 +0100 (not processed: message from valid local sender) X-MDDKIM-Result: neutral (mail1.multiplay.co.uk) X-MDRemoteIP: 82.69.141.170 X-Return-Path: prvs=1940958969=killing@multiplay.co.uk X-Envelope-From: killing@multiplay.co.uk Message-ID: <6A7D5E251D1C40EB9FD5987BB1465FCF@multiplay.co.uk> From: "Steven Hartland" To: "Will Andrews" References: <20130712112117.Horde.JMB3fhsU1VfZRxdV4prj5Q6@mail.vx.sk><51FE1E1E.8080502@FreeBSD.org><5C28F2D739D94E6B928ACD55D207665A@multiplay.co.uk><51FE9BF5.2000301@FreeBSD.org><51FEA530.2090008@FreeBSD.org><4B747F3B634B4A859068FC8D782A7E1D@multiplay.co.uk><52011DE6.8090708@FreeBSD.org><4DA6B66E61374BE68C895C8B9A93BCB7@multiplay.co.uk><520C9001.2040406@FreeBSD.org> Subject: Re: N-way mirror read speedup in zfsonlinux Date: Fri, 16 Aug 2013 18:36:15 +0100 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="UTF-8"; reply-type=original Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.5931 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.6157 Cc: Matthew Ahrens , Xin Li , zfs-devel@freebsd.org X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 Aug 2013 17:35:58 -0000 ----- Original Message ----- From: "Will Andrews" > We have a vdev_geom_attrchanged() in our tree as well, to support > physical path management (an early version of which is in the zfsd > project branch; commits to HEAD are still pending). So, my request > would be that you change vdev_geom_attrchanged() to execute > vdev_geom_set_rotating() explicitly only if the attribute matches, to > ease merging. Sure no problem done. > Does the change pass a STF run on a DEBUG build? I'll look at getting STF installed on this test box to confirm, do you have an updated version with the various issues I raised when testing it a while back? Regards Steve ================================================ This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmaster@multiplay.co.uk. From owner-zfs-devel@FreeBSD.ORG Fri Aug 16 22:15:02 2013 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 7112228E for ; Fri, 16 Aug 2013 22:15:02 +0000 (UTC) (envelope-from will@firepipe.net) Received: from mail-vb0-f43.google.com (mail-vb0-f43.google.com [209.85.212.43]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 2C2AC243D for ; Fri, 16 Aug 2013 22:15:01 +0000 (UTC) Received: by mail-vb0-f43.google.com with SMTP id h11so2045606vbh.30 for ; Fri, 16 Aug 2013 15:14:55 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=u/nYKFQvpz0aQ9uoT+R+fuxTaNBDlwoJlIJ3sAzm9Ms=; b=UIp5dCopD7ASFMbAn10b7pv1mfv+77QFbswIE5mPbg5gXjZU+yYXCr+TDcCMBPipyP js7q+R8UvouED800tU1Nnk+Atfu5t3ANwned+T5GpyAOIYaSvzdAM5mFPdY3wiLrJc/3 kkQzdV7SbVOjNJFLysm1LNmrTqhNPxPXg8ZtqM7/6Q5Lx8aqmwSkjMbDBNgVKRP/1dBL /KO5gQj0+TJzUT/g+2JPLgKAf6/vFePETEdUpKvePXhTi1YqYUPkL/ElavMUupQ06oOQ phX/gZl9wOi25ZbONiHO7V5+dkbtA2Tulr4Boi9DlPsudIjtFRNfNGuM/MWJ04YXaM9l 0Q2A== X-Gm-Message-State: ALoCoQndHFTLLYtxNwl0FZBC1poFPfi2qzXf71XHhB4QD5Drb7MntL5FywkELFoJWvKs1VB/jUAJ MIME-Version: 1.0 X-Received: by 10.221.56.194 with SMTP id wd2mr3205052vcb.7.1376691295511; Fri, 16 Aug 2013 15:14:55 -0700 (PDT) Received: by 10.58.226.66 with HTTP; Fri, 16 Aug 2013 15:14:55 -0700 (PDT) In-Reply-To: <6A7D5E251D1C40EB9FD5987BB1465FCF@multiplay.co.uk> References: <20130712112117.Horde.JMB3fhsU1VfZRxdV4prj5Q6@mail.vx.sk> <51FE1E1E.8080502@FreeBSD.org> <5C28F2D739D94E6B928ACD55D207665A@multiplay.co.uk> <51FE9BF5.2000301@FreeBSD.org> <51FEA530.2090008@FreeBSD.org> <4B747F3B634B4A859068FC8D782A7E1D@multiplay.co.uk> <52011DE6.8090708@FreeBSD.org> <4DA6B66E61374BE68C895C8B9A93BCB7@multiplay.co.uk> <520C9001.2040406@FreeBSD.org> <6A7D5E251D1C40EB9FD5987BB1465FCF@multiplay.co.uk> Date: Fri, 16 Aug 2013 16:14:55 -0600 Message-ID: Subject: Re: N-way mirror read speedup in zfsonlinux From: Will Andrews To: Steven Hartland Content-Type: text/plain; charset=UTF-8 Cc: Matthew Ahrens , Xin Li , zfs-devel@freebsd.org X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 Aug 2013 22:15:02 -0000 On Fri, Aug 16, 2013 at 11:36 AM, Steven Hartland wrote: >> We have a vdev_geom_attrchanged() in our tree as well, to support >> physical path management (an early version of which is in the zfsd >> project branch; commits to HEAD are still pending). So, my request >> would be that you change vdev_geom_attrchanged() to execute >> vdev_geom_set_rotating() explicitly only if the attribute matches, to >> ease merging. > > > Sure no problem done. Thanks, I've made a similar change locally. >> Does the change pass a STF run on a DEBUG build? > > > I'll look at getting STF installed on this test box to confirm, do you have > an updated version with the various issues I raised when testing it > a while back? I thought most if not all the issues you raised were due to differences in the ZFS implementation, not test case issues. We do have some updates for the test suite itself, but mostly in response to more fixes that have not been pushed out yet. I'm currently working to push out some of the fixes. Thanks. --Will. From owner-zfs-devel@FreeBSD.ORG Fri Aug 16 23:45:54 2013 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id AF4F5D4; Fri, 16 Aug 2013 23:45:54 +0000 (UTC) (envelope-from prvs=1940958969=killing@multiplay.co.uk) Received: from mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) (using TLSv1 with cipher RC4-MD5 (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 2090027ED; Fri, 16 Aug 2013 23:45:53 +0000 (UTC) Received: from r2d2 ([82.69.141.170]) by mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) (MDaemon PRO v10.0.4) with ESMTP id md50005530354.msg; Sat, 17 Aug 2013 00:45:46 +0100 X-Spam-Processed: mail1.multiplay.co.uk, Sat, 17 Aug 2013 00:45:46 +0100 (not processed: message from valid local sender) X-MDDKIM-Result: neutral (mail1.multiplay.co.uk) X-MDRemoteIP: 82.69.141.170 X-Return-Path: prvs=1940958969=killing@multiplay.co.uk X-Envelope-From: killing@multiplay.co.uk Message-ID: <385CBEC316C84F96985829DA757D8057@multiplay.co.uk> From: "Steven Hartland" To: "Will Andrews" References: <20130712112117.Horde.JMB3fhsU1VfZRxdV4prj5Q6@mail.vx.sk><51FE1E1E.8080502@FreeBSD.org><5C28F2D739D94E6B928ACD55D207665A@multiplay.co.uk><51FE9BF5.2000301@FreeBSD.org><51FEA530.2090008@FreeBSD.org><4B747F3B634B4A859068FC8D782A7E1D@multiplay.co.uk><52011DE6.8090708@FreeBSD.org><4DA6B66E61374BE68C895C8B9A93BCB7@multiplay.co.uk><520C9001.2040406@FreeBSD.org><6A7D5E251D1C40EB9FD5987BB1465FCF@multiplay.co.uk> Subject: Re: N-way mirror read speedup in zfsonlinux Date: Sat, 17 Aug 2013 00:46:06 +0100 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="UTF-8"; reply-type=original Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.5931 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.6157 Cc: Matthew Ahrens , Xin Li , zfs-devel@freebsd.org X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 Aug 2013 23:45:54 -0000 ----- Original Message ----- From: "Will Andrews" To: "Steven Hartland" Cc: "Matthew Ahrens" ; ; "Xin Li" Sent: Friday, August 16, 2013 11:14 PM Subject: Re: N-way mirror read speedup in zfsonlinux > On Fri, Aug 16, 2013 at 11:36 AM, Steven Hartland > wrote: >>> We have a vdev_geom_attrchanged() in our tree as well, to support >>> physical path management (an early version of which is in the zfsd >>> project branch; commits to HEAD are still pending). So, my request >>> would be that you change vdev_geom_attrchanged() to execute >>> vdev_geom_set_rotating() explicitly only if the attribute matches, to >>> ease merging. >> >> >> Sure no problem done. > > Thanks, I've made a similar change locally. > >>> Does the change pass a STF run on a DEBUG build? >> >> >> I'll look at getting STF installed on this test box to confirm, do you have >> an updated version with the various issues I raised when testing it >> a while back? > > I thought most if not all the issues you raised were due to > differences in the ZFS implementation, not test case issues. We do > have some updates for the test suite itself, but mostly in response to > more fixes that have not been pushed out yet. > > I'm currently working to push out some of the fixes. It looks like the version of STF no longer builds under head all I get is:- make MAKEOBJDIRPREFIX= Doing nothing for objlink echo "PWD=`pwd` STF_TOOLS="/usr/home/smh/freebsd/base/head/tools/regression/stc/src/tools"/stf/ CHECKENV="/usr/home/smh/freebsd/base/head/tools/regression/stc/src/tools"/checkenv/" PWD=/usr/obj/usr/home/smh/freebsd/base/head/tools/regression/stc STF_TOOLS=/usr/home/smh/freebsd/base/head/tools/regression/stc/src/tools/stf/ CHECKENV=/usr/home/smh/freebsd/base/head/tools/regression/stc/src/tools/checkenv/ make[1]: don't know how to make sh. Stop make[1]: stopped in /usr/home/smh/freebsd/base/head/tools/regression/stc/src/tools/stf *** Error code 2 Regards Steve Stop. make: stopped in /usr/home/smh/freebsd/base/head/tools/regression/stc ================================================ This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmaster@multiplay.co.uk. From owner-zfs-devel@FreeBSD.ORG Sat Aug 17 07:22:30 2013 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 5222F8C9; Sat, 17 Aug 2013 07:22:30 +0000 (UTC) (envelope-from gibbs@FreeBSD.org) Received: from aslan.scsiguy.com (aslan.scsiguy.com [70.89.174.89]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 158DE299D; Sat, 17 Aug 2013 07:22:29 +0000 (UTC) Received: from [192.168.0.99] (macbook.scsiguy.com [192.168.0.99]) (authenticated bits=0) by aslan.scsiguy.com (8.14.7/8.14.5) with ESMTP id r7H7MQ2N079638 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NO); Sat, 17 Aug 2013 07:22:27 GMT (envelope-from gibbs@FreeBSD.org) Subject: Re: N-way mirror read speedup in zfsonlinux Mime-Version: 1.0 (Mac OS X Mail 6.5 \(1508\)) Content-Type: multipart/mixed; boundary="Apple-Mail=_6ED084CA-F2B4-442B-BA80-57405113623F" From: "Justin T. Gibbs" X-Priority: 3 In-Reply-To: Date: Sat, 17 Aug 2013 01:22:26 -0600 Message-Id: <78BAB3D2-86A8-404C-8D5A-5B4E89AF1208@FreeBSD.org> References: <20130712112117.Horde.JMB3fhsU1VfZRxdV4prj5Q6@mail.vx.sk><51FE1E1E.8080502@FreeBSD.org><5C28F2D739D94E6B928ACD55D207665A@multiplay.co.uk><51FE9BF5.2000301@FreeBSD.org><51FEA530.2090008@FreeBSD.org><4B747F3B634B4A859068FC8D782A7E1D@multiplay.co.uk><52011DE6.8090708@FreeBSD.org><4DA6B66E61374BE68C895C8B9A93BCB7@multiplay.co.uk><520C9001.2040406@FreeBSD.org> To: Steven Hartland X-Mailer: Apple Mail (2.1508) X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.4.3 (aslan.scsiguy.com [70.89.174.89]); Sat, 17 Aug 2013 07:22:27 +0000 (UTC) Cc: Matthew Ahrens , Xin Li , zfs-devel@freebsd.org X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 17 Aug 2013 07:22:30 -0000 --Apple-Mail=_6ED084CA-F2B4-442B-BA80-57405113623F Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=us-ascii Hi Steve, First off, thanks for taking this on. It looks like it will pay high dividends. Here's my feedback: 1) For the GEOM::rotating attribute, and struct disk's d_nonrotating field, I think a rotational speed attribute would be more generically useful. For example, in Spectra's disk product, we use the rotational speed to help group disks into related classes. I would suggest using the SCSI/ATA conventions of 0 = unreported, 1 = non-rotating, N > 1 = reported RPM. 2) do {} while() loops have their place, but only if they improve code clarity and/or yield a significant performance benefit. I don't see either applying here. 3) Seeks do have a penalty for SSDs in the form of additional command overhead. If given the choice between SSDs of similar load, we should prefer one with perfect locality in the hope that vdev_queue will aggregate the I/O. I have my eye on adding "unmapped ZIOs" for use in vdev_queue so that we can avoid the data copy and aggregate beyond the current 128k limit. 4) mm_replacing can be deceiving. In a patch we have yet to upstream, we actually renamed it to mm_resilvering and set it by interrogate the dsl to see if a resilver task is active. Alan Somer's comment in the code describes part of the problem: /* * If we are resilvering, then we should handle scrub reads * differently; we shouldn't issue them to the resilvering * device because it might not have those blocks. * * We are resilvering iff: * 1) We are a replacing vdev (ie our name is "replacing-1" or * "spare-1" or something like that), and * 2) The pool is currently being resilvered. * * We cannot simply check vd->vdev_resilvering, because that * variable has never been correctly calculated. * * Nor can we just check our vdev_ops; there are cases (such as * when a user types "zpool replace pool odev spare_dev" and * spare_dev is in the spare list, or when a spare device is * automatically used to replace a DEGRADED device) when * resilvering is complete but both the original vdev and the * spare vdev remain in the pool. That behavior is intentional. * It helps implement the policy that a spare should be * automatically removed from the pool after the user replaces * the device that originally failed. */ Further complicating things is that when a scrub or resilver operation is active, not all reads will be "scrub reads". So if we want to be 100% correct here, we want to select the least loaded vdev that does not contain the block being read in its DTL, unless the read is a SCRUB read and resilvering is not active. 5) Even if we don't use load to determine the best device to use, we still want to update the last offset issued so that future calculations have correct information. 6) Can't vdev_queue maintain vq_lastoffset on its own? Having it in vdev_queue also avoids any future bugs in updating the wrong device (e.g. updating mm_preferred when mm_preferred isn't the only device read from or isn't read from at all due to the state of its DTL). Attached are updated patches (only for files I changed *after* applying your patch) for my progress so far in trying to address 2-6. I haven't done more than compile and boot test them so far, and it's way past my bedtime tonight. But I will perform more testing tomorrow and perhaps try to more fully address #4 and tune the parameters for #3. -- Justin --Apple-Mail=_6ED084CA-F2B4-442B-BA80-57405113623F Content-Disposition: attachment; filename=vdev_mirror.diffs Content-Type: application/octet-stream; x-unix-mode=0644; name="vdev_mirror.diffs" Content-Transfer-Encoding: 7bit --- //SpectraBSD/stable/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_mirror.c 2013-07-19 20:54:44.000000000 -0600 +++ /usr/home/justing/perforce/SpectraBSD/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_mirror.c 2013-07-19 20:54:44.000000000 -0600 @@ -44,6 +44,7 @@ vdev_t *mc_vd; uint64_t mc_offset; int mc_error; + int mc_load; uint8_t mc_tried; uint8_t mc_skipped; uint8_t mc_speculative; @@ -57,7 +58,41 @@ mirror_child_t mm_child[1]; } mirror_map_t; -int vdev_mirror_shift = 21; +SYSCTL_DECL(_vfs_zfs_vdev); +static SYSCTL_NODE(_vfs_zfs_vdev, OID_AUTO, mirror, CTLFLAG_RD, 0, + "ZFS VDEV Mirror"); + +/* Rotating media load calculation configuration. */ +static int rotating_inc = 0; +TUNABLE_INT("vfs.zfs.vdev.mirror.rotating_inc", &rotating_inc); +SYSCTL_INT(_vfs_zfs_vdev_mirror, OID_AUTO, rotating_inc, CTLFLAG_RW, + &rotating_inc, 0, "Rotating media load increment for non-seeking IO's"); + +static int rotating_seek_inc = 5; +TUNABLE_INT("vfs.zfs.vdev.mirror.rotating_seek_inc", &rotating_seek_inc); +SYSCTL_INT(_vfs_zfs_vdev_mirror, OID_AUTO, rotating_seek_inc, CTLFLAG_RW, + &rotating_seek_inc, 0, "Rotating media load increment for seeking IO's"); + +static int rotating_seek_offset = 1 * 1024 * 1024; +TUNABLE_INT("vfs.zfs.vdev.mirror.rotating_seek_offset", &rotating_seek_offset); +SYSCTL_INT(_vfs_zfs_vdev_mirror, OID_AUTO, rotating_seek_offset, CTLFLAG_RW, + &rotating_seek_offset, 0, "Offset in bytes from the last IO which " + "triggers a reduced rotating media seek increment"); + +/* Non-rotating media load calculation configuration. */ +static int non_rotating_inc = 0; +TUNABLE_INT("vfs.zfs.vdev.mirror.non_rotating_inc", &non_rotating_inc); +SYSCTL_INT(_vfs_zfs_vdev_mirror, OID_AUTO, non_rotating_inc, CTLFLAG_RW, + &non_rotating_inc, 0, "Non-rotating media load increment"); + +static int non_rotating_seek_inc = 1; +TUNABLE_INT("vfs.zfs.vdev.mirror.non_rotating_seek_inc", + &non_rotating_seek_inc); +SYSCTL_INT(_vfs_zfs_vdev_mirror, OID_AUTO, non_rotating_seek_inc, CTLFLAG_RW, + &non_rotating_seek_inc, 0, + "Non-rotating media load increment for seeking IO's"); + +static int vdev_mirror_shift = 21; static void vdev_mirror_map_free(zio_t *zio) @@ -72,6 +107,50 @@ zio_vsd_default_cksum_report }; +static int +vdev_mirror_load(vdev_t *vd, uint64_t zio_offset) +{ + uint64_t lastoffset; + int load; + + /* If the vdev isn't readable use the maximum load. */ + if (!vdev_readable(vd)) + return (INT_MAX); + + /* Standard load based on pending queue length. */ + load = vdev_queue_length(vd); + lastoffset = vdev_queue_lastoffset(vd); + + /* Non-rotating media. */ + if (!vd->vdev_rotating) { + if (lastoffset == zio_offset) + return (load + non_rotating_inc); + + /* + * Apply a minor seek penalty even for non-rotating devices. + * Sequential I/O can be aggregated into fewer operations + * on the device, thus avoiding unnecessary per-command + * overhead and boosting performance. + */ + return (load + non_rotating_seek_inc); + } + + /* IO's which directly follow the last IO. */ + if (lastoffset == zio_offset) + return (load + rotating_inc); + + /* + * Apply half the seek increment to IO's within seek offset + * of the last IO queued to this vdev as it should incure a + * smaller seek penalty or have the data in its track cache. + */ + if (ABS(lastoffset - zio_offset) < rotating_seek_offset) + return (load + (rotating_seek_inc / 2)); + + /* Apply the full seek increment to all other IO's. */ + return (load + rotating_seek_inc); +} + static mirror_map_t * vdev_mirror_map_alloc(zio_t *zio) { @@ -111,12 +190,13 @@ mc->mc_offset = DVA_GET_OFFSET(&dva[c]); } } else { + uint64_t zio_offset; + int lowest_load, lowest_child, lowest_nr; int replacing; c = vd->vdev_children; mm = kmem_zalloc(offsetof(mirror_map_t, mm_child[c]), KM_SLEEP); - mm->mm_children = c; /* * If we are resilvering, then we should handle scrub reads * differently; we shouldn't issue them to the resilvering @@ -156,15 +236,51 @@ mm->mm_resilvering = B_FALSE; } - mm->mm_preferred = mm->mm_resilvering ? 0 : - (zio->io_offset >> vdev_mirror_shift) % c; mm->mm_root = B_FALSE; + mm->mm_children = c; + zio_offset = zio->io_offset; + lowest_load = INT_MAX; + lowest_child = 0; + lowest_nr = 1; + for (c = 0; c < mm->mm_children; c++) { mc = &mm->mm_child[c]; mc->mc_vd = vd->vdev_child[c]; - mc->mc_offset = zio->io_offset; + mc->mc_offset = zio_offset; + + if (zio->io_type != ZIO_TYPE_READ) + continue; + + mc->mc_load = vdev_mirror_load(mc->mc_vd, zio_offset); + if (mc->mc_load < lowest_load) { + lowest_load = mc->mc_load; + lowest_child = c; + lowest_nr = 1; + } else if (mc->mc_load == lowest_load) { + lowest_nr++; + } + } + + if (lowest_nr != 1) { + int d; + + /* + * To ensure we don't always favour the first + * matching vdev, which could lead to wear + * leveling issues on SSD's, we use the IO + * offset as a sudo random seed into the vdevs + * which have the lowest load. + */ + d = (zio_offset >> vdev_mirror_shift) % lowest_nr; + for (c = lowest_child + 1; d != 0; c++) { + if (mm->mm_child[c].mc_load == lowest_load) { + lowest_child = c; + d--; + } + } } + mm->mm_preferred = lowest_child; } zio->io_vsd = mm; --- //SpectraBSD/stable/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c 2013-07-19 20:54:44.000000000 -0600 +++ /usr/home/justing/perforce/SpectraBSD/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c 2013-07-19 20:54:44.000000000 -0600 @@ -155,6 +155,8 @@ avl_create(&vq->vq_pending_tree, vdev_queue_offset_compare, sizeof (zio_t), offsetof(struct zio, io_offset_node)); + + vq->vq_lastoffset = 0; } void @@ -383,6 +385,9 @@ ASSERT(zio->io_type == ZIO_TYPE_READ || zio->io_type == ZIO_TYPE_WRITE); + if (zio->io_type == ZIO_TYPE_READ) + vq->vq_lastoffset = zio->io_offset + zio->io_size; + if (zio->io_flags & ZIO_FLAG_DONT_QUEUE) return (zio); @@ -446,3 +451,21 @@ mutex_exit(&vq->vq_lock); } + +/* + * As these two methods are only used for load calculations we're not + * concerned if we get an incorrect value on 32bit platforms due to lack + * of vq_lock mutex uses here. Instead we prefer to keep it lock free for + * performance. + */ +int +vdev_queue_length(vdev_t *vd) +{ + return (avl_numnodes(&vd->vdev_queue.vq_pending_tree)); +} + +uint64_t +vdev_queue_lastoffset(vdev_t *vd) +{ + return (vd->vdev_queue.vq_lastoffset); +} --Apple-Mail=_6ED084CA-F2B4-442B-BA80-57405113623F Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii On Aug 16, 2013, at 10:17 AM, Steven Hartland = wrote: > Baring any more feedback I intend to commit this soon, so please shout > if there's anything outstanding ;-) >=20 > ----- Original Message ----- From: "Steven Hartland" >> ----- Original Message ----- From: "Matthew Ahrens" = >>> The platform-independent code in the patch looks good to me with one >>> exceptions: >>> mm->mm_children =3D c--; >>>=20 >>> It's non-obvious why c should be decremented *here*, as opposed to = as part >>> of the loop logic. If there's some trick going on with the loop = then it's >>> lost on me. >> This just configures 'c' so the loop starts with the last child, = could split >> to a c--; on its own line if people prefer? >>> Does it matter that we go through the children in reverse >>> order? Is there something wrong with the existing loop logic: >>> for (c =3D 0; c < mm->mm_children; c++) { >> Order is not important, but using a do{} while(); provides a small = optimisation >> which avoids the first loop test totally and then on subisquent = iterations >> compare against a constant not through a struct pointer. >=20 > Regards > Steve >=20 > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > This e.mail is private and confidential between Multiplay (UK) Ltd. = and the person or entity to whom it is addressed. In the event of = misdirection, the recipient is prohibited from using, copying, printing = or otherwise disseminating it or any information contained in it.=20 > In the event of misdirection, illegible or incomplete transmission = please telephone +44 845 868 1337 > or return the E.mail to postmaster@multiplay.co.uk. >=20 > _______________________________________________ > zfs-devel@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/zfs-devel > To unsubscribe, send any mail to "zfs-devel-unsubscribe@freebsd.org" >=20 --Apple-Mail=_6ED084CA-F2B4-442B-BA80-57405113623F-- From owner-zfs-devel@FreeBSD.ORG Sun Aug 18 03:08:12 2013 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id A3C75A81; Sun, 18 Aug 2013 03:08:12 +0000 (UTC) (envelope-from prvs=1942b2d7cb=killing@multiplay.co.uk) Received: from mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) (using TLSv1 with cipher RC4-MD5 (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 9116C2ABD; Sun, 18 Aug 2013 03:08:11 +0000 (UTC) Received: from r2d2 ([82.69.141.170]) by mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) (MDaemon PRO v10.0.4) with ESMTP id md50005549402.msg; Sun, 18 Aug 2013 04:08:03 +0100 X-Spam-Processed: mail1.multiplay.co.uk, Sun, 18 Aug 2013 04:08:03 +0100 (not processed: message from valid local sender) X-MDDKIM-Result: neutral (mail1.multiplay.co.uk) X-MDRemoteIP: 82.69.141.170 X-Return-Path: prvs=1942b2d7cb=killing@multiplay.co.uk X-Envelope-From: killing@multiplay.co.uk Message-ID: <12BBEDC5C5AF4F8180D6A6B09AD695A6@multiplay.co.uk> From: "Steven Hartland" To: "Justin T. Gibbs" References: <20130712112117.Horde.JMB3fhsU1VfZRxdV4prj5Q6@mail.vx.sk><51FE1E1E.8080502@FreeBSD.org><5C28F2D739D94E6B928ACD55D207665A@multiplay.co.uk><51FE9BF5.2000301@FreeBSD.org><51FEA530.2090008@FreeBSD.org><4B747F3B634B4A859068FC8D782A7E1D@multiplay.co.uk><52011DE6.8090708@FreeBSD.org><4DA6B66E61374BE68C895C8B9A93BCB7@multiplay.co.uk><520C9001.2040406@FreeBSD.org> <78BAB3D2-86A8-404C-8D5A-5B4E89AF1208@FreeBSD.org> Subject: Re: N-way mirror read speedup in zfsonlinux Date: Sun, 18 Aug 2013 04:08:22 +0100 MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="----=_NextPart_000_0417_01CE9BC8.94502D50" X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.5931 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.6157 Cc: Matthew Ahrens , Xin Li , zfs-devel@freebsd.org X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 18 Aug 2013 03:08:12 -0000 This is a multi-part message in MIME format. ------=_NextPart_000_0417_01CE9BC8.94502D50 Content-Type: text/plain; format=flowed; charset="iso-8859-1"; reply-type=original Content-Transfer-Encoding: 7bit Thanks for the feedback Justin, comments inline below. ----- Original Message ----- From: "Justin T. Gibbs" > First off, thanks for taking this on. It looks like it will pay > high dividends. > > Here's my feedback: > > 1) For the GEOM::rotating attribute, and struct disk's d_nonrotating > field, I think a rotational speed attribute would be more > generically useful. For example, in Spectra's disk product, we > use the rotational speed to help group disks into related classes. > I would suggest using the SCSI/ATA conventions of 0 = unreported, > 1 = non-rotating, N > 1 = reported RPM. I like this, it provides more flexibility for choices morning forward so I've changed from d_nonrotating -> d_rotation_rate. > 2) do {} while() loops have their place, but only if they improve > code clarity and/or yield a significant performance benefit. I > don't see either applying here. I'd have to disagree, unless using it makes it difficult to understand, a do while should be used is when you know there's always going to be one iteration of the loop and hence no need to test before performing the first iteration. This is the case when calculating the load value, as a mirror must have at least two vdevs. The case for lowest_nr is always one comparison anyway so this should be a for loop, changed. > 3) Seeks do have a penalty for SSDs in the form of additional command > overhead. If given the choice between SSDs of similar load, we > should prefer one with perfect locality in the hope that vdev_queue > will aggregate the I/O. I have my eye on adding "unmapped ZIOs" for > use in vdev_queue so that we can avoid the data copy and aggregate > beyond the current 128k limit. Sounds sensible, especially as it gives a more quantifiable reason for choosing one non-rotating device over another, instead of relying on random choice. It does however produce a noticable drop in performance for the mixed drive case, in my setup here 284MB/s down to 279MB/s, so I've documented this in the comments. I need to confirm it doesn't cause any generic performance drops, but need to switch my test rig round to all SSD's to do that. > 4) mm_replacing can be deceiving. In a patch we have yet to upstream, > we actually renamed it to mm_resilvering and set it by interrogate > the dsl to see if a resilver task is active. Alan Somer's comment in the > code describes part of the problem: > > /* > * If we are resilvering, then we should handle scrub reads > * differently; we shouldn't issue them to the resilvering > * device because it might not have those blocks. > * > * We are resilvering iff: > * 1) We are a replacing vdev (ie our name is "replacing-1" or > * "spare-1" or something like that), and > * 2) The pool is currently being resilvered. > * > * We cannot simply check vd->vdev_resilvering, because that > * variable has never been correctly calculated. > * > * Nor can we just check our vdev_ops; there are cases (such as > * when a user types "zpool replace pool odev spare_dev" and > * spare_dev is in the spare list, or when a spare device is > * automatically used to replace a DEGRADED device) when > * resilvering is complete but both the original vdev and the > * spare vdev remain in the pool. That behavior is intentional. > * It helps implement the policy that a spare should be > * automatically removed from the pool after the user replaces > * the device that originally failed. > */ > > Further complicating things is that when a scrub or resilver > operation is active, not all reads will be "scrub reads". So > if we want to be 100% correct here, we want to select the least > loaded vdev that does not contain the block being read in its > DTL, unless the read is a SCRUB read and resilvering is not > active. Removing mm_replacing checks resulted in zero measurable performance difference in my tests, so given that plus your description I see no real reason to keep it. I tried making vdev_mirror_load return INT_MAX when mm_vdev_resilver_txg != 0 (when resilvering ) but that actually resulted in slightly worse performance in the resilvering case, so noted in a comment and removed. > 5) Even if we don't use load to determine the best device to use, > we still want to update the last offset issued so that future > calculations have correct information. Good catch, changed. > 6) Can't vdev_queue maintain vq_lastoffset on its own? Having it > in vdev_queue also avoids any future bugs in updating the wrong > device (e.g. updating mm_preferred when mm_preferred isn't the > only device read from or isn't read from at all due to the state > of its DTL). I actually tried this before but it produces significantly worse results, 236MB/s instead of 286MB/s on my 3 disk setup. I suspect its because doing it there means it happens too late for it to have the full effect. One of my initial attempts also used the tree queue directly instead of storing a seperate value but that too produced poorer results. > Attached are updated patches (only for files I changed *after* > applying your patch) for my progress so far in trying to address > 2-6. I haven't done more than compile and boot test them so far, > and it's way past my bedtime tonight. But I will perform more > testing tomorrow and perhaps try to more fully address #4 and tune > the parameters for #3. Updated patch attached. Regards Steve ================================================ This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmaster@multiplay.co.uk. ------=_NextPart_000_0417_01CE9BC8.94502D50 Content-Type: application/octet-stream; name="zfs-mirror-load.patch" Content-Transfer-Encoding: quoted-printable Content-Disposition: attachment; filename="zfs-mirror-load.patch" Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/vdev.h=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/vdev.h (revision = 254419)=0A= +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/vdev.h (working = copy)=0A= @@ -119,6 +119,9 @@=0A= extern void vdev_queue_fini(vdev_t *vd);=0A= extern zio_t *vdev_queue_io(zio_t *zio);=0A= extern void vdev_queue_io_done(zio_t *zio);=0A= +extern int vdev_queue_length(vdev_t *vd);=0A= +extern uint64_t vdev_queue_lastoffset(vdev_t *vd);=0A= +extern void vdev_queue_register_lastoffset(vdev_t *vd, zio_t *zio);=0A= =0A= extern void vdev_config_dirty(vdev_t *vd);=0A= extern void vdev_config_clean(vdev_t *vd);=0A= Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/vdev_impl.h=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/vdev_impl.h = (revision 254419)=0A= +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/vdev_impl.h = (working copy)=0A= @@ -106,6 +106,7 @@=0A= avl_tree_t vq_pending_tree;=0A= hrtime_t vq_io_complete_ts;=0A= kmutex_t vq_lock;=0A= + uint64_t vq_lastoffset;=0A= };=0A= =0A= /*=0A= @@ -199,7 +200,10 @@=0A= spa_aux_vdev_t *vdev_aux; /* for l2cache vdevs */=0A= zio_t *vdev_probe_zio; /* root of current probe */=0A= vdev_aux_t vdev_label_aux; /* on-disk aux state */=0A= - struct trim_map *vdev_trimmap;=0A= + struct trim_map *vdev_trimmap; /* map on outstanding trims */ =0A= + uint16_t vdev_rotation_rate; /* rotational rate of the media */=0A= +#define VDEV_RATE_UNKNOWN 0=0A= +#define VDEV_RATE_NON_ROTATING 1=0A= =0A= /*=0A= * For DTrace to work in userland (libzpool) context, these fields must=0A= Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_geom.c=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_geom.c (revision = 254419)=0A= +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_geom.c (working = copy)=0A= @@ -42,9 +42,11 @@=0A= * Virtual device vector for GEOM.=0A= */=0A= =0A= +static g_attrchanged_t vdev_geom_attrchanged;=0A= struct g_class zfs_vdev_class =3D {=0A= .name =3D "ZFS::VDEV",=0A= .version =3D G_VERSION,=0A= + .attrchanged =3D vdev_geom_attrchanged,=0A= };=0A= =0A= DECLARE_GEOM_CLASS(zfs_vdev_class, zfs_vdev);=0A= @@ -62,6 +64,34 @@=0A= &vdev_geom_bio_delete_disable, 0, "Disable BIO_DELETE");=0A= =0A= static void=0A= +vdev_geom_set_rotation_rate(vdev_t *vd, struct g_consumer *cp)=0A= +{ =0A= + int error;=0A= + uint16_t rate;=0A= +=0A= + error =3D g_getattr("GEOM::rotation_rate", cp, &rate);=0A= + if (error =3D=3D 0)=0A= + vd->vdev_rotation_rate =3D rate;=0A= + else=0A= + vd->vdev_rotation_rate =3D VDEV_RATE_UNKNOWN;=0A= +}=0A= +=0A= +static void=0A= +vdev_geom_attrchanged(struct g_consumer *cp, const char *attr)=0A= +{=0A= + vdev_t *vd;=0A= +=0A= + vd =3D cp->private;=0A= + if (vd =3D=3D NULL)=0A= + return;=0A= +=0A= + if (strcmp(attr, "GEOM::rotation_rate") =3D=3D 0) {=0A= + vdev_geom_set_rotation_rate(vd, cp);=0A= + return;=0A= + }=0A= +}=0A= +=0A= +static void=0A= vdev_geom_orphan(struct g_consumer *cp)=0A= {=0A= vdev_t *vd;=0A= @@ -678,6 +708,11 @@=0A= vd->vdev_physpath =3D kmem_alloc(bufsize, KM_SLEEP);=0A= snprintf(vd->vdev_physpath, bufsize, "/dev/%s", pp->name);=0A= =0A= + /*=0A= + * Determine the device's rotation rate.=0A= + */=0A= + vdev_geom_set_rotation_rate(vd, cp);=0A= +=0A= return (0);=0A= }=0A= =0A= Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_mirror.c=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_mirror.c = (revision 254419)=0A= +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_mirror.c = (working copy)=0A= @@ -41,6 +41,7 @@=0A= vdev_t *mc_vd;=0A= uint64_t mc_offset;=0A= int mc_error;=0A= + int mc_load;=0A= uint8_t mc_tried;=0A= uint8_t mc_skipped;=0A= uint8_t mc_speculative;=0A= @@ -54,8 +55,53 @@=0A= mirror_child_t mm_child[1];=0A= } mirror_map_t;=0A= =0A= -int vdev_mirror_shift =3D 21;=0A= +SYSCTL_DECL(_vfs_zfs_vdev);=0A= +static SYSCTL_NODE(_vfs_zfs_vdev, OID_AUTO, mirror, CTLFLAG_RD, 0,=0A= + "ZFS VDEV Mirror");=0A= =0A= +/*=0A= + * The load configuration settings below are tuned by default for=0A= + * the case where all devices are of the same rotational type.=0A= + *=0A= + * If there is a mixture of rotating and non-rotating media, setting=0A= + * non_rotating_seek_inc to 0 may well provide better results as it=0A= + * will direct more reads to the non-rotating vdevs which are more=0A= + * likely to have a higher performance.=0A= + */=0A= +=0A= +/* Rotating media load calculation configuration. */=0A= +static int rotating_inc =3D 0;=0A= +TUNABLE_INT("vfs.zfs.vdev.mirror.rotating_inc", &rotating_inc);=0A= +SYSCTL_INT(_vfs_zfs_vdev_mirror, OID_AUTO, rotating_inc, CTLFLAG_RW,=0A= + &rotating_inc, 0, "Rotating media load increment for non-seeking = I/O's");=0A= +=0A= +static int rotating_seek_inc =3D 5;=0A= +TUNABLE_INT("vfs.zfs.vdev.mirror.rotating_seek_inc", = &rotating_seek_inc);=0A= +SYSCTL_INT(_vfs_zfs_vdev_mirror, OID_AUTO, rotating_seek_inc, = CTLFLAG_RW,=0A= + &rotating_seek_inc, 0, "Rotating media load increment for seeking = I/O's");=0A= +=0A= +static int rotating_seek_offset =3D 1 * 1024 * 1024;=0A= +TUNABLE_INT("vfs.zfs.vdev.mirror.rotating_seek_offset", = &rotating_seek_offset);=0A= +SYSCTL_INT(_vfs_zfs_vdev_mirror, OID_AUTO, rotating_seek_offset, = CTLFLAG_RW,=0A= + &rotating_seek_offset, 0, "Offset in bytes from the last I/O which "=0A= + "triggers a reduced rotating media seek increment");=0A= +=0A= +/* Non-rotating media load calculation configuration. */=0A= +static int non_rotating_inc =3D 0;=0A= +TUNABLE_INT("vfs.zfs.vdev.mirror.non_rotating_inc", &non_rotating_inc);=0A= +SYSCTL_INT(_vfs_zfs_vdev_mirror, OID_AUTO, non_rotating_inc, CTLFLAG_RW,=0A= + &non_rotating_inc, 0,=0A= + "Non-rotating media load increment for non-seeking I/O's");=0A= +=0A= +static int non_rotating_seek_inc =3D 1;=0A= +TUNABLE_INT("vfs.zfs.vdev.mirror.non_rotating_seek_inc",=0A= + &non_rotating_seek_inc);=0A= +SYSCTL_INT(_vfs_zfs_vdev_mirror, OID_AUTO, non_rotating_seek_inc, = CTLFLAG_RW,=0A= + &non_rotating_seek_inc, 0,=0A= + "Non-rotating media load increment for seeking I/O's");=0A= +=0A= +static int vdev_mirror_shift =3D 21;=0A= +=0A= static void=0A= vdev_mirror_map_free(zio_t *zio)=0A= {=0A= @@ -69,6 +115,56 @@=0A= zio_vsd_default_cksum_report=0A= };=0A= =0A= +static int=0A= +vdev_mirror_load(vdev_t *vd, uint64_t zio_offset)=0A= +{=0A= + uint64_t lastoffset;=0A= + int load;=0A= +=0A= + /*=0A= + * We don't return INT_MAX if the device is resilvering i.e.=0A= + * vdev_resilver_txg !=3D 0 as when tested performance was slightly=0A= + * worse overall when resilvering with compared to without.=0A= + */=0A= +=0A= + /* If the vdev isn't readable use the maximum load. */=0A= + if (!vdev_readable(vd))=0A= + return (INT_MAX);=0A= +=0A= + /* Standard load based on pending queue length. */=0A= + load =3D vdev_queue_length(vd);=0A= + lastoffset =3D vdev_queue_lastoffset(vd);=0A= +=0A= + /* Non-rotating media. */=0A= + if (vd->vdev_rotation_rate =3D=3D VDEV_RATE_NON_ROTATING) {=0A= + if (lastoffset =3D=3D zio_offset)=0A= + return (load + non_rotating_inc);=0A= +=0A= + /*=0A= + * Apply a seek penalty even for non-rotating devices as=0A= + * sequential I/O'a can be aggregated into fewer operations=0A= + * on the device, thus avoiding unnecessary per-command=0A= + * overhead and boosting performance.=0A= + */=0A= + return (load + non_rotating_seek_inc);=0A= + }=0A= +=0A= + /* I/O's which directly follow the last I/O. */=0A= + if (lastoffset =3D=3D zio_offset)=0A= + return (load + rotating_inc);=0A= +=0A= + /*=0A= + * Apply half the seek increment to I/O's within seek offset=0A= + * of the last I/O queued to this vdev as they should incure less=0A= + * of a seek increment.=0A= + */=0A= + if (ABS(lastoffset - zio_offset) < rotating_seek_offset)=0A= + return (load + (rotating_seek_inc / 2));=0A= +=0A= + /* Apply the full seek increment to all other I/O's. */=0A= + return (load + rotating_seek_inc);=0A= +}=0A= +=0A= static mirror_map_t *=0A= vdev_mirror_map_alloc(zio_t *zio)=0A= {=0A= @@ -108,21 +204,62 @@=0A= mc->mc_offset =3D DVA_GET_OFFSET(&dva[c]);=0A= }=0A= } else {=0A= + int lowest_load, lowest_child, lowest_nr;=0A= + uint64_t zio_offset;=0A= +=0A= c =3D vd->vdev_children;=0A= =0A= mm =3D kmem_zalloc(offsetof(mirror_map_t, mm_child[c]), KM_SLEEP);=0A= - mm->mm_children =3D c;=0A= + mm->mm_children =3D c--;=0A= mm->mm_replacing =3D (vd->vdev_ops =3D=3D &vdev_replacing_ops ||=0A= vd->vdev_ops =3D=3D &vdev_spare_ops);=0A= - mm->mm_preferred =3D mm->mm_replacing ? 0 :=0A= - (zio->io_offset >> vdev_mirror_shift) % c;=0A= mm->mm_root =3D B_FALSE;=0A= =0A= - for (c =3D 0; c < mm->mm_children; c++) {=0A= + zio_offset =3D zio->io_offset;=0A= + lowest_load =3D INT_MAX;=0A= + lowest_child =3D 0;=0A= + lowest_nr =3D 1;=0A= +=0A= + do {=0A= mc =3D &mm->mm_child[c];=0A= mc->mc_vd =3D vd->vdev_child[c];=0A= - mc->mc_offset =3D zio->io_offset;=0A= + mc->mc_offset =3D zio_offset;=0A= +=0A= + if (zio->io_type !=3D ZIO_TYPE_READ)=0A= + continue;=0A= +=0A= + mc->mc_load =3D vdev_mirror_load(mc->mc_vd, zio_offset);=0A= + if (mc->mc_load < lowest_load) {=0A= + lowest_load =3D mc->mc_load;=0A= + lowest_child =3D c; =0A= + lowest_nr =3D 1;=0A= + } else if (mc->mc_load =3D=3D lowest_load) {=0A= + lowest_nr++;=0A= + }=0A= + } while (--c >=3D 0);=0A= +=0A= + if (lowest_nr !=3D 1) {=0A= + int d;=0A= +=0A= + /*=0A= + * To ensure we don't always favour the first=0A= + * matching vdev, which could lead to wear=0A= + * leveling issues on SSD's, we use the I/O=0A= + * offset as a sudo random seed into the vdevs=0A= + * which have the lowest load.=0A= + */=0A= + d =3D (zio_offset >> vdev_mirror_shift)=0A= + % lowest_nr;=0A= + for (c =3D lowest_child - 1; d !=3D 0; c--) {=0A= + if (mm->mm_child[c].mc_load =3D=3D lowest_load) {=0A= + lowest_child =3D c;=0A= + d--;=0A= + }=0A= + }=0A= }=0A= + mm->mm_preferred =3D lowest_child;=0A= + vdev_queue_register_lastoffset(vd->vdev_child[lowest_child],=0A= + zio);=0A= }=0A= =0A= zio->io_vsd =3D mm;=0A= Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c = (revision 254419)=0A= +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c (working = copy)=0A= @@ -155,6 +155,8 @@=0A= =0A= avl_create(&vq->vq_pending_tree, vdev_queue_offset_compare,=0A= sizeof (zio_t), offsetof(struct zio, io_offset_node));=0A= +=0A= + vq->vq_lastoffset =3D 0;=0A= }=0A= =0A= void=0A= @@ -446,3 +448,26 @@=0A= =0A= mutex_exit(&vq->vq_lock);=0A= }=0A= +=0A= +/*=0A= + * As these three methods are only used for load calculations we're not = precious=0A= + * if we get an incorrect value on 32bit platforms due to lack of = vq_lock mutex=0A= + * uses here. Instead we prefer to keep it lock free for the = performance.=0A= + */ =0A= +int=0A= +vdev_queue_length(vdev_t *vd)=0A= +{=0A= + return (avl_numnodes(&vd->vdev_queue.vq_pending_tree));=0A= +}=0A= +=0A= +uint64_t=0A= +vdev_queue_lastoffset(vdev_t *vd)=0A= +{=0A= + return (vd->vdev_queue.vq_lastoffset);=0A= +}=0A= +=0A= +void=0A= +vdev_queue_register_lastoffset(vdev_t *vd, zio_t *zio)=0A= +{=0A= + vd->vdev_queue.vq_lastoffset =3D zio->io_offset + zio->io_size;=0A= +}=0A= Index: sys/geom/geom.h=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/geom/geom.h (revision 254419)=0A= +++ sys/geom/geom.h (working copy)=0A= @@ -270,6 +270,7 @@=0A= int len);=0A= int g_handleattr_int(struct bio *bp, const char *attribute, int val);=0A= int g_handleattr_off_t(struct bio *bp, const char *attribute, off_t = val);=0A= +int g_handleattr_uint16_t(struct bio *bp, const char *attribute, = uint16_t val);=0A= int g_handleattr_str(struct bio *bp, const char *attribute, const char = *str);=0A= struct g_consumer * g_new_consumer(struct g_geom *gp);=0A= struct g_geom * g_new_geomf(struct g_class *mp, const char *fmt, ...)=0A= Index: sys/geom/geom_disk.c=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/geom/geom_disk.c (revision 254419)=0A= +++ sys/geom/geom_disk.c (working copy)=0A= @@ -376,22 +376,25 @@=0A= break;=0A= else if (g_handleattr_str(bp, "GEOM::ident", dp->d_ident))=0A= break;=0A= - else if (g_handleattr(bp, "GEOM::hba_vendor",=0A= - &dp->d_hba_vendor, 2))=0A= + else if (g_handleattr_uint16_t(bp, "GEOM::hba_vendor",=0A= + dp->d_hba_vendor))=0A= break;=0A= - else if (g_handleattr(bp, "GEOM::hba_device",=0A= - &dp->d_hba_device, 2))=0A= + else if (g_handleattr_uint16_t(bp, "GEOM::hba_device",=0A= + dp->d_hba_device))=0A= break;=0A= - else if (g_handleattr(bp, "GEOM::hba_subvendor",=0A= - &dp->d_hba_subvendor, 2))=0A= + else if (g_handleattr_uint16_t(bp, "GEOM::hba_subvendor",=0A= + dp->d_hba_subvendor))=0A= break;=0A= - else if (g_handleattr(bp, "GEOM::hba_subdevice",=0A= - &dp->d_hba_subdevice, 2))=0A= + else if (g_handleattr_uint16_t(bp, "GEOM::hba_subdevice",=0A= + dp->d_hba_subdevice))=0A= break;=0A= else if (!strcmp(bp->bio_attribute, "GEOM::kerneldump"))=0A= g_disk_kerneldump(bp, dp);=0A= else if (!strcmp(bp->bio_attribute, "GEOM::setstate"))=0A= g_disk_setstate(bp, sc);=0A= + else if (g_handleattr_uint16_t(bp, "GEOM::rotation_rate",=0A= + dp->d_rotation_rate))=0A= + break;=0A= else =0A= error =3D ENOIOCTL;=0A= break;=0A= Index: sys/geom/geom_disk.h=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/geom/geom_disk.h (revision 254419)=0A= +++ sys/geom/geom_disk.h (working copy)=0A= @@ -97,6 +97,7 @@=0A= uint16_t d_hba_device;=0A= uint16_t d_hba_subvendor;=0A= uint16_t d_hba_subdevice;=0A= + uint16_t d_rotation_rate;=0A= =0A= /* Fields private to the driver */=0A= void *d_drv1;=0A= @@ -121,7 +122,8 @@=0A= #define DISK_VERSION_01 0x5856105a=0A= #define DISK_VERSION_02 0x5856105b=0A= #define DISK_VERSION_03 0x5856105c=0A= -#define DISK_VERSION DISK_VERSION_03=0A= +#define DISK_VERSION_04 0x5856105d=0A= +#define DISK_VERSION DISK_VERSION_04=0A= =0A= #endif /* _KERNEL */=0A= #endif /* _GEOM_GEOM_DISK_H_ */=0A= Index: sys/geom/geom_subr.c=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/geom/geom_subr.c (revision 254419)=0A= +++ sys/geom/geom_subr.c (working copy)=0A= @@ -949,6 +949,13 @@=0A= }=0A= =0A= int=0A= +g_handleattr_uint16_t(struct bio *bp, const char *attribute, uint16_t = val)=0A= +{=0A= +=0A= + return (g_handleattr(bp, attribute, &val, sizeof val));=0A= +}=0A= +=0A= +int=0A= g_handleattr_off_t(struct bio *bp, const char *attribute, off_t val)=0A= {=0A= =0A= Index: sys/cam/ata/ata_da.c=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/cam/ata/ata_da.c (revision 254419)=0A= +++ sys/cam/ata/ata_da.c (working copy)=0A= @@ -1224,13 +1224,14 @@=0A= snprintf(announce_buf, sizeof(announce_buf),=0A= "kern.cam.ada.%d.write_cache", periph->unit_number);=0A= TUNABLE_INT_FETCH(announce_buf, &softc->write_cache);=0A= + adagetparams(periph, cgd);=0A= + softc->disk =3D disk_alloc();=0A= /* Disable queue sorting for non-rotational media by default. */=0A= - if (cgd->ident_data.media_rotation_rate =3D=3D 1)=0A= + if (cgd->ident_data.media_rotation_rate =3D=3D ATA_RATE_NON_ROTATING)=0A= softc->sort_io_queue =3D 0;=0A= else=0A= softc->sort_io_queue =3D -1;=0A= - adagetparams(periph, cgd);=0A= - softc->disk =3D disk_alloc();=0A= + softc->disk->d_rotation_rate =3D cgd->ident_data.media_rotation_rate;=0A= softc->disk->d_devstat =3D devstat_new_entry(periph->periph_name,=0A= periph->unit_number, softc->params.secsize,=0A= DEVSTAT_ALL_SUPPORTED,=0A= Index: sys/cam/scsi/scsi_all.h=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/cam/scsi/scsi_all.h (revision 254419)=0A= +++ sys/cam/scsi/scsi_all.h (working copy)=0A= @@ -1451,7 +1451,7 @@=0A= u_int8_t page_length[2];=0A= u_int8_t medium_rotation_rate[2];=0A= #define SVPD_BDC_RATE_NOT_REPORTED 0x00=0A= -#define SVPD_BDC_RATE_NONE_ROTATING 0x01=0A= +#define SVPD_BDC_RATE_NON_ROTATING 0x01=0A= u_int8_t reserved1;=0A= u_int8_t nominal_form_factor;=0A= #define SVPD_BDC_FORM_NOT_REPORTED 0x00=0A= Index: sys/cam/scsi/scsi_da.c=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/cam/scsi/scsi_da.c (revision 254419)=0A= +++ sys/cam/scsi/scsi_da.c (working copy)=0A= @@ -3386,9 +3386,17 @@=0A= * Disable queue sorting for non-rotational media=0A= * by default.=0A= */=0A= - if (scsi_2btoul(bdc->medium_rotation_rate) =3D=3D=0A= - SVPD_BDC_RATE_NONE_ROTATING)=0A= + u_int old_rate =3D softc->disk->d_rotation_rate;=0A= + softc->disk->d_rotation_rate =3D=0A= + scsi_2btoul(bdc->medium_rotation_rate);=0A= + if (softc->disk->d_rotation_rate =3D=3D=0A= + SVPD_BDC_RATE_NON_ROTATING) {=0A= softc->sort_io_queue =3D 0;=0A= + }=0A= + if (softc->disk->d_rotation_rate !=3D old_rate) {=0A= + disk_attr_changed(softc->disk,=0A= + "GEOM::rotation_rate", M_NOWAIT);=0A= + }=0A= } else {=0A= int error;=0A= error =3D daerror(done_ccb, CAM_RETRY_SELTO,=0A= @@ -3423,6 +3431,8 @@=0A= ptr =3D (uint16_t *)ata_params;=0A= =0A= if ((csio->ccb_h.status & CAM_STATUS_MASK) =3D=3D CAM_REQ_CMP) {=0A= + uint16_t old_rate;=0A= +=0A= for (i =3D 0; i < sizeof(*ata_params) / 2; i++)=0A= ptr[i] =3D le16toh(ptr[i]);=0A= if (ata_params->support_dsm & ATA_SUPPORT_DSM_TRIM) {=0A= @@ -3437,8 +3447,18 @@=0A= * Disable queue sorting for non-rotational media=0A= * by default.=0A= */=0A= - if (ata_params->media_rotation_rate =3D=3D 1)=0A= + old_rate =3D softc->disk->d_rotation_rate;=0A= + softc->disk->d_rotation_rate =3D=0A= + ata_params->media_rotation_rate;=0A= + if (softc->disk->d_rotation_rate =3D=3D=0A= + ATA_RATE_NON_ROTATING) {=0A= softc->sort_io_queue =3D 0;=0A= + }=0A= +=0A= + if (softc->disk->d_rotation_rate !=3D old_rate) {=0A= + disk_attr_changed(softc->disk,=0A= + "GEOM::rotation_rate", M_NOWAIT);=0A= + }=0A= } else {=0A= int error;=0A= error =3D daerror(done_ccb, CAM_RETRY_SELTO,=0A= ------=_NextPart_000_0417_01CE9BC8.94502D50-- From owner-zfs-devel@FreeBSD.ORG Sun Aug 18 05:06:28 2013 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id C02C6563 for ; Sun, 18 Aug 2013 05:06:28 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: from mail-pa0-x22a.google.com (mail-pa0-x22a.google.com [IPv6:2607:f8b0:400e:c03::22a]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 853CB2EB7 for ; Sun, 18 Aug 2013 05:06:28 +0000 (UTC) Received: by mail-pa0-f42.google.com with SMTP id lj1so3437475pab.15 for ; Sat, 17 Aug 2013 22:06:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=delphix.com; s=google; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=hFd3YSHbBuX9q1pecEZqZ1+o9rqbVy7Nl6X71BtYfQo=; b=dwZH/Myr7+IeiKsKzuMnO0vztzK0B9nwW3FM6W7IDEoEV4menfwm7AzRWMuIKemK+u ZU4Bv0KJk0LKOi8tOWh2Bj2iY/9PKKXDUbDzg/Q242KFmt2M4f32IfFwWC8BvwqoFOYi CTgycQtnwQgqf71e/Cgg61mKv/bXjkBHLz7lM= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=hFd3YSHbBuX9q1pecEZqZ1+o9rqbVy7Nl6X71BtYfQo=; b=djIvKEXEQxQYQNg5cKZbyp4GGL3UxDSsnvnn4IW8QLhVHU8MS0xbnv4RiFNs8H2wwN hzv2Jo6HtRJ47KtBrtzfoIOC2A0rhhhaZekPtDztPfZB3sb92qmafLB/pV34pbS4kB6/ yLRs5gAbDCN8uZUtz2a69N34IXAdcRSiL7VmC107gV6MW6Ie4evtC3rzB6COs4KqauHO qu0KfQgNesJSwei+8yT+NEU7MhfM3sB8HPfy6hN0e1LvOobdcc0Zw/6fzPiEjeDB2yMC GAuyRuHLCnG0ffDHet5VEM0hE9c2vJfbkiCCp6C1H1AFKngIrvGWY5+Z0fUkp/h0wJBv yLwA== X-Gm-Message-State: ALoCoQlj7M6/E9FqvDo8EZyyTgvi+PEqYvr2RS1rAOizj8J4PsfKsEhHI4XpzRBWaeTqPUNcDi/Z MIME-Version: 1.0 X-Received: by 10.67.3.34 with SMTP id bt2mr6309256pad.3.1376802388213; Sat, 17 Aug 2013 22:06:28 -0700 (PDT) Received: by 10.70.132.66 with HTTP; Sat, 17 Aug 2013 22:06:28 -0700 (PDT) In-Reply-To: <12BBEDC5C5AF4F8180D6A6B09AD695A6@multiplay.co.uk> References: <20130712112117.Horde.JMB3fhsU1VfZRxdV4prj5Q6@mail.vx.sk> <51FE1E1E.8080502@FreeBSD.org> <5C28F2D739D94E6B928ACD55D207665A@multiplay.co.uk> <51FE9BF5.2000301@FreeBSD.org> <51FEA530.2090008@FreeBSD.org> <4B747F3B634B4A859068FC8D782A7E1D@multiplay.co.uk> <52011DE6.8090708@FreeBSD.org> <4DA6B66E61374BE68C895C8B9A93BCB7@multiplay.co.uk> <520C9001.2040406@FreeBSD.org> <78BAB3D2-86A8-404C-8D5A-5B4E89AF1208@FreeBSD.org> <12BBEDC5C5AF4F8180D6A6B09AD695A6@multiplay.co.uk> Date: Sat, 17 Aug 2013 22:06:28 -0700 Message-ID: Subject: Re: N-way mirror read speedup in zfsonlinux From: Matthew Ahrens To: Steven Hartland Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: zfs-devel@freebsd.org, Xin Li X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 18 Aug 2013 05:06:28 -0000 On Sat, Aug 17, 2013 at 8:08 PM, Steven Hartland wrote: > 2) do {} while() loops have their place, but only if they improve >> code clarity and/or yield a significant performance benefit. I >> don't see either applying here. >> > > I'd have to disagree, unless using it makes it difficult to understand, > a do while should be used is when you know there's always going to be > one iteration of the loop and hence no need to test before performing > the first iteration. > I agree with Justin. This nonstandard iteration makes it more difficult to understand. I would expect there is no measurable performance benefit, either. --matt From owner-zfs-devel@FreeBSD.ORG Sun Aug 18 15:52:16 2013 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id CC0F7E8B; Sun, 18 Aug 2013 15:52:16 +0000 (UTC) (envelope-from prvs=1942b2d7cb=killing@multiplay.co.uk) Received: from mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) (using TLSv1 with cipher RC4-MD5 (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 4230F2D90; Sun, 18 Aug 2013 15:52:15 +0000 (UTC) Received: from r2d2 ([82.69.141.170]) by mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) (MDaemon PRO v10.0.4) with ESMTP id md50005557372.msg; Sun, 18 Aug 2013 16:52:14 +0100 X-Spam-Processed: mail1.multiplay.co.uk, Sun, 18 Aug 2013 16:52:14 +0100 (not processed: message from valid local sender) X-MDDKIM-Result: neutral (mail1.multiplay.co.uk) X-MDRemoteIP: 82.69.141.170 X-Return-Path: prvs=1942b2d7cb=killing@multiplay.co.uk X-Envelope-From: killing@multiplay.co.uk Message-ID: <57C8A903D51A44D7A8A38C19F494CF99@multiplay.co.uk> From: "Steven Hartland" To: "Matthew Ahrens" References: <20130712112117.Horde.JMB3fhsU1VfZRxdV4prj5Q6@mail.vx.sk> <51FE1E1E.8080502@FreeBSD.org> <5C28F2D739D94E6B928ACD55D207665A@multiplay.co.uk> <51FE9BF5.2000301@FreeBSD.org> <51FEA530.2090008@FreeBSD.org> <4B747F3B634B4A859068FC8D782A7E1D@multiplay.co.uk> <52011DE6.8090708@FreeBSD.org> <4DA6B66E61374BE68C895C8B9A93BCB7@multiplay.co.uk> <520C9001.2040406@FreeBSD.org> <78BAB3D2-86A8-404C-8D5A-5B4E89AF1208@FreeBSD.org> <12BBEDC5C5AF4F8180D6A6B09AD695A6@multiplay.co.uk> Subject: Re: N-way mirror read speedup in zfsonlinux Date: Sun, 18 Aug 2013 16:52:36 +0100 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="iso-8859-1"; reply-type=original Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.5931 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.6157 Cc: zfs-devel@freebsd.org, Xin Li X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 18 Aug 2013 15:52:16 -0000 ----- Original Message ----- From: "Matthew Ahrens" > On Sat, Aug 17, 2013 at 8:08 PM, Steven Hartland wrote: > >> 2) do {} while() loops have their place, but only if they improve >>> code clarity and/or yield a significant performance benefit. I >>> don't see either applying here. >>> >> >> I'd have to disagree, unless using it makes it difficult to understand, >> a do while should be used is when you know there's always going to be >> one iteration of the loop and hence no need to test before performing >> the first iteration. >> > > > I agree with Justin. This nonstandard iteration makes it more difficult to > understand. I would expect there is no measurable performance benefit, > either. I'd like to understand why its more difficult to understand? Its not like its a none standard language construct, its already used in the code base in lots of place e.g. uts/common/zmod/deflate.c: do {...} while (--len > 0); uts/common/dtrace/dtrace.c: do {...} } while (--length != 0); Compared to this case: do {...} while (--c >= 0); Yes its only saving one comparison per call but thats exactly what it was added to language to do. Not trying to be be a pain, I'm just trying to understand why using the correct code for the job is confusing? Regards Steve ================================================ This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmaster@multiplay.co.uk. From owner-zfs-devel@FreeBSD.ORG Sun Aug 18 16:38:48 2013 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id F0DD57FA for ; Sun, 18 Aug 2013 16:38:48 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: from mail-pa0-x234.google.com (mail-pa0-x234.google.com [IPv6:2607:f8b0:400e:c03::234]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id BDEC52F5A for ; Sun, 18 Aug 2013 16:38:48 +0000 (UTC) Received: by mail-pa0-f52.google.com with SMTP id kq13so3733162pab.11 for ; Sun, 18 Aug 2013 09:38:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=delphix.com; s=google; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=0C6qflGeIO+mNr5w/9RUxnkI7pco/Rej4sL5V28KJCM=; b=Ym5OVzVmhjq50PMEZr7OfR5ySujTdHWGVGVoqA7XtM0p18W+0z/OG4w4Jfm1E45Eq2 fZLwXdfa8jocxy+QmPumshwXAb6QljvDrb3G/75zbvgCpHLJh8INF8gpEvR65IPpeclH v/bHRdmPaDfVr71qPsjuxHi+Qvn72r+34PPh8= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=0C6qflGeIO+mNr5w/9RUxnkI7pco/Rej4sL5V28KJCM=; b=FQQKt4HErX92BS0IG+oEpvhbbqBJhCAlz6gXf/PRV8PyrbMkMc1rjLXTzv1XZ+5y0+ XDzngAVBAQtdZePIb6BZC077RxcDKxGV9Q81nejIveVWCtsfmaWJXCsbqNixPrIxihMN 5MPp/zVdzjORjgrIquf79VM13UWwVMuow8uBFy60EG1AOmNMbQydxBhaG9CmXULqIRmq FCdu5hg9qOSwH10Tfrpsq/SKdu40S5wVeFKxkOCVyE+JeyYfWyGBVwD2jchVLtDoz/mT yVAf/U21rjlVpf3t3Wuzpyg/l5i3ltMbQk7FUZXs3P2q2uyUHSdHNikrD4UsgCnXD2vX HxfA== X-Gm-Message-State: ALoCoQkg9FrmCBkjC67J7NUbGHFyJj/hmzBe7gKNsZl/b7NEzhOoxBZH4vDvWtaX61nhq8k8tFVZ MIME-Version: 1.0 X-Received: by 10.66.235.39 with SMTP id uj7mr3334060pac.122.1376843928377; Sun, 18 Aug 2013 09:38:48 -0700 (PDT) Received: by 10.70.132.66 with HTTP; Sun, 18 Aug 2013 09:38:48 -0700 (PDT) In-Reply-To: <57C8A903D51A44D7A8A38C19F494CF99@multiplay.co.uk> References: <20130712112117.Horde.JMB3fhsU1VfZRxdV4prj5Q6@mail.vx.sk> <51FE1E1E.8080502@FreeBSD.org> <5C28F2D739D94E6B928ACD55D207665A@multiplay.co.uk> <51FE9BF5.2000301@FreeBSD.org> <51FEA530.2090008@FreeBSD.org> <4B747F3B634B4A859068FC8D782A7E1D@multiplay.co.uk> <52011DE6.8090708@FreeBSD.org> <4DA6B66E61374BE68C895C8B9A93BCB7@multiplay.co.uk> <520C9001.2040406@FreeBSD.org> <78BAB3D2-86A8-404C-8D5A-5B4E89AF1208@FreeBSD.org> <12BBEDC5C5AF4F8180D6A6B09AD695A6@multiplay.co.uk> <57C8A903D51A44D7A8A38C19F494CF99@multiplay.co.uk> Date: Sun, 18 Aug 2013 09:38:48 -0700 Message-ID: Subject: Re: N-way mirror read speedup in zfsonlinux From: Matthew Ahrens To: Steven Hartland Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: zfs-devel@freebsd.org, Xin Li X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 18 Aug 2013 16:38:49 -0000 On Sun, Aug 18, 2013 at 8:52 AM, Steven Hartland wrote: > ----- Original Message ----- From: "Matthew Ahrens" > >> On Sat, Aug 17, 2013 at 8:08 PM, Steven Hartland wrote: >> >> 2) do {} while() loops have their place, but only if they improve >>> >>>> code clarity and/or yield a significant performance benefit. I >>>> don't see either applying here. >>>> >>>> >>> I'd have to disagree, unless using it makes it difficult to understand, >>> a do while should be used is when you know there's always going to be >>> one iteration of the loop and hence no need to test before performing >>> the first iteration. >>> >>> >> >> I agree with Justin. This nonstandard iteration makes it more difficult >> to >> understand. I would expect there is no measurable performance benefit, >> either. >> > > I'd like to understand why its more difficult to understand? > As I said, it is more difficult to understand because it is nonstandard. There are many more instances in ZFS of for loops than of do/while loops that could be trivially replaced with for loops, because the for loop is the standard way of doing iteration in this codebase. My understanding is that your argument is that the do/while is faster. If you can show (measure) that it is faster in important use cases, then I will agree that the do/while is better. --matt > > Its not like its a none standard language construct, its already used in > the code base in lots of place e.g. > > uts/common/zmod/deflate.c: > do {...} while (--len > 0); > uts/common/dtrace/dtrace.c: > do {...} } while (--length != 0); > > Compared to this case: > do {...} while (--c >= 0); > > Yes its only saving one comparison per call but thats exactly what it > was added to language to do. > > Not trying to be be a pain, I'm just trying to understand why using the > correct code for the job is confusing? > > > Regards > Steve > > ==============================**================== > This e.mail is private and confidential between Multiplay (UK) Ltd. and > the person or entity to whom it is addressed. In the event of misdirection, > the recipient is prohibited from using, copying, printing or otherwise > disseminating it or any information contained in it. > In the event of misdirection, illegible or incomplete transmission please > telephone +44 845 868 1337 > or return the E.mail to postmaster@multiplay.co.uk. > > From owner-zfs-devel@FreeBSD.ORG Sun Aug 18 22:46:08 2013 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 38E93955 for ; Sun, 18 Aug 2013 22:46:08 +0000 (UTC) (envelope-from will@firepipe.net) Received: from mail-ve0-f179.google.com (mail-ve0-f179.google.com [209.85.128.179]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id E437C2153 for ; Sun, 18 Aug 2013 22:46:07 +0000 (UTC) Received: by mail-ve0-f179.google.com with SMTP id c13so2488800vea.38 for ; Sun, 18 Aug 2013 15:46:00 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=eQD0g++bZE97bOFdLETF6txz+a/Lthm5hbID435yrJo=; b=XY6j5XbkOd8glvKgRqfI0Z0lUNYC8d4b9w8P+njBZw0hFdUVqJhG3+s3E0irW5Emg/ 8En58Fa30I75Nf9TUdeikJNaz+Kk1BSlj3eCSRMdlo0wTqGocewDHpErng+LBFlv9o2R /aUfTgDfCLqB+sGXh73JOQXwlhp25KctDBNVgFC8PsHS7KJR/rcIMWOyxn0e0QXacNYa 7bJi9t+eNZy24k+kGLlq4dma8qGunXPT3NrYlyahjwmmk95aOF8oVhYSEZpyUGNJi3gt KNPjKnsOdnRc54piGFx5Iqqgt50TNEWibE3DCJpXF0Otz6hM14Pfyu3eTyMglMz8Znch Z8ww== X-Gm-Message-State: ALoCoQkawxJ+gY283NF3ywgX7uYSNCEyJvGYH/RJjwea02S4F84KQMxYj4KfRMylyAqzXf1wF/lG MIME-Version: 1.0 X-Received: by 10.58.75.41 with SMTP id z9mr10148393vev.4.1376865522903; Sun, 18 Aug 2013 15:38:42 -0700 (PDT) Received: by 10.58.226.66 with HTTP; Sun, 18 Aug 2013 15:38:42 -0700 (PDT) In-Reply-To: References: <20130712112117.Horde.JMB3fhsU1VfZRxdV4prj5Q6@mail.vx.sk> <51FE1E1E.8080502@FreeBSD.org> <5C28F2D739D94E6B928ACD55D207665A@multiplay.co.uk> <51FE9BF5.2000301@FreeBSD.org> <51FEA530.2090008@FreeBSD.org> <4B747F3B634B4A859068FC8D782A7E1D@multiplay.co.uk> <52011DE6.8090708@FreeBSD.org> <4DA6B66E61374BE68C895C8B9A93BCB7@multiplay.co.uk> <520C9001.2040406@FreeBSD.org> <78BAB3D2-86A8-404C-8D5A-5B4E89AF1208@FreeBSD.org> <12BBEDC5C5AF4F8180D6A6B09AD695A6@multiplay.co.uk> <57C8A903D51A44D7A8A38C19F494CF99@multiplay.co.uk> Date: Sun, 18 Aug 2013 16:38:42 -0600 Message-ID: Subject: Re: N-way mirror read speedup in zfsonlinux From: Will Andrews To: Matthew Ahrens Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: Xin Li , "zfs-devel@freebsd.org" X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 18 Aug 2013 22:46:08 -0000 On Sunday, August 18, 2013, Matthew Ahrens wrote: > > As I said, it is more difficult to understand because it is nonstandard. > There are many more instances in ZFS of for loops than of do/while loops > that could be trivially replaced with for loops, because the for loop is > the standard way of doing iteration in this codebase. > > My understanding is that your argument is that the do/while is faster. If > you can show (measure) that it is faster in important use cases, then I > will agree that the do/while is better. > Not to mention: - the change assumes the compiler isn't already optimizing the code to behave like a do..while loop. It may do so given that a vdev_children value of 0 would yield a zero length buffer and likely page faults for c = 0. - for loops condense the loop conditions & iteration into a single line, so for most cases it is clearer what the bounds of the loop are. - we are talking about *maybe* saving a handful of instructions while performing I/O. --Will. From owner-zfs-devel@FreeBSD.ORG Sun Aug 18 23:06:54 2013 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 428B0F96; Sun, 18 Aug 2013 23:06:54 +0000 (UTC) (envelope-from prvs=1942b2d7cb=killing@multiplay.co.uk) Received: from mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) (using TLSv1 with cipher RC4-MD5 (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id AA7FE225A; Sun, 18 Aug 2013 23:06:53 +0000 (UTC) Received: from r2d2 ([82.69.141.170]) by mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) (MDaemon PRO v10.0.4) with ESMTP id md50005564478.msg; Mon, 19 Aug 2013 00:06:50 +0100 X-Spam-Processed: mail1.multiplay.co.uk, Mon, 19 Aug 2013 00:06:50 +0100 (not processed: message from valid local sender) X-MDDKIM-Result: neutral (mail1.multiplay.co.uk) X-MDRemoteIP: 82.69.141.170 X-Return-Path: prvs=1942b2d7cb=killing@multiplay.co.uk X-Envelope-From: killing@multiplay.co.uk Message-ID: <3F20CA2BA01E4B07899B4CB82EB0E220@multiplay.co.uk> From: "Steven Hartland" To: "Will Andrews" , "Matthew Ahrens" References: <20130712112117.Horde.JMB3fhsU1VfZRxdV4prj5Q6@mail.vx.sk><51FE1E1E.8080502@FreeBSD.org><5C28F2D739D94E6B928ACD55D207665A@multiplay.co.uk><51FE9BF5.2000301@FreeBSD.org><51FEA530.2090008@FreeBSD.org><4B747F3B634B4A859068FC8D782A7E1D@multiplay.co.uk><52011DE6.8090708@FreeBSD.org><4DA6B66E61374BE68C895C8B9A93BCB7@multiplay.co.uk><520C9001.2040406@FreeBSD.org><78BAB3D2-86A8-404C-8D5A-5B4E89AF1208@FreeBSD.org><12BBEDC5C5AF4F8180D6A6B09AD695A6@multiplay.co.uk><57C8A903D51A44D7A8A38C19F494CF99@multiplay.co.uk> Subject: Re: N-way mirror read speedup in zfsonlinux Date: Mon, 19 Aug 2013 00:07:14 +0100 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="utf-8"; reply-type=original Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.5931 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.6157 Cc: zfs-devel@freebsd.org, Xin Li X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 18 Aug 2013 23:06:54 -0000 ----- Original Message ----- From: "Will Andrews" > On Sunday, August 18, 2013, Matthew Ahrens wrote: >> >> As I said, it is more difficult to understand because it is nonstandard. >> There are many more instances in ZFS of for loops than of do/while loops >> that could be trivially replaced with for loops, because the for loop is >> the standard way of doing iteration in this codebase. >> >> My understanding is that your argument is that the do/while is faster. If >> you can show (measure) that it is faster in important use cases, then I >> will agree that the do/while is better. >> > > Not to mention: > - the change assumes the compiler isn't already optimizing the code to > behave like a do..while loop. It may do so given that a vdev_children value > of 0 would yield a zero length buffer and likely page faults for c = 0. > - for loops condense the loop conditions & iteration into a single line, so > for most cases it is clearer what the bounds of the loop are. > - we are talking about *maybe* saving a handful of instructions while > performing I/O. You are indeed correct there Will. I've been playing with various compilers with a little test app that performs code similar to this and for the most part it does optimise out to the same performance. The case where it does make a significant difference is when the test is not just a simple comparison, such as a function call. In that case using a do {...} while() instead of a for(){..} results in a noticeable performance increase. So given the simple loop test I'll concede and switch it back to for loop. Thanks for bearing with me this point guys, hopefully the above will provide useful in the future. Regards steve ================================================ This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmaster@multiplay.co.uk. From owner-zfs-devel@FreeBSD.ORG Sun Aug 18 23:42:16 2013 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 375E563B for ; Sun, 18 Aug 2013 23:42:16 +0000 (UTC) (envelope-from gibbs@FreeBSD.org) Received: from aslan.scsiguy.com (mail.scsiguy.com [70.89.174.89]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id F276F23EE for ; Sun, 18 Aug 2013 23:42:15 +0000 (UTC) Received: from [192.168.0.99] (macbook.scsiguy.com [192.168.0.99]) (authenticated bits=0) by aslan.scsiguy.com (8.14.7/8.14.5) with ESMTP id r7INgD28091765 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NO); Sun, 18 Aug 2013 23:42:13 GMT (envelope-from gibbs@FreeBSD.org) From: "Justin T. Gibbs" Content-Type: multipart/mixed; boundary="Apple-Mail=_60E48F97-EC62-4D76-9B22-B5B59EF22B8D" Subject: Updated ashift optimization patch Date: Sun, 18 Aug 2013 17:42:12 -0600 Message-Id: <984A4829-FBB0-4174-83FB-06A7336F4737@FreeBSD.org> To: "zfs-devel@freebsd.org" , "zfs@lists.illumos.org" Mime-Version: 1.0 (Mac OS X Mail 6.5 \(1508\)) X-Mailer: Apple Mail (2.1508) X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.4.3 (aslan.scsiguy.com [70.89.174.89]); Sun, 18 Aug 2013 23:42:14 +0000 (UTC) Cc: Richard Yao X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 18 Aug 2013 23:42:16 -0000 --Apple-Mail=_60E48F97-EC62-4D76-9B22-B5B59EF22B8D Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii Today I re-merged all of Spectra's ashift related changes into = FreeBSD/head. Here's the current state of the patch. See the top of = the patch for proposed commit message/justification. The majority of it = is ZFS platform agnostic. Later this week I'll get a stock FreeBSD/head system running and run the = test suites in order to verify that I haven't botched anything in the = merge. I'll also see about merging it to one of Will's Illumos VMs so I = can generate a webrev. In the mean time, comments, concerns, = improvements welcome. -- Justin --Apple-Mail=_60E48F97-EC62-4D76-9B22-B5B59EF22B8D Content-Disposition: attachment; filename=zfs_ashift.diffs Content-Type: application/octet-stream; name="zfs_ashift.diffs" Content-Transfer-Encoding: 7bit Enhance the ZFS vdev layer to maintain both a logical and a physical minimum allocation size for devices. Use this information to automatically increase ZFS's minimum allocation size for new top-level vdevs to a value that more closely matches the optimum device allocation size. Calculate the minimum blocksize of each metaslab class. Use the calculated value instead of SPA_MINBLOCKSIZE (512b) when determining the likelyhood of compression yeilding a reduction in physical space usage. Report devices with sub-optimal block size configuration in "zpool status". Also properly fail attempts to attach devices with a logical block size greater than 8kB, since this will cause corruption to ZFS's label area. Background ========== Many modern devices use physical allocation units that are much larger than the minimum logical allocation size accessible by external commands. Two prevalent examples of this are 512e disk drives (512b logical sector, 4K physical sector) and flash devices (512b logical sector, 4K or larger allocation block size, and 128k or larger erase block size). Operations that modify less than the physical sector size result in a costly read-modify-write or garbage collection sequence on these devices. Simply exporting the true physical sector of the device to ZFS would yield optimal performance, but has two serious drawbacks: 1) Existing pools created with devices that have different logical and physical block sizes, but were configured to use the logical block size (e.g. because the OS version used for pool construction reported the logical block size instead of the physical block size) will suddenly find that the vdev allocation size has increased. This can be easily tolerated for active members of the array, but ZFS would prevent replacement of a vdev with another identical device because it now appears that the smaller allocation size required by the pool is not supported by the new device. 2) The device's physical block size may be too large to be supported by ZFS. The optimal allocation size for the vdev may be quite large. For example, a RAID controller may export a vdev that requires read-modify-write cycles unless accessed using 64k aligned/sized requests. ZFS currently has an 8k minimum block size limit. Reporting both the logical and physical allocation sizes for vdevs solves these problems. A device may be used so long as the logical block size is compatible with the configuration. By comparing the logical and physical block sizes, new configurations can be optimized and administrators can be notified of any existing pools that are sub-optimal. sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/spa.h: Add the SPA_ASHIFT constant. ZFS currently has a hard upper limit of 13 (8k) for ashift and this constant is used to both document and enforce this limit. sys/cddl/contrib/opensolaris/uts/common/sys/fs/zfs.h: Add the VDEV_AUX_ASHIFT_TOO_BIG error code. Add fields for exporting the configured, logical, and physical ashift to the vdev_stat_t structure. Add VDEV_STAT_VALID() macro which can be used to verify the presence of required vdev_stat_t fields in nvlist data. sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev.c: Provide a SYSCTL_PROC handler for "max_auto_ashift". Since the limit is only referenced long after boot when a create operation occurs, there's no compelling need for it to be a boot time configurable tunable. This also allows the validation code for the max_auto_ashift value to be contained within the sysctl handler. Populate the new fields in the vdev_stat_t structure. Fail vdev opens if the vdev reports an ashift larger than SPA_MAXASHIFT. Propogate vdev_logical_ashift and vdev_physical_ashift between child and parent vdevs as is done for vdev_ashift. In vdev_open(), restore code that fails opens for devices where vdev_ashift grows. This can only happen now if the device's logical ashift grows, which means it really isn't safe to use the device. sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/vdev_impl.h: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev.c: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_file.c: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_geom.c: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_mirror.c: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_missing.c: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_raidz.c: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_root.c: Update the vdev_open() API so that both logical (what was just ashift before) and physical ashift are reported. sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/vdev_impl.h: Add two new fields, vdev_physical_ashift and vdev_logical_ashift, to vdev_t. sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev.c: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa_config.c: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa.c: Add vdev_ashift_optimize(). Call it anytime a new top-level vdev is allocated. cddl/contrib/opensolaris/cmd/zpool/zpool_main.c: Add text for the VDEV_AUX_ASHIFT_TOO_BIG error. For each sub-optimally configured leaf vdev, report configured and native block sizes. cddl/contrib/opensolaris/cmd/zpool/zpool_main.c: cddl/contrib/opensolaris/lib/libzfs/common/libzfs.h: cddl/contrib/opensolaris/lib/libzfs/common/libzfs_status.c: Introduce a new zpool status: ZPOOL_STATUS_NON_NATIVE_ASHIFT. This status is reported on healthy pools containing vdevs configured to use a block size smaller than their reported physical block size. cddl/contrib/opensolaris/lib/libzfs/common/libzfs_status.c: Update find_vdev_problem() and supporting functions to provide the full vdev_stat_t structure to problem checking routines, and to allow decent into replacing vdevs. Add a vdev_non_native_ashift() validator which is used on the full vdev tree to check for ZPOOL_STATUS_NON_NATIVE_ASHIFT. cddl/contrib/opensolaris/lib/libzpool/common/kernel.c: cddl/contrib/opensolaris/lib/libzpool/common/sys/zfs_context.h: Enhance sysctl userland stubs now that a SYSCTL_PROC handler is used in vdev.c. sys/cddl/contrib/opensolaris/uts/common/fs/zfs/metaslab.c: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/metaslab_impl.h: When the group membership of a metaslab class changes (i.e. when a vdev is added or removed from a pool), walk the group list to determine the smallest block size currently available and record this in the metaslab class. sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/metaslab.h: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/metaslab.c: Add the metaslab_class_get_minblocksize() accessor. sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/zio_compress.h: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio_compress.c: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c: In zio_compress_data(), take the minimum blocksize as an input parameter instead of assuming SPA_MINBLOCKSIZE. sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c: In l2arc_compress_buf(), pass SPA_MINBLOCKSIZE as the minimum blocksize of the device. The l2arc code performs has it's own code for deciding if compression is worth while, so this effectively disables zio_compress_data() from second guessing the original decision. sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c: In zio_write_bp_init(), use the minimum blocksize of the normal metaslab class when compressing data. Index: cddl/contrib/opensolaris/cmd/zpool/zpool_main.c =================================================================== --- cddl/contrib/opensolaris/cmd/zpool/zpool_main.c (revision 254492) +++ cddl/contrib/opensolaris/cmd/zpool/zpool_main.c (working copy) @@ -1295,12 +1295,13 @@ print_status_config(zpool_handle_t *zhp, const cha int namewidth, int depth, boolean_t isspare) { nvlist_t **child; - uint_t c, children; + uint_t c, vsc, children; pool_scan_stat_t *ps = NULL; vdev_stat_t *vs; char rbuf[6], wbuf[6], cbuf[6]; char *vname; uint64_t notpresent; + uint64_t ashift; spare_cbdata_t cb; const char *state; @@ -1309,7 +1310,7 @@ print_status_config(zpool_handle_t *zhp, const cha children = 0; verify(nvlist_lookup_uint64_array(nv, ZPOOL_CONFIG_VDEV_STATS, - (uint64_t **)&vs, &c) == 0); + (uint64_t **)&vs, &vsc) == 0); state = zpool_state_to_name(vs->vs_state, vs->vs_aux); if (isspare) { @@ -1363,6 +1364,10 @@ print_status_config(zpool_handle_t *zhp, const cha (void) printf(gettext("unsupported feature(s)")); break; + case VDEV_AUX_ASHIFT_TOO_BIG: + (void) printf(gettext("unsupported minimum blocksize")); + break; + case VDEV_AUX_SPARED: verify(nvlist_lookup_uint64(nv, ZPOOL_CONFIG_GUID, &cb.cb_guid) == 0); @@ -1405,6 +1410,12 @@ print_status_config(zpool_handle_t *zhp, const cha (void) printf(gettext("corrupted data")); break; } + } else if (children == 0 && !isspare && + VDEV_STAT_VALID(vs_physical_ashift, vsc) && + vs->vs_configured_ashift < vs->vs_physical_ashift) { + (void) printf( + gettext(" block size: %dB configured, %dB native"), + 1 << vs->vs_configured_ashift, 1 << vs->vs_physical_ashift); } (void) nvlist_lookup_uint64_array(nv, ZPOOL_CONFIG_SCAN_STATS, @@ -4268,6 +4279,15 @@ status_callback(zpool_handle_t *zhp, void *data) "'zpool clear'.\n")); break; + case ZPOOL_STATUS_NON_NATIVE_ASHIFT: + (void) printf(gettext("status: One or more devices are " + "configured to use a non-native block size.\n" + "\tExpect reduced performance.\n")); + (void) printf(gettext("action: Replace affected devices with " + "devices that support the\n\tconfigured block size, or " + "migrate data to a properly configured\n\tpool.\n")); + break; + default: /* * The remaining errors can't actually be generated, yet. Index: cddl/contrib/opensolaris/lib/libzfs/common/libzfs.h =================================================================== --- cddl/contrib/opensolaris/lib/libzfs/common/libzfs.h (revision 254492) +++ cddl/contrib/opensolaris/lib/libzfs/common/libzfs.h (working copy) @@ -326,6 +326,7 @@ typedef enum { ZPOOL_STATUS_RESILVERING, /* device being resilvered */ ZPOOL_STATUS_OFFLINE_DEV, /* device online */ ZPOOL_STATUS_REMOVED_DEV, /* removed device */ + ZPOOL_STATUS_NON_NATIVE_ASHIFT, /* (e.g. 512e dev with ashift of 9) */ /* * Finally, the following indicates a healthy pool. Index: cddl/contrib/opensolaris/lib/libzfs/common/libzfs_status.c =================================================================== --- cddl/contrib/opensolaris/lib/libzfs/common/libzfs_status.c (revision 254492) +++ cddl/contrib/opensolaris/lib/libzfs/common/libzfs_status.c (working copy) @@ -73,57 +73,66 @@ static char *zfs_msgid_table[] = { /* ARGSUSED */ static int -vdev_missing(uint64_t state, uint64_t aux, uint64_t errs) +vdev_missing(vdev_stat_t *vs, uint_t vsc) { - return (state == VDEV_STATE_CANT_OPEN && - aux == VDEV_AUX_OPEN_FAILED); + return (vs->vs_state == VDEV_STATE_CANT_OPEN && + vs->vs_aux == VDEV_AUX_OPEN_FAILED); } /* ARGSUSED */ static int -vdev_faulted(uint64_t state, uint64_t aux, uint64_t errs) +vdev_faulted(vdev_stat_t *vs, uint_t vsc) { - return (state == VDEV_STATE_FAULTED); + return (vs->vs_state == VDEV_STATE_FAULTED); } /* ARGSUSED */ static int -vdev_errors(uint64_t state, uint64_t aux, uint64_t errs) +vdev_errors(vdev_stat_t *vs, uint_t vsc) { - return (state == VDEV_STATE_DEGRADED || errs != 0); + return (vs->vs_state == VDEV_STATE_DEGRADED || + vs->vs_read_errors != 0 || vs->vs_write_errors != 0 || + vs->vs_checksum_errors != 0); } /* ARGSUSED */ static int -vdev_broken(uint64_t state, uint64_t aux, uint64_t errs) +vdev_broken(vdev_stat_t *vs, uint_t vsc) { - return (state == VDEV_STATE_CANT_OPEN); + return (vs->vs_state == VDEV_STATE_CANT_OPEN); } /* ARGSUSED */ static int -vdev_offlined(uint64_t state, uint64_t aux, uint64_t errs) +vdev_offlined(vdev_stat_t *vs, uint_t vsc) { - return (state == VDEV_STATE_OFFLINE); + return (vs->vs_state == VDEV_STATE_OFFLINE); } /* ARGSUSED */ static int -vdev_removed(uint64_t state, uint64_t aux, uint64_t errs) +vdev_removed(vdev_stat_t *vs, uint_t vsc) { - return (state == VDEV_STATE_REMOVED); + return (vs->vs_state == VDEV_STATE_REMOVED); } +static int +vdev_non_native_ashift(vdev_stat_t *vs, uint_t vsc) +{ + return (VDEV_STAT_VALID(vs_physical_ashift, vsc) && + vs->vs_configured_ashift < vs->vs_physical_ashift); +} + /* * Detect if any leaf devices that have seen errors or could not be opened. */ static boolean_t -find_vdev_problem(nvlist_t *vdev, int (*func)(uint64_t, uint64_t, uint64_t)) +find_vdev_problem(nvlist_t *vdev, int (*func)(vdev_stat_t *, uint_t), + boolean_t ignore_replacing) { nvlist_t **child; vdev_stat_t *vs; - uint_t c, children; - char *type; + uint_t c, vsc, children; /* * Ignore problems within a 'replacing' vdev, since we're presumably in @@ -131,23 +140,25 @@ static boolean_t * out again. We'll pick up the fact that a resilver is happening * later. */ - verify(nvlist_lookup_string(vdev, ZPOOL_CONFIG_TYPE, &type) == 0); - if (strcmp(type, VDEV_TYPE_REPLACING) == 0) - return (B_FALSE); + if (ignore_replacing == B_TRUE) { + char *type; + verify(nvlist_lookup_string(vdev, ZPOOL_CONFIG_TYPE, + &type) == 0); + if (strcmp(type, VDEV_TYPE_REPLACING) == 0) + return (B_FALSE); + } + if (nvlist_lookup_nvlist_array(vdev, ZPOOL_CONFIG_CHILDREN, &child, &children) == 0) { for (c = 0; c < children; c++) - if (find_vdev_problem(child[c], func)) + if (find_vdev_problem(child[c], func, ignore_replacing)) return (B_TRUE); } else { verify(nvlist_lookup_uint64_array(vdev, ZPOOL_CONFIG_VDEV_STATS, - (uint64_t **)&vs, &c) == 0); + (uint64_t **)&vs, &vsc) == 0); - if (func(vs->vs_state, vs->vs_aux, - vs->vs_read_errors + - vs->vs_write_errors + - vs->vs_checksum_errors)) + if (func(vs, vsc) != 0) return (B_TRUE); } @@ -157,7 +168,7 @@ static boolean_t if (nvlist_lookup_nvlist_array(vdev, ZPOOL_CONFIG_L2CACHE, &child, &children) == 0) { for (c = 0; c < children; c++) - if (find_vdev_problem(child[c], func)) + if (find_vdev_problem(child[c], func, ignore_replacing)) return (B_TRUE); } @@ -270,15 +281,15 @@ check_status(nvlist_t *config, boolean_t isimport) * Bad devices in non-replicated config. */ if (vs->vs_state == VDEV_STATE_CANT_OPEN && - find_vdev_problem(nvroot, vdev_faulted)) + find_vdev_problem(nvroot, vdev_faulted, B_TRUE)) return (ZPOOL_STATUS_FAULTED_DEV_NR); if (vs->vs_state == VDEV_STATE_CANT_OPEN && - find_vdev_problem(nvroot, vdev_missing)) + find_vdev_problem(nvroot, vdev_missing, B_TRUE)) return (ZPOOL_STATUS_MISSING_DEV_NR); if (vs->vs_state == VDEV_STATE_CANT_OPEN && - find_vdev_problem(nvroot, vdev_broken)) + find_vdev_problem(nvroot, vdev_broken, B_TRUE)) return (ZPOOL_STATUS_CORRUPT_LABEL_NR); /* @@ -300,32 +311,38 @@ check_status(nvlist_t *config, boolean_t isimport) /* * Missing devices in a replicated config. */ - if (find_vdev_problem(nvroot, vdev_faulted)) + if (find_vdev_problem(nvroot, vdev_faulted, B_TRUE)) return (ZPOOL_STATUS_FAULTED_DEV_R); - if (find_vdev_problem(nvroot, vdev_missing)) + if (find_vdev_problem(nvroot, vdev_missing, B_TRUE)) return (ZPOOL_STATUS_MISSING_DEV_R); - if (find_vdev_problem(nvroot, vdev_broken)) + if (find_vdev_problem(nvroot, vdev_broken, B_TRUE)) return (ZPOOL_STATUS_CORRUPT_LABEL_R); /* * Devices with errors */ - if (!isimport && find_vdev_problem(nvroot, vdev_errors)) + if (!isimport && find_vdev_problem(nvroot, vdev_errors, B_TRUE)) return (ZPOOL_STATUS_FAILING_DEV); /* * Offlined devices */ - if (find_vdev_problem(nvroot, vdev_offlined)) + if (find_vdev_problem(nvroot, vdev_offlined, B_TRUE)) return (ZPOOL_STATUS_OFFLINE_DEV); /* * Removed device */ - if (find_vdev_problem(nvroot, vdev_removed)) + if (find_vdev_problem(nvroot, vdev_removed, B_TRUE)) return (ZPOOL_STATUS_REMOVED_DEV); /* + * Suboptimal, but usable, ashift configuration. + */ + if (find_vdev_problem(nvroot, vdev_non_native_ashift, B_FALSE)) + return (ZPOOL_STATUS_NON_NATIVE_ASHIFT); + + /* * Outdated, but usable, version */ if (SPA_VERSION_IS_SUPPORTED(version) && version != SPA_VERSION) Index: cddl/contrib/opensolaris/lib/libzpool/common/kernel.c =================================================================== --- cddl/contrib/opensolaris/lib/libzpool/common/kernel.c (revision 254492) +++ cddl/contrib/opensolaris/lib/libzpool/common/kernel.c (working copy) @@ -591,6 +591,12 @@ dprintf_setup(int *argc, char **argv) dprintf_print_all = 1; } +int +sysctl_handle_64(SYSCTL_HANDLER_ARGS) +{ + return (0); +} + /* * ========================================================================= * debug printfs Index: cddl/contrib/opensolaris/lib/libzpool/common/sys/zfs_context.h =================================================================== --- cddl/contrib/opensolaris/lib/libzpool/common/sys/zfs_context.h (revision 254492) +++ cddl/contrib/opensolaris/lib/libzpool/common/sys/zfs_context.h (working copy) @@ -659,11 +659,55 @@ typedef uint32_t idmap_rid_t; #define SX_SYSINIT(name, lock, desc) +#define SYSCTL_HANDLER_ARGS struct sysctl_oid *oidp, void *arg1, \ + intptr_t arg2, struct sysctl_req *req + +/* + * This describes the access space for a sysctl request. This is needed + * so that we can use the interface from the kernel or from user-space. + */ +struct sysctl_req { + struct thread *td; /* used for access checking */ + int lock; /* wiring state */ + void *oldptr; + size_t oldlen; + size_t oldidx; + int (*oldfunc)(struct sysctl_req *, const void *, size_t); + void *newptr; + size_t newlen; + size_t newidx; + int (*newfunc)(struct sysctl_req *, void *, size_t); + size_t validlen; + int flags; +}; + +SLIST_HEAD(sysctl_oid_list, sysctl_oid); + +/* + * This describes one "oid" in the MIB tree. Potentially more nodes can + * be hidden behind it, expanded by the handler. + */ +struct sysctl_oid { + struct sysctl_oid_list *oid_parent; + SLIST_ENTRY(sysctl_oid) oid_link; + int oid_number; + u_int oid_kind; + void *oid_arg1; + intptr_t oid_arg2; + const char *oid_name; + int (*oid_handler)(SYSCTL_HANDLER_ARGS); + const char *oid_fmt; + int oid_refcnt; + u_int oid_running; + const char *oid_descr; +}; + #define SYSCTL_DECL(...) #define SYSCTL_NODE(...) #define SYSCTL_INT(...) #define SYSCTL_UINT(...) #define SYSCTL_ULONG(...) +#define SYSCTL_PROC(...) #define SYSCTL_QUAD(...) #define SYSCTL_UQUAD(...) #ifdef TUNABLE_INT @@ -675,6 +719,8 @@ typedef uint32_t idmap_rid_t; #define TUNABLE_ULONG(...) #define TUNABLE_QUAD(...) +int sysctl_handle_64(SYSCTL_HANDLER_ARGS); + /* Errors */ #ifndef ERESTART Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c =================================================================== --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c (revision 254492) +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c (working copy) @@ -5147,7 +5147,7 @@ l2arc_compress_buf(l2arc_buf_hdr_t *l2hdr) len = l2hdr->b_asize; cdata = zio_data_buf_alloc(len); csize = zio_compress_data(ZIO_COMPRESS_LZ4, l2hdr->b_tmp_cdata, - cdata, l2hdr->b_asize); + cdata, l2hdr->b_asize, (size_t)SPA_MINBLOCKSIZE); if (csize == 0) { /* zero block, indicate that there's nothing to write */ Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/metaslab.c =================================================================== --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/metaslab.c (revision 254492) +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/metaslab.c (working copy) @@ -180,6 +180,27 @@ metaslab_class_space_update(metaslab_class_t *mc, atomic_add_64(&mc->mc_dspace, dspace_delta); } +void +metaslab_class_minblocksize_update(metaslab_class_t *mc) +{ + metaslab_group_t *mg; + vdev_t *vd; + uint64_t minashift = UINT64_MAX; + + if ((mg = mc->mc_rotor) == NULL) { + mc->mc_minblocksize = SPA_MINBLOCKSIZE; + return; + } + + do { + vd = mg->mg_vd; + if (vd->vdev_ashift < minashift) + minashift = vd->vdev_ashift; + } while ((mg = mg->mg_next) != mc->mc_rotor); + + mc->mc_minblocksize = 1ULL << minashift; +} + uint64_t metaslab_class_get_alloc(metaslab_class_t *mc) { @@ -204,6 +225,12 @@ metaslab_class_get_dspace(metaslab_class_t *mc) return (spa_deflate(mc->mc_spa) ? mc->mc_dspace : mc->mc_space); } +uint64_t +metaslab_class_get_minblocksize(metaslab_class_t *mc) +{ + return (mc->mc_minblocksize); +} + /* * ========================================================================== * Metaslab groups @@ -295,6 +322,7 @@ metaslab_group_activate(metaslab_group_t *mg) mgnext->mg_prev = mg; } mc->mc_rotor = mg; + metaslab_class_minblocksize_update(mc); } void @@ -326,6 +354,7 @@ metaslab_group_passivate(metaslab_group_t *mg) mg->mg_prev = NULL; mg->mg_next = NULL; + metaslab_class_minblocksize_update(mc); } static void Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa.c =================================================================== --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa.c (revision 254492) +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa.c (working copy) @@ -3424,6 +3424,7 @@ spa_create(const char *pool, nvlist_t *nvroot, nvl (error = spa_validate_aux(spa, nvroot, txg, VDEV_ALLOC_ADD)) == 0) { for (int c = 0; c < rvd->vdev_children; c++) { + vdev_ashift_optimize(rvd->vdev_child[c]); vdev_metaslab_set_size(rvd->vdev_child[c]); vdev_expand(rvd->vdev_child[c], txg); } Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa_config.c =================================================================== --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa_config.c (revision 254492) +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa_config.c (working copy) @@ -519,8 +519,10 @@ spa_config_update(spa_t *spa, int what) */ for (c = 0; c < rvd->vdev_children; c++) { vdev_t *tvd = rvd->vdev_child[c]; - if (tvd->vdev_ms_array == 0) + if (tvd->vdev_ms_array == 0) { + vdev_ashift_optimize(tvd); vdev_metaslab_set_size(tvd); + } vdev_expand(tvd, txg); } } Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/metaslab.h =================================================================== --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/metaslab.h (revision 254492) +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/metaslab.h (working copy) @@ -70,6 +70,7 @@ extern uint64_t metaslab_class_get_alloc(metaslab_ extern uint64_t metaslab_class_get_space(metaslab_class_t *mc); extern uint64_t metaslab_class_get_dspace(metaslab_class_t *mc); extern uint64_t metaslab_class_get_deferred(metaslab_class_t *mc); +extern uint64_t metaslab_class_get_minblocksize(metaslab_class_t *mc); extern metaslab_group_t *metaslab_group_create(metaslab_class_t *mc, vdev_t *vd); Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/metaslab_impl.h =================================================================== --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/metaslab_impl.h (revision 254492) +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/metaslab_impl.h (working copy) @@ -49,6 +49,7 @@ struct metaslab_class { uint64_t mc_deferred; /* total deferred frees */ uint64_t mc_space; /* total space (alloc + free) */ uint64_t mc_dspace; /* total deflated space */ + uint64_t mc_minblocksize; }; struct metaslab_group { Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/spa.h =================================================================== --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/spa.h (revision 254492) +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/spa.h (working copy) @@ -93,6 +93,17 @@ struct dsl_dataset; #define SPA_BLOCKSIZES (SPA_MAXBLOCKSHIFT - SPA_MINBLOCKSHIFT + 1) /* + * Maximum supported logical ashift. + * + * The current 8k allocation block size limit is due to the 8k + * aligned/sized operations performed by vdev_probe() on + * vdev_label->vl_pad2. Using another "safe region" for these tests + * would allow the limit to be raised to 16k, at the expense of + * only having 8 available uberblocks in the label area. + */ +#define SPA_MAXASHIFT 13 + +/* * Size of block to hold the configuration data (a packed nvlist) */ #define SPA_CONFIG_BLOCKSIZE (1ULL << 14) Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/vdev.h =================================================================== --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/vdev.h (revision 254492) +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/vdev.h (working copy) @@ -78,6 +78,7 @@ extern void vdev_rele(vdev_t *); extern int vdev_metaslab_init(vdev_t *vd, uint64_t txg); extern void vdev_metaslab_fini(vdev_t *vd); extern void vdev_metaslab_set_size(vdev_t *); +extern void vdev_ashift_optimize(vdev_t *); extern void vdev_expand(vdev_t *vd, uint64_t txg); extern void vdev_split(vdev_t *vd); extern void vdev_deadman(vdev_t *vd); Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/vdev_impl.h =================================================================== --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/vdev_impl.h (revision 254492) +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/vdev_impl.h (working copy) @@ -57,7 +57,7 @@ typedef struct vdev_cache_entry vdev_cache_entry_t * Virtual device operations */ typedef int vdev_open_func_t(vdev_t *vd, uint64_t *size, uint64_t *max_size, - uint64_t *ashift); + uint64_t *logical_ashift, uint64_t *physical_ashift); typedef void vdev_close_func_t(vdev_t *vd); typedef uint64_t vdev_asize_func_t(vdev_t *vd, uint64_t psize); typedef int vdev_io_start_func_t(zio_t *zio); @@ -123,6 +123,24 @@ struct vdev { uint64_t vdev_min_asize; /* min acceptable asize */ uint64_t vdev_max_asize; /* max acceptable asize */ uint64_t vdev_ashift; /* block alignment shift */ + /* + * Logical block alignment shift + * + * The smallest sized/aligned I/O supported by the device. + */ + uint64_t vdev_logical_ashift; + /* + * Physical block alignment shift + * + * The device supports logical I/Os with vdev_logical_ashift + * size/alignment, but optimum performance will be achieved by + * aligning/sizing requests to vdev_physical_ashift. Smaller + * requests may be inflated or incur device level read-modify-write + * operations. + * + * May be 0 to indicate no preference (i.e. use vdev_logical_ashift). + */ + uint64_t vdev_physical_ashift; uint64_t vdev_state; /* see VDEV_STATE_* #defines */ uint64_t vdev_prevstate; /* used when reopening a vdev */ vdev_ops_t *vdev_ops; /* vdev operations */ Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/zio_compress.h =================================================================== --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/zio_compress.h (revision 254492) +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/zio_compress.h (working copy) @@ -79,7 +79,7 @@ extern int lz4_decompress(void *src, void *dst, si * Compress and decompress data if necessary. */ extern size_t zio_compress_data(enum zio_compress c, void *src, void *dst, - size_t s_len); + size_t s_len, size_t minblocksize); extern int zio_decompress_data(enum zio_compress c, void *src, void *dst, size_t s_len, size_t d_len); Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev.c =================================================================== --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev.c (revision 254492) +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev.c (working copy) @@ -52,6 +52,51 @@ SYSCTL_NODE(_vfs_zfs, OID_AUTO, vdev, CTLFLAG_RW, * Virtual device management. */ +/** + * The limit for ZFS to automatically increase a top-level vdev's ashift + * from logical ashift to physical ashift. + * + * Example: one or more 512b emulation child vdevs + * child->vdev_ashift = 9 (512 bytes) + * child->vdev_physical_ashift = 12 (4096 bytes) + * zfs_max_auto_ashift = 11 (2048 bytes) + * + * On pool creation or the addition of a new top-leve vdev, ZFS will + * bump the ashift of the top-level vdev to 2048. + * + * Example: one or more 512b emulation child vdevs + * child->vdev_ashift = 9 (512 bytes) + * child->vdev_physical_ashift = 12 (4096 bytes) + * zfs_max_auto_ashift = 13 (8192 bytes) + * + * On pool creation or the addition of a new top-leve vdev, ZFS will + * bump the ashift of the top-level vdev to 4096. + */ +static uint64_t zfs_max_auto_ashift = SPA_MAXASHIFT; + +static int +sysctl_vfs_zfs_max_auto_ashift(SYSCTL_HANDLER_ARGS) +{ + uint64_t val; + int err; + + val = zfs_max_auto_ashift; + err = sysctl_handle_64(oidp, &val, 0, req); + if (err != 0 || req->newptr == NULL) + return (err); + + if (val > SPA_MAXASHIFT) + val = SPA_MAXASHIFT; + + zfs_max_auto_ashift = val; + + return (0); +} +SYSCTL_PROC(_vfs_zfs, OID_AUTO, max_auto_ashift, + CTLTYPE_U64 | CTLFLAG_MPSAFE | CTLFLAG_RW, 0, sizeof(uint64_t), + sysctl_vfs_zfs_max_auto_ashift, "QU", + "Cap on logical -> physical ashift adjustment on new top-level vdevs."); + static vdev_ops_t *vdev_ops_table[] = { &vdev_root_ops, &vdev_raidz_ops, @@ -746,6 +791,8 @@ vdev_add_parent(vdev_t *cvd, vdev_ops_t *ops) mvd->vdev_min_asize = cvd->vdev_min_asize; mvd->vdev_max_asize = cvd->vdev_max_asize; mvd->vdev_ashift = cvd->vdev_ashift; + mvd->vdev_logical_ashift = cvd->vdev_logical_ashift; + mvd->vdev_physical_ashift = cvd->vdev_physical_ashift; mvd->vdev_state = cvd->vdev_state; mvd->vdev_crtxg = cvd->vdev_crtxg; @@ -777,6 +824,8 @@ vdev_remove_parent(vdev_t *cvd) mvd->vdev_ops == &vdev_replacing_ops || mvd->vdev_ops == &vdev_spare_ops); cvd->vdev_ashift = mvd->vdev_ashift; + cvd->vdev_logical_ashift = mvd->vdev_logical_ashift; + cvd->vdev_physical_ashift = mvd->vdev_physical_ashift; vdev_remove_child(mvd, cvd); vdev_remove_child(pvd, mvd); @@ -1120,7 +1169,8 @@ vdev_open(vdev_t *vd) uint64_t osize = 0; uint64_t max_osize = 0; uint64_t asize, max_asize, psize; - uint64_t ashift = 0; + uint64_t logical_ashift = 0; + uint64_t physical_ashift = 0; ASSERT(vd->vdev_open_thread == curthread || spa_config_held(spa, SCL_STATE_ALL, RW_WRITER) == SCL_STATE_ALL); @@ -1150,7 +1200,8 @@ vdev_open(vdev_t *vd) return (SET_ERROR(ENXIO)); } - error = vd->vdev_ops->vdev_op_open(vd, &osize, &max_osize, &ashift); + error = vd->vdev_ops->vdev_op_open(vd, &osize, &max_osize, + &logical_ashift, &physical_ashift); /* * Reset the vdev_reopening flag so that we actually close @@ -1248,6 +1299,17 @@ vdev_open(vdev_t *vd) return (SET_ERROR(EINVAL)); } + vd->vdev_physical_ashift = + MAX(physical_ashift, vd->vdev_physical_ashift); + vd->vdev_logical_ashift = MAX(logical_ashift, vd->vdev_logical_ashift); + vd->vdev_ashift = MAX(vd->vdev_logical_ashift, vd->vdev_ashift); + + if (vd->vdev_logical_ashift > SPA_MAXASHIFT) { + vdev_set_state(vd, B_TRUE, VDEV_STATE_CANT_OPEN, + VDEV_AUX_ASHIFT_TOO_BIG); + return (EINVAL); + } + if (vd->vdev_asize == 0) { /* * This is the first-ever open, so use the computed values. @@ -1255,19 +1317,15 @@ vdev_open(vdev_t *vd) */ vd->vdev_asize = asize; vd->vdev_max_asize = max_asize; - vd->vdev_ashift = MAX(ashift, vd->vdev_ashift); } else { /* - * Detect if the alignment requirement has increased. - * We don't want to make the pool unavailable, just - * issue a warning instead. + * Make sure the alignment requirement hasn't increased. */ - if (ashift > vd->vdev_top->vdev_ashift && + if (vd->vdev_ashift > vd->vdev_top->vdev_ashift && vd->vdev_ops->vdev_op_leaf) { - cmn_err(CE_WARN, - "Disk, '%s', has a block alignment that is " - "larger than the pool's alignment\n", - vd->vdev_path); + vdev_set_state(vd, B_TRUE, VDEV_STATE_CANT_OPEN, + VDEV_AUX_BAD_LABEL); + return (EINVAL); } vd->vdev_max_asize = max_asize; } @@ -1577,7 +1635,24 @@ vdev_metaslab_set_size(vdev_t *vd) vd->vdev_ms_shift = MAX(vd->vdev_ms_shift, SPA_MAXBLOCKSHIFT); } +/* + * Maximize performance by inflating the configured ashift for + * top level vdevs to be as close to the physical ashift as + * possible without exceeding the administrator specified + * limit. + */ void +vdev_ashift_optimize(vdev_t *vd) +{ + if (vd == vd->vdev_top && + (vd->vdev_ashift < vd->vdev_physical_ashift) && + (vd->vdev_ashift < zfs_max_auto_ashift)) { + vd->vdev_ashift = MIN(zfs_max_auto_ashift, + vd->vdev_physical_ashift); + } +} + +void vdev_dirty(vdev_t *vd, int flags, void *arg, uint64_t txg) { ASSERT(vd == vd->vdev_top); @@ -2595,6 +2670,10 @@ vdev_get_stats(vdev_t *vd, vdev_stat_t *vs) if (vd->vdev_ops->vdev_op_leaf) vs->vs_rsize += VDEV_LABEL_START_SIZE + VDEV_LABEL_END_SIZE; vs->vs_esize = vd->vdev_max_asize - vd->vdev_asize; + vs->vs_configured_ashift = vd->vdev_top != NULL + ? vd->vdev_top->vdev_ashift : vd->vdev_ashift; + vs->vs_logical_ashift = vd->vdev_logical_ashift; + vs->vs_physical_ashift = vd->vdev_physical_ashift; mutex_exit(&vd->vdev_stat_lock); /* Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_file.c =================================================================== --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_file.c (revision 254492) +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_file.c (working copy) @@ -49,7 +49,7 @@ vdev_file_rele(vdev_t *vd) static int vdev_file_open(vdev_t *vd, uint64_t *psize, uint64_t *max_psize, - uint64_t *ashift) + uint64_t *logical_ashift, uint64_t *physical_ashift) { vdev_file_t *vf; vnode_t *vp; @@ -130,7 +130,8 @@ skip_open: } *max_psize = *psize = vattr.va_size; - *ashift = SPA_MINBLOCKSHIFT; + *logical_ashift = SPA_MINBLOCKSHIFT; + *physical_ashift = SPA_MINBLOCKSHIFT; return (0); } Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_geom.c =================================================================== --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_geom.c (revision 254492) +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_geom.c (working copy) @@ -576,7 +576,7 @@ vdev_geom_open_by_path(vdev_t *vd, int check_guid) static int vdev_geom_open(vdev_t *vd, uint64_t *psize, uint64_t *max_psize, - uint64_t *ashift) + uint64_t *logical_ashift, uint64_t *physical_ashift) { struct g_provider *pp; struct g_consumer *cp; @@ -662,9 +662,13 @@ vdev_geom_open(vdev_t *vd, uint64_t *psize, uint64 *max_psize = *psize = pp->mediasize; /* - * Determine the device's minimum transfer size. + * Determine the device's minimum transfer size and preferred + * transfer size. */ - *ashift = highbit(MAX(pp->sectorsize, SPA_MINBLOCKSIZE)) - 1; + *logical_ashift = highbit(MAX(pp->sectorsize, SPA_MINBLOCKSIZE)) - 1; + *physical_ashift = 0; + if (pp->stripesize) + *physical_ashift = highbit(pp->stripesize) - 1; /* * Clear the nowritecache settings, so that on a vdev_reopen() Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_mirror.c =================================================================== --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_mirror.c (revision 254492) +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_mirror.c (working copy) @@ -132,7 +132,7 @@ vdev_mirror_map_alloc(zio_t *zio) static int vdev_mirror_open(vdev_t *vd, uint64_t *asize, uint64_t *max_asize, - uint64_t *ashift) + uint64_t *logical_ashift, uint64_t *physical_ashift) { int numerrors = 0; int lasterror = 0; @@ -155,7 +155,9 @@ vdev_mirror_open(vdev_t *vd, uint64_t *asize, uint *asize = MIN(*asize - 1, cvd->vdev_asize - 1) + 1; *max_asize = MIN(*max_asize - 1, cvd->vdev_max_asize - 1) + 1; - *ashift = MAX(*ashift, cvd->vdev_ashift); + *logical_ashift = MAX(*logical_ashift, cvd->vdev_ashift); + *physical_ashift = MAX(*physical_ashift, + cvd->vdev_physical_ashift); } if (numerrors == vd->vdev_children) { Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_missing.c =================================================================== --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_missing.c (revision 254492) +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_missing.c (working copy) @@ -45,7 +45,7 @@ /* ARGSUSED */ static int vdev_missing_open(vdev_t *vd, uint64_t *psize, uint64_t *max_psize, - uint64_t *ashift) + uint64_t *logical_ashift, uint64_t *physical_ashift) { /* * Really this should just fail. But then the root vdev will be in the @@ -55,7 +55,8 @@ vdev_missing_open(vdev_t *vd, uint64_t *psize, uin */ *psize = 0; *max_psize = 0; - *ashift = 0; + *logical_ashift = 0; + *physical_ashift = 0; return (0); } Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_raidz.c =================================================================== --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_raidz.c (revision 254492) +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_raidz.c (working copy) @@ -1478,7 +1478,7 @@ vdev_raidz_reconstruct(raidz_map_t *rm, int *t, in static int vdev_raidz_open(vdev_t *vd, uint64_t *asize, uint64_t *max_asize, - uint64_t *ashift) + uint64_t *logical_ashift, uint64_t *physical_ashift) { vdev_t *cvd; uint64_t nparity = vd->vdev_nparity; @@ -1507,7 +1507,9 @@ vdev_raidz_open(vdev_t *vd, uint64_t *asize, uint6 *asize = MIN(*asize - 1, cvd->vdev_asize - 1) + 1; *max_asize = MIN(*max_asize - 1, cvd->vdev_max_asize - 1) + 1; - *ashift = MAX(*ashift, cvd->vdev_ashift); + *logical_ashift = MAX(*logical_ashift, cvd->vdev_ashift); + *physical_ashift = MAX(*physical_ashift, + cvd->vdev_physical_ashift); } *asize *= vd->vdev_children; Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_root.c =================================================================== --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_root.c (revision 254492) +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_root.c (working copy) @@ -55,7 +55,7 @@ too_many_errors(vdev_t *vd, int numerrors) static int vdev_root_open(vdev_t *vd, uint64_t *asize, uint64_t *max_asize, - uint64_t *ashift) + uint64_t *logical_ashift, uint64_t *physical_ashift) { int lasterror = 0; int numerrors = 0; @@ -83,7 +83,8 @@ vdev_root_open(vdev_t *vd, uint64_t *asize, uint64 *asize = 0; *max_asize = 0; - *ashift = 0; + *logical_ashift = 0; + *physical_ashift = 0; return (0); } Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c =================================================================== --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c (revision 254492) +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c (working copy) @@ -1137,8 +1137,10 @@ zio_write_bp_init(zio_t *zio) } if (compress != ZIO_COMPRESS_OFF) { + metaslab_class_t *mc = spa_normal_class(spa); void *cbuf = zio_buf_alloc(lsize); - psize = zio_compress_data(compress, zio->io_data, cbuf, lsize); + psize = zio_compress_data(compress, zio->io_data, cbuf, lsize, + (size_t)metaslab_class_get_minblocksize(mc)); if (psize == 0 || psize == lsize) { compress = ZIO_COMPRESS_OFF; zio_buf_free(cbuf, lsize); Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio_compress.c =================================================================== --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio_compress.c (revision 254492) +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio_compress.c (working copy) @@ -77,7 +77,8 @@ zio_compress_select(enum zio_compress child, enum } size_t -zio_compress_data(enum zio_compress c, void *src, void *dst, size_t s_len) +zio_compress_data(enum zio_compress c, void *src, void *dst, size_t s_len, + size_t minblocksize) { uint64_t *word, *word_end; size_t c_len, d_len, r_len; @@ -102,7 +103,7 @@ size_t return (s_len); /* Compress at least 12.5% */ - d_len = P2ALIGN(s_len - (s_len >> 3), (size_t)SPA_MINBLOCKSIZE); + d_len = P2ALIGN(s_len - (s_len >> 3), minblocksize); if (d_len == 0) return (s_len); @@ -115,14 +116,14 @@ size_t * Cool. We compressed at least as much as we were hoping to. * For both security and repeatability, pad out the last sector. */ - r_len = P2ROUNDUP(c_len, (size_t)SPA_MINBLOCKSIZE); + r_len = P2ROUNDUP(c_len, minblocksize); if (r_len > c_len) { bzero((char *)dst + c_len, r_len - c_len); c_len = r_len; } ASSERT3U(c_len, <=, d_len); - ASSERT(P2PHASE(c_len, (size_t)SPA_MINBLOCKSIZE) == 0); + ASSERT(P2PHASE(c_len, minblocksize) == 0); return (c_len); } Index: sys/cddl/contrib/opensolaris/uts/common/sys/fs/zfs.h =================================================================== --- sys/cddl/contrib/opensolaris/uts/common/sys/fs/zfs.h (revision 254492) +++ sys/cddl/contrib/opensolaris/uts/common/sys/fs/zfs.h (working copy) @@ -621,7 +621,8 @@ typedef enum vdev_aux { VDEV_AUX_IO_FAILURE, /* experienced I/O failure */ VDEV_AUX_BAD_LOG, /* cannot read log chain(s) */ VDEV_AUX_EXTERNAL, /* external diagnosis */ - VDEV_AUX_SPLIT_POOL /* vdev was split off into another pool */ + VDEV_AUX_SPLIT_POOL, /* vdev was split off into another pool */ + VDEV_AUX_ASHIFT_TOO_BIG /* vdev's min block size is too large */ } vdev_aux_t; /* @@ -715,7 +716,13 @@ typedef struct vdev_stat { uint64_t vs_self_healed; /* self-healed bytes */ uint64_t vs_scan_removing; /* removing? */ uint64_t vs_scan_processed; /* scan processed bytes */ + uint64_t vs_configured_ashift; /* TLV vdev_ashift */ + uint64_t vs_logical_ashift; /* vdev_logical_ashift */ + uint64_t vs_physical_ashift; /* vdev_physical_ashift */ } vdev_stat_t; +#define VDEV_STAT_VALID(field, uint64_t_field_count) \ + ((uint64_t_field_count * sizeof(uint64_t)) >= \ + (offsetof(vdev_stat_t, field) + sizeof(((vdev_stat_t *)NULL)->field))) /* * DDT statistics. Note: all fields should be 64-bit because this --Apple-Mail=_60E48F97-EC62-4D76-9B22-B5B59EF22B8D-- From owner-zfs-devel@FreeBSD.ORG Mon Aug 19 00:23:34 2013 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id A51D193B; Mon, 19 Aug 2013 00:23:34 +0000 (UTC) (envelope-from prvs=194305c2a8=killing@multiplay.co.uk) Received: from mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) (using TLSv1 with cipher RC4-MD5 (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 23B0C254B; Mon, 19 Aug 2013 00:23:33 +0000 (UTC) Received: from r2d2 ([82.69.141.170]) by mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) (MDaemon PRO v10.0.4) with ESMTP id md50005565426.msg; Mon, 19 Aug 2013 01:23:30 +0100 X-Spam-Processed: mail1.multiplay.co.uk, Mon, 19 Aug 2013 01:23:30 +0100 (not processed: message from valid local sender) X-MDDKIM-Result: neutral (mail1.multiplay.co.uk) X-MDRemoteIP: 82.69.141.170 X-Return-Path: prvs=194305c2a8=killing@multiplay.co.uk X-Envelope-From: killing@multiplay.co.uk Message-ID: From: "Steven Hartland" To: "Justin T. Gibbs" , , References: <984A4829-FBB0-4174-83FB-06A7336F4737@FreeBSD.org> Subject: Re: Updated ashift optimization patch Date: Mon, 19 Aug 2013 01:23:55 +0100 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="iso-8859-1"; reply-type=original Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.5931 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.6157 Cc: Richard Yao X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Aug 2013 00:23:34 -0000 Few things I've noticed:- 1. in vdev_geom.c is there a reason you use: + *physical_ashift = 0; + if (pp->stripesize) + *physical_ashift = highbit(pp->stripesize) - 1; Instead of: + *physical_ashift = highbit(MAX(pp->stripesize, SPA_MINBLOCKSIZE)) - 1; 2. sysctl_vfs_zfs_max_auto_ashift should also limit the min to > SPA_MINBLOCKSHIFT 3. sysctl_vfs_zfs_max_auto_ashift should error if the value is out of range instead of clamping it. One thing we did here, which would be good to see in this patch, is to add an overall min system ashift as thie enables admins to configure pools to be compatible with future disks they are likely to use e.g. min ashift 12 (4k compatible). This could be left at 9 by default for max compatibility but personally I'd suggest 12. Regard Steve ----- Original Message ----- From: "Justin T. Gibbs" To: ; Cc: "Richard Yao" Sent: Monday, August 19, 2013 12:42 AM Subject: Updated ashift optimization patch Today I re-merged all of Spectra's ashift related changes into FreeBSD/head. Here's the current state of the patch. See the top of the patch for proposed commit message/justification. The majority of it is ZFS platform agnostic. Later this week I'll get a stock FreeBSD/head system running and run the test suites in order to verify that I haven't botched anything in the merge. I'll also see about merging it to one of Will's Illumos VMs so I can generate a webrev. In the mean time, comments, concerns, improvements welcome. -- Justin -------------------------------------------------------------------------------- > _______________________________________________ > zfs-devel@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/zfs-devel > To unsubscribe, send any mail to "zfs-devel-unsubscribe@freebsd.org" ================================================ This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmaster@multiplay.co.uk. From owner-zfs-devel@FreeBSD.ORG Mon Aug 19 02:26:04 2013 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 377EFDCF for ; Mon, 19 Aug 2013 02:26:04 +0000 (UTC) (envelope-from gibbs@FreeBSD.org) Received: from aslan.scsiguy.com (aslan.scsiguy.com [70.89.174.89]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id DC7862AC3 for ; Mon, 19 Aug 2013 02:26:03 +0000 (UTC) Received: from [192.168.0.99] (macbook.scsiguy.com [192.168.0.99]) (authenticated bits=0) by aslan.scsiguy.com (8.14.7/8.14.5) with ESMTP id r7J2Q0TX092338 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NO); Mon, 19 Aug 2013 02:26:01 GMT (envelope-from gibbs@FreeBSD.org) Subject: Re: Updated ashift optimization patch Mime-Version: 1.0 (Mac OS X Mail 6.5 \(1508\)) Content-Type: text/plain; charset=iso-8859-1 From: "Justin T. Gibbs" X-Priority: 3 In-Reply-To: Date: Sun, 18 Aug 2013 20:26:00 -0600 Content-Transfer-Encoding: quoted-printable Message-Id: <8375F87B-F406-491F-BE75-0FAECED680E6@FreeBSD.org> References: <984A4829-FBB0-4174-83FB-06A7336F4737@FreeBSD.org> To: "Steven Hartland" X-Mailer: Apple Mail (2.1508) X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.4.3 (aslan.scsiguy.com [70.89.174.89]); Mon, 19 Aug 2013 02:26:02 +0000 (UTC) Cc: zfs-devel@freebsd.org, zfs@lists.illumos.org, Richard Yao X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Aug 2013 02:26:04 -0000 On Aug 18, 2013, at 6:23 PM, "Steven Hartland" = wrote: > Few things I've noticed:- >=20 > 1. in vdev_geom.c is there a reason you use: > + *physical_ashift =3D 0; > + if (pp->stripesize) > + *physical_ashift =3D highbit(pp->stripesize) - 1; >=20 > Instead of: > + *physical_ashift =3D highbit(MAX(pp->stripesize, SPA_MINBLOCKSIZE)) = - 1; 0 in ashift as in stripesize means "unset" or "don't care". This gives = more information during debugging ("Oh. The driver for this device must = not be setting this.") than if it were clamped. If we decide it should = always be set, I think "MAX(pp->sectorsize, pp->stripesize)" would make = more sense. > 2. sysctl_vfs_zfs_max_auto_ashift should also limit the min to > > SPA_MINBLOCKSHIFT Setting it too low (any value less than the device's logical ashift) = just disables the feature. I don't see any value in adding code to = error out or clamp considering the most likely reason for this to occur = is an administrator trying to turn it off (e.g. set it to 0). > 3. sysctl_vfs_zfs_max_auto_ashift should error if the value is out > of range instead of clamping it. Ok. I now return EINVAL if the value exceeds SPA_MAXASHIFT. > One thing we did here, which would be good to see in this patch, is to = add > an overall min system ashift as thie enables admins to configure pools > to be compatible with future disks they are likely to use e.g. min = ashift > 12 (4k compatible). This could be left at 9 by default for max = compatibility > but personally I'd suggest 12. it would be nice to hear more consensus come out of the recent zpool = ashift discussions before doing something here. Whatever that is, I = agree that the default should be 9 until someone fixes the RAIDZ space = penalty for going to 4k on 512N drives. -- Justin= From owner-zfs-devel@FreeBSD.ORG Mon Aug 19 12:04:41 2013 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 04283CC8; Mon, 19 Aug 2013 12:04:41 +0000 (UTC) (envelope-from prvs=194305c2a8=killing@multiplay.co.uk) Received: from mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) (using TLSv1 with cipher RC4-MD5 (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 3B1A829E9; Mon, 19 Aug 2013 12:04:39 +0000 (UTC) Received: from r2d2 ([82.69.141.170]) by mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) (MDaemon PRO v10.0.4) with ESMTP id md50005572791.msg; Mon, 19 Aug 2013 13:04:37 +0100 X-Spam-Processed: mail1.multiplay.co.uk, Mon, 19 Aug 2013 13:04:37 +0100 (not processed: message from valid local sender) X-MDDKIM-Result: neutral (mail1.multiplay.co.uk) X-MDRemoteIP: 82.69.141.170 X-Return-Path: prvs=194305c2a8=killing@multiplay.co.uk X-Envelope-From: killing@multiplay.co.uk Message-ID: <37A9F30CF2AB4445BFAE1402BA705B9B@multiplay.co.uk> From: "Steven Hartland" To: "Will Andrews" , "Matthew Ahrens" References: <20130712112117.Horde.JMB3fhsU1VfZRxdV4prj5Q6@mail.vx.sk><51FE1E1E.8080502@FreeBSD.org><5C28F2D739D94E6B928ACD55D207665A@multiplay.co.uk><51FE9BF5.2000301@FreeBSD.org><51FEA530.2090008@FreeBSD.org><4B747F3B634B4A859068FC8D782A7E1D@multiplay.co.uk><52011DE6.8090708@FreeBSD.org><4DA6B66E61374BE68C895C8B9A93BCB7@multiplay.co.uk><520C9001.2040406@FreeBSD.org><78BAB3D2-86A8-404C-8D5A-5B4E89AF1208@FreeBSD.org><12BBEDC5C5AF4F8180D6A6B09AD695A6@multiplay.co.uk><57C8A903D51A44D7A8A38C19F494CF99@multiplay.co.uk> <3F20CA2BA01E4B07899B4CB82EB0E220@multiplay.co.uk> Subject: Re: N-way mirror read speedup in zfsonlinux Date: Mon, 19 Aug 2013 13:04:58 +0100 MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="----=_NextPart_000_05EF_01CE9CDC.B5538190" X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.5931 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.6157 Cc: zfs-devel@freebsd.org, Xin Li X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Aug 2013 12:04:41 -0000 This is a multi-part message in MIME format. ------=_NextPart_000_05EF_01CE9CDC.B5538190 Content-Type: text/plain; format=flowed; charset="iso-8859-1"; reply-type=response Content-Transfer-Encoding: 7bit Ok so latest version attached with the following changes: * Switched from d_nonrotating -> d_rotation_rate for more flexibility in the future. * Added g_handleattr_uint16_t to correctly handle uint16_t without hard coding size. * Switched back to for() {...} from do {..} while() * Added non_rotating_seek_inc option for controlling the seek increment for non rotating media. Regards Steve ================================================ This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmaster@multiplay.co.uk. ------=_NextPart_000_05EF_01CE9CDC.B5538190 Content-Type: application/octet-stream; name="zfs-mirror-load.patch" Content-Transfer-Encoding: quoted-printable Content-Disposition: attachment; filename="zfs-mirror-load.patch" Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/vdev.h=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/vdev.h (revision = 254419)=0A= +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/vdev.h (working = copy)=0A= @@ -119,6 +119,9 @@=0A= extern void vdev_queue_fini(vdev_t *vd);=0A= extern zio_t *vdev_queue_io(zio_t *zio);=0A= extern void vdev_queue_io_done(zio_t *zio);=0A= +extern int vdev_queue_length(vdev_t *vd);=0A= +extern uint64_t vdev_queue_lastoffset(vdev_t *vd);=0A= +extern void vdev_queue_register_lastoffset(vdev_t *vd, zio_t *zio);=0A= =0A= extern void vdev_config_dirty(vdev_t *vd);=0A= extern void vdev_config_clean(vdev_t *vd);=0A= Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/vdev_impl.h=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/vdev_impl.h = (revision 254419)=0A= +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/vdev_impl.h = (working copy)=0A= @@ -106,6 +106,7 @@=0A= avl_tree_t vq_pending_tree;=0A= hrtime_t vq_io_complete_ts;=0A= kmutex_t vq_lock;=0A= + uint64_t vq_lastoffset;=0A= };=0A= =0A= /*=0A= @@ -199,7 +200,10 @@=0A= spa_aux_vdev_t *vdev_aux; /* for l2cache vdevs */=0A= zio_t *vdev_probe_zio; /* root of current probe */=0A= vdev_aux_t vdev_label_aux; /* on-disk aux state */=0A= - struct trim_map *vdev_trimmap;=0A= + struct trim_map *vdev_trimmap; /* map on outstanding trims */ =0A= + uint16_t vdev_rotation_rate; /* rotational rate of the media */=0A= +#define VDEV_RATE_UNKNOWN 0=0A= +#define VDEV_RATE_NON_ROTATING 1=0A= =0A= /*=0A= * For DTrace to work in userland (libzpool) context, these fields must=0A= Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_geom.c=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_geom.c (revision = 254419)=0A= +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_geom.c (working = copy)=0A= @@ -42,9 +42,11 @@=0A= * Virtual device vector for GEOM.=0A= */=0A= =0A= +static g_attrchanged_t vdev_geom_attrchanged;=0A= struct g_class zfs_vdev_class =3D {=0A= .name =3D "ZFS::VDEV",=0A= .version =3D G_VERSION,=0A= + .attrchanged =3D vdev_geom_attrchanged,=0A= };=0A= =0A= DECLARE_GEOM_CLASS(zfs_vdev_class, zfs_vdev);=0A= @@ -62,6 +64,34 @@=0A= &vdev_geom_bio_delete_disable, 0, "Disable BIO_DELETE");=0A= =0A= static void=0A= +vdev_geom_set_rotation_rate(vdev_t *vd, struct g_consumer *cp)=0A= +{ =0A= + int error;=0A= + uint16_t rate;=0A= +=0A= + error =3D g_getattr("GEOM::rotation_rate", cp, &rate);=0A= + if (error =3D=3D 0)=0A= + vd->vdev_rotation_rate =3D rate;=0A= + else=0A= + vd->vdev_rotation_rate =3D VDEV_RATE_UNKNOWN;=0A= +}=0A= +=0A= +static void=0A= +vdev_geom_attrchanged(struct g_consumer *cp, const char *attr)=0A= +{=0A= + vdev_t *vd;=0A= +=0A= + vd =3D cp->private;=0A= + if (vd =3D=3D NULL)=0A= + return;=0A= +=0A= + if (strcmp(attr, "GEOM::rotation_rate") =3D=3D 0) {=0A= + vdev_geom_set_rotation_rate(vd, cp);=0A= + return;=0A= + }=0A= +}=0A= +=0A= +static void=0A= vdev_geom_orphan(struct g_consumer *cp)=0A= {=0A= vdev_t *vd;=0A= @@ -678,6 +708,11 @@=0A= vd->vdev_physpath =3D kmem_alloc(bufsize, KM_SLEEP);=0A= snprintf(vd->vdev_physpath, bufsize, "/dev/%s", pp->name);=0A= =0A= + /*=0A= + * Determine the device's rotation rate.=0A= + */=0A= + vdev_geom_set_rotation_rate(vd, cp);=0A= +=0A= return (0);=0A= }=0A= =0A= Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_mirror.c=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_mirror.c = (revision 254419)=0A= +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_mirror.c = (working copy)=0A= @@ -41,6 +41,7 @@=0A= vdev_t *mc_vd;=0A= uint64_t mc_offset;=0A= int mc_error;=0A= + int mc_load;=0A= uint8_t mc_tried;=0A= uint8_t mc_skipped;=0A= uint8_t mc_speculative;=0A= @@ -54,8 +55,53 @@=0A= mirror_child_t mm_child[1];=0A= } mirror_map_t;=0A= =0A= -int vdev_mirror_shift =3D 21;=0A= +SYSCTL_DECL(_vfs_zfs_vdev);=0A= +static SYSCTL_NODE(_vfs_zfs_vdev, OID_AUTO, mirror, CTLFLAG_RD, 0,=0A= + "ZFS VDEV Mirror");=0A= =0A= +/*=0A= + * The load configuration settings below are tuned by default for=0A= + * the case where all devices are of the same rotational type.=0A= + *=0A= + * If there is a mixture of rotating and non-rotating media, setting=0A= + * non_rotating_seek_inc to 0 may well provide better results as it=0A= + * will direct more reads to the non-rotating vdevs which are more=0A= + * likely to have a higher performance.=0A= + */=0A= +=0A= +/* Rotating media load calculation configuration. */=0A= +static int rotating_inc =3D 0;=0A= +TUNABLE_INT("vfs.zfs.vdev.mirror.rotating_inc", &rotating_inc);=0A= +SYSCTL_INT(_vfs_zfs_vdev_mirror, OID_AUTO, rotating_inc, CTLFLAG_RW,=0A= + &rotating_inc, 0, "Rotating media load increment for non-seeking = I/O's");=0A= +=0A= +static int rotating_seek_inc =3D 5;=0A= +TUNABLE_INT("vfs.zfs.vdev.mirror.rotating_seek_inc", = &rotating_seek_inc);=0A= +SYSCTL_INT(_vfs_zfs_vdev_mirror, OID_AUTO, rotating_seek_inc, = CTLFLAG_RW,=0A= + &rotating_seek_inc, 0, "Rotating media load increment for seeking = I/O's");=0A= +=0A= +static int rotating_seek_offset =3D 1 * 1024 * 1024;=0A= +TUNABLE_INT("vfs.zfs.vdev.mirror.rotating_seek_offset", = &rotating_seek_offset);=0A= +SYSCTL_INT(_vfs_zfs_vdev_mirror, OID_AUTO, rotating_seek_offset, = CTLFLAG_RW,=0A= + &rotating_seek_offset, 0, "Offset in bytes from the last I/O which "=0A= + "triggers a reduced rotating media seek increment");=0A= +=0A= +/* Non-rotating media load calculation configuration. */=0A= +static int non_rotating_inc =3D 0;=0A= +TUNABLE_INT("vfs.zfs.vdev.mirror.non_rotating_inc", &non_rotating_inc);=0A= +SYSCTL_INT(_vfs_zfs_vdev_mirror, OID_AUTO, non_rotating_inc, CTLFLAG_RW,=0A= + &non_rotating_inc, 0,=0A= + "Non-rotating media load increment for non-seeking I/O's");=0A= +=0A= +static int non_rotating_seek_inc =3D 1;=0A= +TUNABLE_INT("vfs.zfs.vdev.mirror.non_rotating_seek_inc",=0A= + &non_rotating_seek_inc);=0A= +SYSCTL_INT(_vfs_zfs_vdev_mirror, OID_AUTO, non_rotating_seek_inc, = CTLFLAG_RW,=0A= + &non_rotating_seek_inc, 0,=0A= + "Non-rotating media load increment for seeking I/O's");=0A= +=0A= +static int vdev_mirror_shift =3D 21;=0A= +=0A= static void=0A= vdev_mirror_map_free(zio_t *zio)=0A= {=0A= @@ -69,6 +115,56 @@=0A= zio_vsd_default_cksum_report=0A= };=0A= =0A= +static int=0A= +vdev_mirror_load(vdev_t *vd, uint64_t zio_offset)=0A= +{=0A= + uint64_t lastoffset;=0A= + int load;=0A= +=0A= + /*=0A= + * We don't return INT_MAX if the device is resilvering i.e.=0A= + * vdev_resilver_txg !=3D 0 as when tested performance was slightly=0A= + * worse overall when resilvering with compared to without.=0A= + */=0A= +=0A= + /* If the vdev isn't readable use the maximum load. */=0A= + if (!vdev_readable(vd))=0A= + return (INT_MAX);=0A= +=0A= + /* Standard load based on pending queue length. */=0A= + load =3D vdev_queue_length(vd);=0A= + lastoffset =3D vdev_queue_lastoffset(vd);=0A= +=0A= + /* Non-rotating media. */=0A= + if (vd->vdev_rotation_rate =3D=3D VDEV_RATE_NON_ROTATING) {=0A= + if (lastoffset =3D=3D zio_offset)=0A= + return (load + non_rotating_inc);=0A= +=0A= + /*=0A= + * Apply a seek penalty even for non-rotating devices as=0A= + * sequential I/O'a can be aggregated into fewer operations=0A= + * on the device, thus avoiding unnecessary per-command=0A= + * overhead and boosting performance.=0A= + */=0A= + return (load + non_rotating_seek_inc);=0A= + }=0A= +=0A= + /* I/O's which directly follow the last I/O. */=0A= + if (lastoffset =3D=3D zio_offset)=0A= + return (load + rotating_inc);=0A= +=0A= + /*=0A= + * Apply half the seek increment to I/O's within seek offset=0A= + * of the last I/O queued to this vdev as they should incure less=0A= + * of a seek increment.=0A= + */=0A= + if (ABS(lastoffset - zio_offset) < rotating_seek_offset)=0A= + return (load + (rotating_seek_inc / 2));=0A= +=0A= + /* Apply the full seek increment to all other I/O's. */=0A= + return (load + rotating_seek_inc);=0A= +}=0A= +=0A= static mirror_map_t *=0A= vdev_mirror_map_alloc(zio_t *zio)=0A= {=0A= @@ -108,21 +204,62 @@=0A= mc->mc_offset =3D DVA_GET_OFFSET(&dva[c]);=0A= }=0A= } else {=0A= + int lowest_load, lowest_child, lowest_nr;=0A= + uint64_t zio_offset;=0A= +=0A= c =3D vd->vdev_children;=0A= =0A= mm =3D kmem_zalloc(offsetof(mirror_map_t, mm_child[c]), KM_SLEEP);=0A= mm->mm_children =3D c;=0A= mm->mm_replacing =3D (vd->vdev_ops =3D=3D &vdev_replacing_ops ||=0A= vd->vdev_ops =3D=3D &vdev_spare_ops);=0A= - mm->mm_preferred =3D mm->mm_replacing ? 0 :=0A= - (zio->io_offset >> vdev_mirror_shift) % c;=0A= mm->mm_root =3D B_FALSE;=0A= =0A= + zio_offset =3D zio->io_offset;=0A= + lowest_load =3D INT_MAX;=0A= + lowest_child =3D 0;=0A= + lowest_nr =3D 1;=0A= +=0A= for (c =3D 0; c < mm->mm_children; c++) {=0A= mc =3D &mm->mm_child[c];=0A= mc->mc_vd =3D vd->vdev_child[c];=0A= - mc->mc_offset =3D zio->io_offset;=0A= + mc->mc_offset =3D zio_offset;=0A= +=0A= + if (zio->io_type !=3D ZIO_TYPE_READ)=0A= + continue;=0A= +=0A= + mc->mc_load =3D vdev_mirror_load(mc->mc_vd, zio_offset);=0A= + if (mc->mc_load < lowest_load) {=0A= + lowest_load =3D mc->mc_load;=0A= + lowest_child =3D c; =0A= + lowest_nr =3D 1;=0A= + } else if (mc->mc_load =3D=3D lowest_load) {=0A= + lowest_nr++;=0A= + }=0A= }=0A= +=0A= + if (lowest_nr !=3D 1) {=0A= + int d;=0A= +=0A= + /*=0A= + * To ensure we don't always favour the first=0A= + * matching vdev, which could lead to wear=0A= + * leveling issues on SSD's, we use the I/O=0A= + * offset as a sudo random seed into the vdevs=0A= + * which have the lowest load.=0A= + */=0A= + d =3D (zio_offset >> vdev_mirror_shift)=0A= + % lowest_nr;=0A= + for (c =3D lowest_child + 1; d !=3D 0; c++) {=0A= + if (mm->mm_child[c].mc_load =3D=3D lowest_load) {=0A= + lowest_child =3D c;=0A= + d--;=0A= + }=0A= + }=0A= + }=0A= + mm->mm_preferred =3D lowest_child;=0A= + vdev_queue_register_lastoffset(vd->vdev_child[lowest_child],=0A= + zio);=0A= }=0A= =0A= zio->io_vsd =3D mm;=0A= Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c = (revision 254419)=0A= +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c (working = copy)=0A= @@ -155,6 +155,8 @@=0A= =0A= avl_create(&vq->vq_pending_tree, vdev_queue_offset_compare,=0A= sizeof (zio_t), offsetof(struct zio, io_offset_node));=0A= +=0A= + vq->vq_lastoffset =3D 0;=0A= }=0A= =0A= void=0A= @@ -446,3 +448,26 @@=0A= =0A= mutex_exit(&vq->vq_lock);=0A= }=0A= +=0A= +/*=0A= + * As these three methods are only used for load calculations we're not = precious=0A= + * if we get an incorrect value on 32bit platforms due to lack of = vq_lock mutex=0A= + * uses here. Instead we prefer to keep it lock free for the = performance.=0A= + */ =0A= +int=0A= +vdev_queue_length(vdev_t *vd)=0A= +{=0A= + return (avl_numnodes(&vd->vdev_queue.vq_pending_tree));=0A= +}=0A= +=0A= +uint64_t=0A= +vdev_queue_lastoffset(vdev_t *vd)=0A= +{=0A= + return (vd->vdev_queue.vq_lastoffset);=0A= +}=0A= +=0A= +void=0A= +vdev_queue_register_lastoffset(vdev_t *vd, zio_t *zio)=0A= +{=0A= + vd->vdev_queue.vq_lastoffset =3D zio->io_offset + zio->io_size;=0A= +}=0A= Index: sys/geom/geom.h=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/geom/geom.h (revision 254419)=0A= +++ sys/geom/geom.h (working copy)=0A= @@ -270,6 +270,7 @@=0A= int len);=0A= int g_handleattr_int(struct bio *bp, const char *attribute, int val);=0A= int g_handleattr_off_t(struct bio *bp, const char *attribute, off_t = val);=0A= +int g_handleattr_uint16_t(struct bio *bp, const char *attribute, = uint16_t val);=0A= int g_handleattr_str(struct bio *bp, const char *attribute, const char = *str);=0A= struct g_consumer * g_new_consumer(struct g_geom *gp);=0A= struct g_geom * g_new_geomf(struct g_class *mp, const char *fmt, ...)=0A= Index: sys/geom/geom_disk.c=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/geom/geom_disk.c (revision 254419)=0A= +++ sys/geom/geom_disk.c (working copy)=0A= @@ -376,22 +376,25 @@=0A= break;=0A= else if (g_handleattr_str(bp, "GEOM::ident", dp->d_ident))=0A= break;=0A= - else if (g_handleattr(bp, "GEOM::hba_vendor",=0A= - &dp->d_hba_vendor, 2))=0A= + else if (g_handleattr_uint16_t(bp, "GEOM::hba_vendor",=0A= + dp->d_hba_vendor))=0A= break;=0A= - else if (g_handleattr(bp, "GEOM::hba_device",=0A= - &dp->d_hba_device, 2))=0A= + else if (g_handleattr_uint16_t(bp, "GEOM::hba_device",=0A= + dp->d_hba_device))=0A= break;=0A= - else if (g_handleattr(bp, "GEOM::hba_subvendor",=0A= - &dp->d_hba_subvendor, 2))=0A= + else if (g_handleattr_uint16_t(bp, "GEOM::hba_subvendor",=0A= + dp->d_hba_subvendor))=0A= break;=0A= - else if (g_handleattr(bp, "GEOM::hba_subdevice",=0A= - &dp->d_hba_subdevice, 2))=0A= + else if (g_handleattr_uint16_t(bp, "GEOM::hba_subdevice",=0A= + dp->d_hba_subdevice))=0A= break;=0A= else if (!strcmp(bp->bio_attribute, "GEOM::kerneldump"))=0A= g_disk_kerneldump(bp, dp);=0A= else if (!strcmp(bp->bio_attribute, "GEOM::setstate"))=0A= g_disk_setstate(bp, sc);=0A= + else if (g_handleattr_uint16_t(bp, "GEOM::rotation_rate",=0A= + dp->d_rotation_rate))=0A= + break;=0A= else =0A= error =3D ENOIOCTL;=0A= break;=0A= Index: sys/geom/geom_disk.h=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/geom/geom_disk.h (revision 254419)=0A= +++ sys/geom/geom_disk.h (working copy)=0A= @@ -97,6 +97,7 @@=0A= uint16_t d_hba_device;=0A= uint16_t d_hba_subvendor;=0A= uint16_t d_hba_subdevice;=0A= + uint16_t d_rotation_rate;=0A= =0A= /* Fields private to the driver */=0A= void *d_drv1;=0A= @@ -121,7 +122,8 @@=0A= #define DISK_VERSION_01 0x5856105a=0A= #define DISK_VERSION_02 0x5856105b=0A= #define DISK_VERSION_03 0x5856105c=0A= -#define DISK_VERSION DISK_VERSION_03=0A= +#define DISK_VERSION_04 0x5856105d=0A= +#define DISK_VERSION DISK_VERSION_04=0A= =0A= #endif /* _KERNEL */=0A= #endif /* _GEOM_GEOM_DISK_H_ */=0A= Index: sys/geom/geom_subr.c=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/geom/geom_subr.c (revision 254419)=0A= +++ sys/geom/geom_subr.c (working copy)=0A= @@ -949,6 +949,13 @@=0A= }=0A= =0A= int=0A= +g_handleattr_uint16_t(struct bio *bp, const char *attribute, uint16_t = val)=0A= +{=0A= +=0A= + return (g_handleattr(bp, attribute, &val, sizeof val));=0A= +}=0A= +=0A= +int=0A= g_handleattr_off_t(struct bio *bp, const char *attribute, off_t val)=0A= {=0A= =0A= Index: sys/cam/ata/ata_da.c=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/cam/ata/ata_da.c (revision 254419)=0A= +++ sys/cam/ata/ata_da.c (working copy)=0A= @@ -1224,13 +1224,14 @@=0A= snprintf(announce_buf, sizeof(announce_buf),=0A= "kern.cam.ada.%d.write_cache", periph->unit_number);=0A= TUNABLE_INT_FETCH(announce_buf, &softc->write_cache);=0A= + adagetparams(periph, cgd);=0A= + softc->disk =3D disk_alloc();=0A= /* Disable queue sorting for non-rotational media by default. */=0A= - if (cgd->ident_data.media_rotation_rate =3D=3D 1)=0A= + if (cgd->ident_data.media_rotation_rate =3D=3D ATA_RATE_NON_ROTATING)=0A= softc->sort_io_queue =3D 0;=0A= else=0A= softc->sort_io_queue =3D -1;=0A= - adagetparams(periph, cgd);=0A= - softc->disk =3D disk_alloc();=0A= + softc->disk->d_rotation_rate =3D cgd->ident_data.media_rotation_rate;=0A= softc->disk->d_devstat =3D devstat_new_entry(periph->periph_name,=0A= periph->unit_number, softc->params.secsize,=0A= DEVSTAT_ALL_SUPPORTED,=0A= Index: sys/cam/scsi/scsi_all.h=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/cam/scsi/scsi_all.h (revision 254419)=0A= +++ sys/cam/scsi/scsi_all.h (working copy)=0A= @@ -1451,7 +1451,7 @@=0A= u_int8_t page_length[2];=0A= u_int8_t medium_rotation_rate[2];=0A= #define SVPD_BDC_RATE_NOT_REPORTED 0x00=0A= -#define SVPD_BDC_RATE_NONE_ROTATING 0x01=0A= +#define SVPD_BDC_RATE_NON_ROTATING 0x01=0A= u_int8_t reserved1;=0A= u_int8_t nominal_form_factor;=0A= #define SVPD_BDC_FORM_NOT_REPORTED 0x00=0A= Index: sys/cam/scsi/scsi_da.c=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/cam/scsi/scsi_da.c (revision 254419)=0A= +++ sys/cam/scsi/scsi_da.c (working copy)=0A= @@ -3386,9 +3386,17 @@=0A= * Disable queue sorting for non-rotational media=0A= * by default.=0A= */=0A= - if (scsi_2btoul(bdc->medium_rotation_rate) =3D=3D=0A= - SVPD_BDC_RATE_NONE_ROTATING)=0A= + u_int old_rate =3D softc->disk->d_rotation_rate;=0A= + softc->disk->d_rotation_rate =3D=0A= + scsi_2btoul(bdc->medium_rotation_rate);=0A= + if (softc->disk->d_rotation_rate =3D=3D=0A= + SVPD_BDC_RATE_NON_ROTATING) {=0A= softc->sort_io_queue =3D 0;=0A= + }=0A= + if (softc->disk->d_rotation_rate !=3D old_rate) {=0A= + disk_attr_changed(softc->disk,=0A= + "GEOM::rotation_rate", M_NOWAIT);=0A= + }=0A= } else {=0A= int error;=0A= error =3D daerror(done_ccb, CAM_RETRY_SELTO,=0A= @@ -3423,6 +3431,8 @@=0A= ptr =3D (uint16_t *)ata_params;=0A= =0A= if ((csio->ccb_h.status & CAM_STATUS_MASK) =3D=3D CAM_REQ_CMP) {=0A= + uint16_t old_rate;=0A= +=0A= for (i =3D 0; i < sizeof(*ata_params) / 2; i++)=0A= ptr[i] =3D le16toh(ptr[i]);=0A= if (ata_params->support_dsm & ATA_SUPPORT_DSM_TRIM) {=0A= @@ -3437,8 +3447,18 @@=0A= * Disable queue sorting for non-rotational media=0A= * by default.=0A= */=0A= - if (ata_params->media_rotation_rate =3D=3D 1)=0A= + old_rate =3D softc->disk->d_rotation_rate;=0A= + softc->disk->d_rotation_rate =3D=0A= + ata_params->media_rotation_rate;=0A= + if (softc->disk->d_rotation_rate =3D=3D=0A= + ATA_RATE_NON_ROTATING) {=0A= softc->sort_io_queue =3D 0;=0A= + }=0A= +=0A= + if (softc->disk->d_rotation_rate !=3D old_rate) {=0A= + disk_attr_changed(softc->disk,=0A= + "GEOM::rotation_rate", M_NOWAIT);=0A= + }=0A= } else {=0A= int error;=0A= error =3D daerror(done_ccb, CAM_RETRY_SELTO,=0A= ------=_NextPart_000_05EF_01CE9CDC.B5538190-- From owner-zfs-devel@FreeBSD.ORG Mon Aug 19 20:48:56 2013 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 4BB0CF09; Mon, 19 Aug 2013 20:48:56 +0000 (UTC) (envelope-from ryao@gentoo.org) Received: from smtp.gentoo.org (smtp.gentoo.org [140.211.166.183]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 2721927ED; Mon, 19 Aug 2013 20:48:55 +0000 (UTC) Received: from [192.168.1.2] (pool-173-52-119-53.nycmny.fios.verizon.net [173.52.119.53]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) (Authenticated sender: ryao) by smtp.gentoo.org (Postfix) with ESMTPSA id 1FCEF33EBE7; Mon, 19 Aug 2013 20:48:48 +0000 (UTC) Message-ID: <52128494.2030206@gentoo.org> Date: Mon, 19 Aug 2013 16:48:20 -0400 From: Richard Yao User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:17.0) Gecko/20130815 Thunderbird/17.0.8 MIME-Version: 1.0 To: "Justin T. Gibbs" Subject: Re: Updated ashift optimization patch References: <984A4829-FBB0-4174-83FB-06A7336F4737@FreeBSD.org> <8375F87B-F406-491F-BE75-0FAECED680E6@FreeBSD.org> In-Reply-To: <8375F87B-F406-491F-BE75-0FAECED680E6@FreeBSD.org> X-Enigmail-Version: 1.5.2 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="jxV0X1C0XFh9uC8ICh4wOvOPNh6mpHB58" Cc: zfs-devel@freebsd.org, zfs@lists.illumos.org X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Aug 2013 20:48:56 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --jxV0X1C0XFh9uC8ICh4wOvOPNh6mpHB58 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable On 08/18/2013 10:26 PM, Justin T. Gibbs wrote:>> One thing we did here, which would be good to see in this patch, is to add >> an overall min system ashift as thie enables admins to configure pools= >> to be compatible with future disks they are likely to use e.g. min ash= ift >> 12 (4k compatible). This could be left at 9 by default for max compatibility >> but personally I'd suggest 12. > > it would be nice to hear more consensus come out of the recent zpool ashift discussions before doing something here. Whatever that is, I agree that the default should be 9 until someone fixes the RAIDZ space penalty for going to 4k on 512N drives. > > -- > Justin > The default ashift is determined dynamically per top-level vdev by the highest block_size reported by any member at creation time. There is no fixed default ashift. However, many drives lie, so it could appear that w= ay. --jxV0X1C0XFh9uC8ICh4wOvOPNh6mpHB58 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.20 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQIcBAEBAgAGBQJSEoSbAAoJECDuEZm+6ExkAlkP/3i4jtpTkevWWPuvONPQTOQy 2rlVHXCbmve6PU0hytMdQyvYPR8/VreXHc2L4cP14gBYQsmLFNuOy8dqzLOw6iNo G2ZgyaX3OzbNCDp4p0YvWuc4B20gpMvaI0LEU3rV5FFC5hALkDmE5ZCDk5rwEfks 61prS2TmpdCLt4ZJvwgWVYFDQT3iIpxHevcdqlQh7tPyonGqXOU5nvlZVEoczDLt PWFbycFhxT+EtRU0iiL5mNl3rPI/A/igpeM4W8ap+kVuxDa2967D/cdluBIB25Ov US9mmM7SA2o87vmhG46DsPGdahIoC4QvIXGGPOh1yZoNcxV6tER5eE0qf1UIMUeH kGeseawYOSYqzN6xj/7BQIMkiTxhPQNtGqNoOgVGa5nuqN044wYkj0KD5EdgBoDR ghucFhnLj/MjUpAl5WmsokANj7U6q7ikDMhIC+QnAjHUcSq0/8BA6UH4sx948v93 fk/mS9Y3rDB4TlqRoJm87NaDiRiGowp59PNdgdihRoRnVjx7n2DoQR3jWog8qo9I BsjF1BA/KEMgRGt85aVg/amKfm+SIHypmjRhaDyElJdvZu/Lq6JzMw4BG/1V8nza 6XSxcwJyC9wupp+tZO85+GOWoaB9GrZcDq/RxHZdDEqqbdClTuKoaNRNVPnUy9Kd R6QyczwVrCtDSk1onI6j =rypD -----END PGP SIGNATURE----- --jxV0X1C0XFh9uC8ICh4wOvOPNh6mpHB58-- From owner-zfs-devel@FreeBSD.ORG Mon Aug 19 22:23:07 2013 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 1E275A02; Mon, 19 Aug 2013 22:23:07 +0000 (UTC) (envelope-from gibbs@FreeBSD.org) Received: from aslan.scsiguy.com (mail.scsiguy.com [70.89.174.89]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id D26C02CB6; Mon, 19 Aug 2013 22:23:06 +0000 (UTC) Received: from [192.168.6.155] (207-225-98-3.dia.static.qwest.net [207.225.98.3]) (authenticated bits=0) by aslan.scsiguy.com (8.14.7/8.14.5) with ESMTP id r7JMN1k7099735 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NO); Mon, 19 Aug 2013 22:23:02 GMT (envelope-from gibbs@FreeBSD.org) Subject: Re: N-way mirror read speedup in zfsonlinux Mime-Version: 1.0 (Mac OS X Mail 6.5 \(1508\)) Content-Type: multipart/mixed; boundary="Apple-Mail=_47935BC7-A11B-4756-81AE-A6FEF0A33EDB" From: "Justin T. Gibbs" X-Priority: 3 In-Reply-To: <37A9F30CF2AB4445BFAE1402BA705B9B@multiplay.co.uk> Date: Mon, 19 Aug 2013 16:22:56 -0600 Message-Id: <1A5316AB-4E00-4C2B-B023-86ED6F1E6CC1@FreeBSD.org> References: <20130712112117.Horde.JMB3fhsU1VfZRxdV4prj5Q6@mail.vx.sk><51FE1E1E.8080502@FreeBSD.org><5C28F2D739D94E6B928ACD55D207665A@multiplay.co.uk><51FE9BF5.2000301@FreeBSD.org><51FEA530.2090008@FreeBSD.org><4B747F3B634B4A859068FC8D782A7E1D@multiplay.co.uk><52011DE6.8090708@FreeBSD.org><4DA6B66E61374BE68C895C8B9A93BCB7@multiplay.co.uk><520C9001.2040406@FreeBSD.org><78BAB3D2-86A8-404C-8D5A-5B4E89AF1208@FreeBSD.org><12BBEDC5C5AF4F8180D6A6B09AD695A6@multiplay.co.uk><57C8A903D51A44D7A8A38C19F494CF99@multiplay.co.uk> <3F20CA2BA01E4B07899B4CB82EB0E220@multiplay.co! .uk> <37A9F30CF2AB4445BFAE1402BA705B9B@multiplay.co.uk> To: Steven Hartland X-Mailer: Apple Mail (2.1508) X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.4.3 (aslan.scsiguy.com [70.89.174.89]); Mon, 19 Aug 2013 22:23:03 +0000 (UTC) Cc: Xin Li , Will Andrews , zfs-devel@freebsd.org, Matthew Ahrens X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Aug 2013 22:23:07 -0000 --Apple-Mail=_47935BC7-A11B-4756-81AE-A6FEF0A33EDB Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=iso-8859-1 On Aug 19, 2013, at 6:04 AM, Steven Hartland = wrote: > Ok so latest version attached with the following changes: > * Switched from d_nonrotating -> d_rotation_rate for more flexibility > in the future. > * Added g_handleattr_uint16_t to correctly handle uint16_t without > hard coding size. > * Switched back to for() {...} from do {..} while() Thanks for doing this. Can you also fix the comment above vdev_queue_length()? Perhaps you = meant "concerned" instead of "precious"? I also don't know if there is a policy about American vs. British = english. e.g. "favour", vs "favor". > * Added non_rotating_seek_inc option for controlling the seek = increment > for non rotating media. When you tested on an all SSD complement, did you see any impact from = this change? I would only expect it to do anything if the record size = for the fs is small or compression is enabled because of the 128k = aggregation limit in vdev_queue.c. Even in this case, only having a = penalty of 1 means that that we'll still break up a stream of contiguous = I/Os that could be aggregated. I think this can be revisited later = though once vdev_queue.c is improved and we have more experimental = results. This version still has the problem of potentially setting lastoffset on = the wrong device. It should be set on any leaf device the mirror code = reads. Right now it only sets it on mm_preferred and the DTL checks may = exclude that device before any I/O is issued. I did some dtracing today and found that vdev_queue_io() is always = called with vdev_mirror_io_start(). You mentioned having = vdev_queue_io() record the offset vs. explicitly setting it in = vdev_mirror.c degraded performance. Did you, by chance, have the = recording code after vdev_queue_io()'s "if (zio->io_flags & = ZIO_FLAG_DONT_QUEUE)" check? That's the only thing that comes to mind = to explain this since there should be no appreciable delay. Having = vdev_queue_io() do the recording would fix the lastoffset bugs in your = current rev. The "policy split" in both your version and the original has always = troubled my software "aesthetic sensibilities". State like = "vdev_readable()" gets checked twice, and more loops and code are = required. Load calculations are made for children that may be excluded = due to their DTL, etc. So I pushed all of the selection stuff into = vdev_mirror_child_select(). The result is more correct since the = preferred device is taken only from functional candidates. I think the = result is simpler too, but I will let the list pass judgement. The attached diff doesn't have your RPM clean up yet, but hopefully = still gives a sense of how this might work. -- Justin --Apple-Mail=_47935BC7-A11B-4756-81AE-A6FEF0A33EDB Content-Disposition: attachment; filename=zfs_mirror.diffs Content-Type: application/octet-stream; name="zfs_mirror.diffs" Content-Transfer-Encoding: 7bit --- //SpectraBSD/stable/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_mirror.c 2013-07-19 20:54:44.000000000 -0600 +++ /usr/home/justing/perforce/SpectraBSD/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_mirror.c 2013-07-19 20:54:44.000000000 -0600 @@ -44,27 +44,77 @@ vdev_t *mc_vd; uint64_t mc_offset; int mc_error; + int mc_load; uint8_t mc_tried; uint8_t mc_skipped; uint8_t mc_speculative; } mirror_child_t; typedef struct mirror_map { + int *mm_preferred; + int mm_preferred_cnt; int mm_children; int mm_resilvering; - int mm_preferred; int mm_root; - mirror_child_t mm_child[1]; + mirror_child_t mm_child[]; } mirror_map_t; -int vdev_mirror_shift = 21; +SYSCTL_DECL(_vfs_zfs_vdev); +static SYSCTL_NODE(_vfs_zfs_vdev, OID_AUTO, mirror, CTLFLAG_RD, 0, + "ZFS VDEV Mirror"); + +/* Rotating media load calculation configuration. */ +static int rotating_inc = 0; +TUNABLE_INT("vfs.zfs.vdev.mirror.rotating_inc", &rotating_inc); +SYSCTL_INT(_vfs_zfs_vdev_mirror, OID_AUTO, rotating_inc, CTLFLAG_RW, + &rotating_inc, 0, "Rotating media load increment for non-seeking IO's"); + +static int rotating_seek_inc = 5; +TUNABLE_INT("vfs.zfs.vdev.mirror.rotating_seek_inc", &rotating_seek_inc); +SYSCTL_INT(_vfs_zfs_vdev_mirror, OID_AUTO, rotating_seek_inc, CTLFLAG_RW, + &rotating_seek_inc, 0, "Rotating media load increment for seeking IO's"); + +static int rotating_seek_offset = 1 * 1024 * 1024; +TUNABLE_INT("vfs.zfs.vdev.mirror.rotating_seek_offset", &rotating_seek_offset); +SYSCTL_INT(_vfs_zfs_vdev_mirror, OID_AUTO, rotating_seek_offset, CTLFLAG_RW, + &rotating_seek_offset, 0, "Offset in bytes from the last IO which " + "triggers a reduced rotating media seek increment"); + +/* Non-rotating media load calculation configuration. */ +static int non_rotating_inc = 0; +TUNABLE_INT("vfs.zfs.vdev.mirror.non_rotating_inc", &non_rotating_inc); +SYSCTL_INT(_vfs_zfs_vdev_mirror, OID_AUTO, non_rotating_inc, CTLFLAG_RW, + &non_rotating_inc, 0, "Non-rotating media load increment"); + +static int non_rotating_seek_inc = 1; +TUNABLE_INT("vfs.zfs.vdev.mirror.non_rotating_seek_inc", + &non_rotating_seek_inc); +SYSCTL_INT(_vfs_zfs_vdev_mirror, OID_AUTO, non_rotating_seek_inc, CTLFLAG_RW, + &non_rotating_seek_inc, 0, + "Non-rotating media load increment for seeking IO's"); + +static int vdev_mirror_shift = 21; + +static inline size_t +vdev_mirror_map_size(int children) +{ + return (offsetof(mirror_map_t, mm_child[children]) + + sizeof(int) * children); +} + +static inline void +vdev_mirror_map_preferred_init(mirror_map_t *mm, int children) +{ + mm->mm_preferred = (int *)((uintptr_t)mm + + offsetof(mirror_map_t, mm_child[children])); +} static void vdev_mirror_map_free(zio_t *zio) { mirror_map_t *mm = zio->io_vsd; - kmem_free(mm, offsetof(mirror_map_t, mm_child[mm->mm_children])); + kmem_free(mm, vdev_mirror_map_size(mm->mm_children)); } static const zio_vsd_ops_t vdev_mirror_vsd_ops = { @@ -72,6 +122,50 @@ zio_vsd_default_cksum_report }; +static int +vdev_mirror_load(mirror_map_t *mm, vdev_t *vd, uint64_t zio_offset) +{ + uint64_t lastoffset; + int load; + + /* All DVAs have equal weight at the root. */ + if (mm->mm_root == B_TRUE) + return (INT_MAX); + + /* Standard load based on pending queue length. */ + load = vdev_queue_length(vd); + lastoffset = vdev_queue_lastoffset(vd); + + /* Non-rotating media. */ + if (!vd->vdev_rotating) { + if (lastoffset == zio_offset) + return (load + non_rotating_inc); + + /* + * Apply a seek penalty even for non-rotating devices. + * Sequential I/O can be aggregated into fewer operations + * on the device, thus avoiding unnecessary per-command + * overhead and boosting performance. + */ + return (load + non_rotating_seek_inc); + } + + /* IO's which directly follow the last IO. */ + if (lastoffset == zio_offset) + return (load + rotating_inc); + + /* + * Apply half the seek increment to IO's within seek offset + * of the last IO queued to this vdev as it should incure a + * smaller seek penalty or have the data in its track cache. + */ + if (ABS(lastoffset - zio_offset) < rotating_seek_offset) + return (load + (rotating_seek_inc / 2)); + + /* Apply the full seek increment to all other IO's. */ + return (load + rotating_seek_inc); +} + static mirror_map_t * vdev_mirror_map_alloc(zio_t *zio) { @@ -86,24 +180,12 @@ c = BP_GET_NDVAS(zio->io_bp); - mm = kmem_zalloc(offsetof(mirror_map_t, mm_child[c]), KM_SLEEP); + mm = kmem_zalloc(vdev_mirror_map_size(c), KM_SLEEP); + vdev_mirror_map_preferred_init(mm, c); mm->mm_children = c; mm->mm_resilvering = B_FALSE; - mm->mm_preferred = spa_get_random(c); mm->mm_root = B_TRUE; - /* - * Check the other, lower-index DVAs to see if they're on - * the same vdev as the child we picked. If they are, use - * them since they are likely to have been allocated from - * the primary metaslab in use at the time, and hence are - * more likely to have locality with single-copy data. - */ - for (c = mm->mm_preferred, d = c - 1; d >= 0; d--) { - if (DVA_GET_VDEV(&dva[d]) == DVA_GET_VDEV(&dva[c])) - mm->mm_preferred = d; - } - for (c = 0; c < mm->mm_children; c++) { mc = &mm->mm_child[c]; @@ -115,8 +197,11 @@ c = vd->vdev_children; - mm = kmem_zalloc(offsetof(mirror_map_t, mm_child[c]), KM_SLEEP); + mm = kmem_zalloc(vdev_mirror_map_size(c), KM_SLEEP); + vdev_mirror_map_preferred_init(mm, c); mm->mm_children = c; + mm->mm_root = B_FALSE; + /* * If we are resilvering, then we should handle scrub reads * differently; we shouldn't issue them to the resilvering @@ -156,10 +241,6 @@ mm->mm_resilvering = B_FALSE; } - mm->mm_preferred = mm->mm_resilvering ? 0 : - (zio->io_offset >> vdev_mirror_shift) % c; - mm->mm_root = B_FALSE; - for (c = 0; c < mm->mm_children; c++) { mc = &mm->mm_child[c]; mc->mc_vd = vd->vdev_child[c]; @@ -253,6 +334,48 @@ } /* + * Check the other, lower-index DVAs to see if they're on the same + * vdev as the child we picked. If they are, use them since they + * are likely to have been allocated from the primary metaslab in + * use at the time, and hence are more likely to have locality with + * single-copy data. + */ +static int +vdev_mirror_dva_select(zio_t *zio, int preferred) +{ + dva_t *dva = zio->io_bp->blk_dva; + mirror_map_t *mm = zio->io_vsd; + int c; + + for (c = preferred - 1; c >= 0; c--) { + if (DVA_GET_VDEV(&dva[c]) == DVA_GET_VDEV(&dva[preferred])) + preferred = c; + } + return (preferred); +} + +static int +vdev_mirror_preferred_child_randomize(zio_t *zio) +{ + mirror_map_t *mm = zio->io_vsd; + int p; + + if (mm->mm_root == B_TRUE) { + p = spa_get_random(mm->mm_preferred_cnt); + return (vdev_mirror_dva_select(zio, mm->mm_preferred[p])); + } + + /* + * To ensure we don't always favour the first matching vdev, + * which could lead to wear leveling issues on SSD's, we + * use the I/O offset as a sudo random seed into the vdevs + * which have the lowest load. + */ + p = (zio->io_offset >> vdev_mirror_shift) % mm->mm_preferred_cnt; + return (mm->mm_preferred[p]); +} + +/* * Try to find a child whose DTL doesn't contain the block we want to read. * If we can't, try the read on any vdev we haven't already tried. */ @@ -262,7 +385,7 @@ mirror_map_t *mm = zio->io_vsd; mirror_child_t *mc; uint64_t txg = zio->io_txg; - int i, c; + int c, lowest_load; ASSERT(zio->io_bp == NULL || BP_PHYSICAL_BIRTH(zio->io_bp) == txg); @@ -271,9 +394,9 @@ * If a child is known to be completely inaccessible (indicated by * vdev_readable() returning B_FALSE), don't even try. */ - for (i = 0, c = mm->mm_preferred; i < mm->mm_children; i++, c++) { - if (c >= mm->mm_children) - c = 0; + lowest_load = INT_MAX; + mm->mm_preferred_cnt = 0; + for (c = 0; c < mm->mm_children; c++) { mc = &mm->mm_child[c]; if (mc->mc_tried || mc->mc_skipped) continue; @@ -283,13 +406,31 @@ mc->mc_skipped = 1; continue; } - if (!vdev_dtl_contains(mc->mc_vd, DTL_MISSING, txg, 1)) - return (c); - mc->mc_error = SET_ERROR(ESTALE); - mc->mc_skipped = 1; - mc->mc_speculative = 1; + if (vdev_dtl_contains(mc->mc_vd, DTL_MISSING, txg, 1)) { + mc->mc_error = SET_ERROR(ESTALE); + mc->mc_skipped = 1; + mc->mc_speculative = 1; + continue; + } + + mc->mc_load = vdev_mirror_load(mm, mc->mc_vd, mc->mc_offset); + if (mc->mc_load > lowest_load) + continue; + + if (mc->mc_load < lowest_load) { + lowest_load = mc->mc_load; + mm->mm_preferred_cnt = 0; + } + mm->mm_preferred[mm->mm_preferred_cnt] = c; + mm->mm_preferred_cnt++; } + if (mm->mm_preferred_cnt == 1) + return (mm->mm_preferred[0]); + + if (mm->mm_preferred_cnt > 1) + return (vdev_mirror_preferred_child_randomize(zio)); + /* * Every device is either missing or has this txg in its DTL. * Look for any child we haven't already tried before giving up. --- //SpectraBSD/stable/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c 2013-07-19 20:54:44.000000000 -0600 +++ /usr/home/justing/perforce/SpectraBSD/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c 2013-07-19 20:54:44.000000000 -0600 @@ -155,6 +155,8 @@ avl_create(&vq->vq_pending_tree, vdev_queue_offset_compare, sizeof (zio_t), offsetof(struct zio, io_offset_node)); + + vq->vq_lastoffset = 0; } void @@ -383,6 +385,9 @@ ASSERT(zio->io_type == ZIO_TYPE_READ || zio->io_type == ZIO_TYPE_WRITE); + if (zio->io_type == ZIO_TYPE_READ) + vq->vq_lastoffset = zio->io_offset + zio->io_size; + if (zio->io_flags & ZIO_FLAG_DONT_QUEUE) return (zio); @@ -446,3 +451,21 @@ mutex_exit(&vq->vq_lock); } + +/* + * As these two methods are only used for load calculations we're not + * concerned if we get an incorrect value on 32bit platforms due to lack + * of vq_lock mutex uses here. Instead we prefer to keep it lock free for + * performance. + */ +int +vdev_queue_length(vdev_t *vd) +{ + return (avl_numnodes(&vd->vdev_queue.vq_pending_tree)); +} + +uint64_t +vdev_queue_lastoffset(vdev_t *vd) +{ + return (vd->vdev_queue.vq_lastoffset); +} --Apple-Mail=_47935BC7-A11B-4756-81AE-A6FEF0A33EDB-- From owner-zfs-devel@FreeBSD.ORG Tue Aug 20 13:54:12 2013 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 423AB891; Tue, 20 Aug 2013 13:54:12 +0000 (UTC) (envelope-from prvs=1944ab6227=killing@multiplay.co.uk) Received: from mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) (using TLSv1 with cipher RC4-MD5 (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 2F45620E1; Tue, 20 Aug 2013 13:54:10 +0000 (UTC) Received: from r2d2 ([82.69.141.170]) by mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) (MDaemon PRO v10.0.4) with ESMTP id md50005592855.msg; Tue, 20 Aug 2013 14:54:02 +0100 X-Spam-Processed: mail1.multiplay.co.uk, Tue, 20 Aug 2013 14:54:02 +0100 (not processed: message from valid local sender) X-MDDKIM-Result: neutral (mail1.multiplay.co.uk) X-MDRemoteIP: 82.69.141.170 X-Return-Path: prvs=1944ab6227=killing@multiplay.co.uk X-Envelope-From: killing@multiplay.co.uk Message-ID: <16CCE71F932643ABBC05F800E9956950@multiplay.co.uk> From: "Steven Hartland" To: "Justin T. Gibbs" References: <3F20CA2BA01E4B07899B4CB82EB0E220@multiplay.co! .uk> <37A9F30CF2AB4445BFAE1402BA705B9B@multiplay.co.uk> <1A5316AB-4E00-4C2B-B023-86ED6F1E6CC1@FreeBSD.org> Subject: Re: N-way mirror read speedup in zfsonlinux Date: Tue, 20 Aug 2013 14:54:28 +0100 MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="----=_NextPart_000_0088_01CE9DB5.2B47CBD0" X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.5931 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.6157 Cc: Xin Li , Will Andrews , zfs-devel@FreeBSD.org, Matthew Ahrens X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Aug 2013 13:54:12 -0000 This is a multi-part message in MIME format. ------=_NextPart_000_0088_01CE9DB5.2B47CBD0 Content-Type: text/plain; format=flowed; charset="iso-8859-1"; reply-type=original Content-Transfer-Encoding: 7bit ----- Original Message ----- From: "Justin T. Gibbs" On Aug 19, 2013, at 6:04 AM, Steven Hartland wrote: > > Ok so latest version attached with the following changes: > > * Switched from d_nonrotating -> d_rotation_rate for more flexibility > > in the future. > > * Added g_handleattr_uint16_t to correctly handle uint16_t without > > hard coding size. > > * Switched back to for() {...} from do {..} while() > > Thanks for doing this. > > Can you also fix the comment above vdev_queue_length()? Perhaps you meant > "concerned" instead of "precious"? > > I also don't know if there is a policy about American vs. British english. > e.g. "favour", vs "favor". Done and done ;-) > > * Added non_rotating_seek_inc option for controlling the seek increment > > for non rotating media. > > When you tested on an all SSD complement, did you see any impact from this change? > I would only expect it to do anything if the record size for the fs is small or > compression is enabled because of the 128k aggregation limit in vdev_queue.c. > Even in this case, only having a penalty of 1 means that that we'll still break > up a stream of contiguous I/Os that could be aggregated. I think this can be > revisited later though once vdev_queue.c is improved and we have more experimental > results. Not managed to get this test done yet, will need to wait till I'm back in the office now I'm afraid. > This version still has the problem of potentially setting lastoffset on the wrong > device. It should be set on any leaf device the mirror code reads. Right now it > only sets it on mm_preferred and the DTL checks may exclude that device before any > I/O is issued. In theory with the move of the vdev_queue_register_lastoffset calls to vdev_mirror_child_select, this should be mittergated some what if not totally. > I did some dtracing today and found that vdev_queue_io() is always called with > vdev_mirror_io_start(). > > You mentioned having vdev_queue_io() record the offset vs. explicitly setting it > in vdev_mirror.c degraded performance. Did you, by chance, have the recording > code after vdev_queue_io()'s "if (zio->io_flags & ZIO_FLAG_DONT_QUEUE)" check? > That's the only thing that comes to mind to explain this since there should be > no appreciable delay. Having vdev_queue_io() do the recording would fix the > lastoffset bugs in your current rev. Confirmed I just re-tested with :- zio_t * vdev_queue_io(zio_t *zio) { vdev_queue_t *vq = &zio->io_vd->vdev_queue; zio_t *nio; ASSERT(zio->io_type == ZIO_TYPE_READ || zio->io_type == ZIO_TYPE_WRITE); if (zio->io_type == ZIO_TYPE_READ) vq->vq_lastoffset = zio->io_offset + zio->io_size; if (zio->io_flags & ZIO_FLAG_DONT_QUEUE) return (zio); ... With this the HDD transfer rates dropped from ~40MB/s to ~12MB/s Also trued without zio->io_type == ZIO_TYPE_READ, as the offset should be updated not matter what the type of operation, still no change. > The "policy split" in both your version and the original has always troubled my > software "aesthetic sensibilities". State like "vdev_readable()" gets checked > twice, and more loops and code are required. Load calculations are made for > children that may be excluded due to their DTL, etc. So I pushed all of the > selection stuff into vdev_mirror_child_select(). The result is more correct > since the preferred device is taken only from functional candidates. Like it, I must say it bugged me too the vdev_mirror_map_alloc did part of vdev_mirror_child_select. > I think the result is simpler too, but I will let the list pass judgement. > > The attached diff doesn't have your RPM clean up yet, but hopefully still gives > a sense of how this might work. I've updated my version with your changes along with extracting our the mm structure to a simple inline used in both routes. It would be good to understand why we cant set the offset in vdev_queue_io as having call vdev_queue_register_lastoffset multiple times is a bit messy. FYI: I'm away for a week now so may not be able to read emails as much as usual so if there's a delay replying thats why. Regards Steve ================================================ This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmaster@multiplay.co.uk. ------=_NextPart_000_0088_01CE9DB5.2B47CBD0 Content-Type: application/octet-stream; name="zfs-mirror-load.patch" Content-Transfer-Encoding: quoted-printable Content-Disposition: attachment; filename="zfs-mirror-load.patch" Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/vdev.h=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/vdev.h (revision = 254419)=0A= +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/vdev.h (working = copy)=0A= @@ -119,6 +119,9 @@=0A= extern void vdev_queue_fini(vdev_t *vd);=0A= extern zio_t *vdev_queue_io(zio_t *zio);=0A= extern void vdev_queue_io_done(zio_t *zio);=0A= +extern int vdev_queue_length(vdev_t *vd);=0A= +extern uint64_t vdev_queue_lastoffset(vdev_t *vd);=0A= +extern void vdev_queue_register_lastoffset(vdev_t *vd, zio_t *zio);=0A= =0A= extern void vdev_config_dirty(vdev_t *vd);=0A= extern void vdev_config_clean(vdev_t *vd);=0A= Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/vdev_impl.h=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/vdev_impl.h = (revision 254419)=0A= +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/vdev_impl.h = (working copy)=0A= @@ -106,6 +106,7 @@=0A= avl_tree_t vq_pending_tree;=0A= hrtime_t vq_io_complete_ts;=0A= kmutex_t vq_lock;=0A= + uint64_t vq_lastoffset;=0A= };=0A= =0A= /*=0A= @@ -199,7 +200,10 @@=0A= spa_aux_vdev_t *vdev_aux; /* for l2cache vdevs */=0A= zio_t *vdev_probe_zio; /* root of current probe */=0A= vdev_aux_t vdev_label_aux; /* on-disk aux state */=0A= - struct trim_map *vdev_trimmap;=0A= + struct trim_map *vdev_trimmap; /* map on outstanding trims */ =0A= + uint16_t vdev_rotation_rate; /* rotational rate of the media */=0A= +#define VDEV_RATE_UNKNOWN 0=0A= +#define VDEV_RATE_NON_ROTATING 1=0A= =0A= /*=0A= * For DTrace to work in userland (libzpool) context, these fields must=0A= Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_geom.c=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_geom.c (revision = 254419)=0A= +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_geom.c (working = copy)=0A= @@ -42,9 +42,11 @@=0A= * Virtual device vector for GEOM.=0A= */=0A= =0A= +static g_attrchanged_t vdev_geom_attrchanged;=0A= struct g_class zfs_vdev_class =3D {=0A= .name =3D "ZFS::VDEV",=0A= .version =3D G_VERSION,=0A= + .attrchanged =3D vdev_geom_attrchanged,=0A= };=0A= =0A= DECLARE_GEOM_CLASS(zfs_vdev_class, zfs_vdev);=0A= @@ -62,6 +64,34 @@=0A= &vdev_geom_bio_delete_disable, 0, "Disable BIO_DELETE");=0A= =0A= static void=0A= +vdev_geom_set_rotation_rate(vdev_t *vd, struct g_consumer *cp)=0A= +{ =0A= + int error;=0A= + uint16_t rate;=0A= +=0A= + error =3D g_getattr("GEOM::rotation_rate", cp, &rate);=0A= + if (error =3D=3D 0)=0A= + vd->vdev_rotation_rate =3D rate;=0A= + else=0A= + vd->vdev_rotation_rate =3D VDEV_RATE_UNKNOWN;=0A= +}=0A= +=0A= +static void=0A= +vdev_geom_attrchanged(struct g_consumer *cp, const char *attr)=0A= +{=0A= + vdev_t *vd;=0A= +=0A= + vd =3D cp->private;=0A= + if (vd =3D=3D NULL)=0A= + return;=0A= +=0A= + if (strcmp(attr, "GEOM::rotation_rate") =3D=3D 0) {=0A= + vdev_geom_set_rotation_rate(vd, cp);=0A= + return;=0A= + }=0A= +}=0A= +=0A= +static void=0A= vdev_geom_orphan(struct g_consumer *cp)=0A= {=0A= vdev_t *vd;=0A= @@ -678,6 +708,11 @@=0A= vd->vdev_physpath =3D kmem_alloc(bufsize, KM_SLEEP);=0A= snprintf(vd->vdev_physpath, bufsize, "/dev/%s", pp->name);=0A= =0A= + /*=0A= + * Determine the device's rotation rate.=0A= + */=0A= + vdev_geom_set_rotation_rate(vd, cp);=0A= +=0A= return (0);=0A= }=0A= =0A= Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_mirror.c=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_mirror.c = (revision 254419)=0A= +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_mirror.c = (working copy)=0A= @@ -41,6 +41,7 @@=0A= vdev_t *mc_vd;=0A= uint64_t mc_offset;=0A= int mc_error;=0A= + int mc_load;=0A= uint8_t mc_tried;=0A= uint8_t mc_skipped;=0A= uint8_t mc_speculative;=0A= @@ -54,8 +55,53 @@=0A= mirror_child_t mm_child[1];=0A= } mirror_map_t;=0A= =0A= -int vdev_mirror_shift =3D 21;=0A= +SYSCTL_DECL(_vfs_zfs_vdev);=0A= +static SYSCTL_NODE(_vfs_zfs_vdev, OID_AUTO, mirror, CTLFLAG_RD, 0,=0A= + "ZFS VDEV Mirror");=0A= =0A= +/*=0A= + * The load configuration settings below are tuned by default for=0A= + * the case where all devices are of the same rotational type.=0A= + *=0A= + * If there is a mixture of rotating and non-rotating media, setting=0A= + * non_rotating_seek_inc to 0 may well provide better results as it=0A= + * will direct more reads to the non-rotating vdevs which are more=0A= + * likely to have a higher performance.=0A= + */=0A= +=0A= +/* Rotating media load calculation configuration. */=0A= +static int rotating_inc =3D 0;=0A= +TUNABLE_INT("vfs.zfs.vdev.mirror.rotating_inc", &rotating_inc);=0A= +SYSCTL_INT(_vfs_zfs_vdev_mirror, OID_AUTO, rotating_inc, CTLFLAG_RW,=0A= + &rotating_inc, 0, "Rotating media load increment for non-seeking = I/O's");=0A= +=0A= +static int rotating_seek_inc =3D 5;=0A= +TUNABLE_INT("vfs.zfs.vdev.mirror.rotating_seek_inc", = &rotating_seek_inc);=0A= +SYSCTL_INT(_vfs_zfs_vdev_mirror, OID_AUTO, rotating_seek_inc, = CTLFLAG_RW,=0A= + &rotating_seek_inc, 0, "Rotating media load increment for seeking = I/O's");=0A= +=0A= +static int rotating_seek_offset =3D 1 * 1024 * 1024;=0A= +TUNABLE_INT("vfs.zfs.vdev.mirror.rotating_seek_offset", = &rotating_seek_offset);=0A= +SYSCTL_INT(_vfs_zfs_vdev_mirror, OID_AUTO, rotating_seek_offset, = CTLFLAG_RW,=0A= + &rotating_seek_offset, 0, "Offset in bytes from the last I/O which "=0A= + "triggers a reduced rotating media seek increment");=0A= +=0A= +/* Non-rotating media load calculation configuration. */=0A= +static int non_rotating_inc =3D 0;=0A= +TUNABLE_INT("vfs.zfs.vdev.mirror.non_rotating_inc", &non_rotating_inc);=0A= +SYSCTL_INT(_vfs_zfs_vdev_mirror, OID_AUTO, non_rotating_inc, CTLFLAG_RW,=0A= + &non_rotating_inc, 0,=0A= + "Non-rotating media load increment for non-seeking I/O's");=0A= +=0A= +static int non_rotating_seek_inc =3D 1;=0A= +TUNABLE_INT("vfs.zfs.vdev.mirror.non_rotating_seek_inc",=0A= + &non_rotating_seek_inc);=0A= +SYSCTL_INT(_vfs_zfs_vdev_mirror, OID_AUTO, non_rotating_seek_inc, = CTLFLAG_RW,=0A= + &non_rotating_seek_inc, 0,=0A= + "Non-rotating media load increment for seeking I/O's");=0A= +=0A= +static int vdev_mirror_shift =3D 21;=0A= +=0A= static void=0A= vdev_mirror_map_free(zio_t *zio)=0A= {=0A= @@ -69,6 +115,56 @@=0A= zio_vsd_default_cksum_report=0A= };=0A= =0A= +static int=0A= +vdev_mirror_load(vdev_t *vd, uint64_t zio_offset)=0A= +{=0A= + uint64_t lastoffset;=0A= + int load;=0A= +=0A= + /*=0A= + * We don't return INT_MAX if the device is resilvering i.e.=0A= + * vdev_resilver_txg !=3D 0 as when tested performance was slightly=0A= + * worse overall when resilvering with compared to without.=0A= + */=0A= +=0A= + /* If the vdev isn't readable use the maximum load. */=0A= + if (!vdev_readable(vd))=0A= + return (INT_MAX);=0A= +=0A= + /* Standard load based on pending queue length. */=0A= + load =3D vdev_queue_length(vd);=0A= + lastoffset =3D vdev_queue_lastoffset(vd);=0A= +=0A= + /* Non-rotating media. */=0A= + if (vd->vdev_rotation_rate =3D=3D VDEV_RATE_NON_ROTATING) {=0A= + if (lastoffset =3D=3D zio_offset)=0A= + return (load + non_rotating_inc);=0A= +=0A= + /*=0A= + * Apply a seek penalty even for non-rotating devices as=0A= + * sequential I/O'a can be aggregated into fewer operations=0A= + * on the device, thus avoiding unnecessary per-command=0A= + * overhead and boosting performance.=0A= + */=0A= + return (load + non_rotating_seek_inc);=0A= + }=0A= +=0A= + /* I/O's which directly follow the last I/O. */=0A= + if (lastoffset =3D=3D zio_offset)=0A= + return (load + rotating_inc);=0A= +=0A= + /*=0A= + * Apply half the seek increment to I/O's within seek offset=0A= + * of the last I/O queued to this vdev as they should incure less=0A= + * of a seek increment.=0A= + */=0A= + if (ABS(lastoffset - zio_offset) < rotating_seek_offset)=0A= + return (load + (rotating_seek_inc / 2));=0A= +=0A= + /* Apply the full seek increment to all other I/O's. */=0A= + return (load + rotating_seek_inc);=0A= +}=0A= +=0A= static mirror_map_t *=0A= vdev_mirror_map_alloc(zio_t *zio)=0A= {=0A= @@ -108,21 +204,62 @@=0A= mc->mc_offset =3D DVA_GET_OFFSET(&dva[c]);=0A= }=0A= } else {=0A= + int lowest_load, lowest_child, lowest_nr;=0A= + uint64_t zio_offset;=0A= +=0A= c =3D vd->vdev_children;=0A= =0A= mm =3D kmem_zalloc(offsetof(mirror_map_t, mm_child[c]), KM_SLEEP);=0A= mm->mm_children =3D c;=0A= mm->mm_replacing =3D (vd->vdev_ops =3D=3D &vdev_replacing_ops ||=0A= vd->vdev_ops =3D=3D &vdev_spare_ops);=0A= - mm->mm_preferred =3D mm->mm_replacing ? 0 :=0A= - (zio->io_offset >> vdev_mirror_shift) % c;=0A= mm->mm_root =3D B_FALSE;=0A= =0A= + zio_offset =3D zio->io_offset;=0A= + lowest_load =3D INT_MAX;=0A= + lowest_child =3D 0;=0A= + lowest_nr =3D 1;=0A= +=0A= for (c =3D 0; c < mm->mm_children; c++) {=0A= mc =3D &mm->mm_child[c];=0A= mc->mc_vd =3D vd->vdev_child[c];=0A= - mc->mc_offset =3D zio->io_offset;=0A= + mc->mc_offset =3D zio_offset;=0A= +=0A= + if (zio->io_type !=3D ZIO_TYPE_READ)=0A= + continue;=0A= +=0A= + mc->mc_load =3D vdev_mirror_load(mc->mc_vd, zio_offset);=0A= + if (mc->mc_load < lowest_load) {=0A= + lowest_load =3D mc->mc_load;=0A= + lowest_child =3D c; =0A= + lowest_nr =3D 1;=0A= + } else if (mc->mc_load =3D=3D lowest_load) {=0A= + lowest_nr++;=0A= + }=0A= }=0A= +=0A= + if (lowest_nr !=3D 1) {=0A= + int d;=0A= +=0A= + /*=0A= + * To ensure we don't always favour the first=0A= + * matching vdev, which could lead to wear=0A= + * leveling issues on SSD's, we use the I/O=0A= + * offset as a sudo random seed into the vdevs=0A= + * which have the lowest load.=0A= + */=0A= + d =3D (zio_offset >> vdev_mirror_shift)=0A= + % lowest_nr;=0A= + for (c =3D lowest_child + 1; d !=3D 0; c++) {=0A= + if (mm->mm_child[c].mc_load =3D=3D lowest_load) {=0A= + lowest_child =3D c;=0A= + d--;=0A= + }=0A= + }=0A= + }=0A= + mm->mm_preferred =3D lowest_child;=0A= + vdev_queue_register_lastoffset(vd->vdev_child[lowest_child],=0A= + zio);=0A= }=0A= =0A= zio->io_vsd =3D mm;=0A= Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c = (revision 254419)=0A= +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c (working = copy)=0A= @@ -155,6 +155,8 @@=0A= =0A= avl_create(&vq->vq_pending_tree, vdev_queue_offset_compare,=0A= sizeof (zio_t), offsetof(struct zio, io_offset_node));=0A= +=0A= + vq->vq_lastoffset =3D 0;=0A= }=0A= =0A= void=0A= @@ -446,3 +448,26 @@=0A= =0A= mutex_exit(&vq->vq_lock);=0A= }=0A= +=0A= +/*=0A= + * As these three methods are only used for load calculations we're not = precious=0A= + * if we get an incorrect value on 32bit platforms due to lack of = vq_lock mutex=0A= + * uses here. Instead we prefer to keep it lock free for the = performance.=0A= + */ =0A= +int=0A= +vdev_queue_length(vdev_t *vd)=0A= +{=0A= + return (avl_numnodes(&vd->vdev_queue.vq_pending_tree));=0A= +}=0A= +=0A= +uint64_t=0A= +vdev_queue_lastoffset(vdev_t *vd)=0A= +{=0A= + return (vd->vdev_queue.vq_lastoffset);=0A= +}=0A= +=0A= +void=0A= +vdev_queue_register_lastoffset(vdev_t *vd, zio_t *zio)=0A= +{=0A= + vd->vdev_queue.vq_lastoffset =3D zio->io_offset + zio->io_size;=0A= +}=0A= Index: sys/geom/geom.h=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/geom/geom.h (revision 254419)=0A= +++ sys/geom/geom.h (working copy)=0A= @@ -270,6 +270,7 @@=0A= int len);=0A= int g_handleattr_int(struct bio *bp, const char *attribute, int val);=0A= int g_handleattr_off_t(struct bio *bp, const char *attribute, off_t = val);=0A= +int g_handleattr_uint16_t(struct bio *bp, const char *attribute, = uint16_t val);=0A= int g_handleattr_str(struct bio *bp, const char *attribute, const char = *str);=0A= struct g_consumer * g_new_consumer(struct g_geom *gp);=0A= struct g_geom * g_new_geomf(struct g_class *mp, const char *fmt, ...)=0A= Index: sys/geom/geom_disk.c=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/geom/geom_disk.c (revision 254419)=0A= +++ sys/geom/geom_disk.c (working copy)=0A= @@ -376,22 +376,25 @@=0A= break;=0A= else if (g_handleattr_str(bp, "GEOM::ident", dp->d_ident))=0A= break;=0A= - else if (g_handleattr(bp, "GEOM::hba_vendor",=0A= - &dp->d_hba_vendor, 2))=0A= + else if (g_handleattr_uint16_t(bp, "GEOM::hba_vendor",=0A= + dp->d_hba_vendor))=0A= break;=0A= - else if (g_handleattr(bp, "GEOM::hba_device",=0A= - &dp->d_hba_device, 2))=0A= + else if (g_handleattr_uint16_t(bp, "GEOM::hba_device",=0A= + dp->d_hba_device))=0A= break;=0A= - else if (g_handleattr(bp, "GEOM::hba_subvendor",=0A= - &dp->d_hba_subvendor, 2))=0A= + else if (g_handleattr_uint16_t(bp, "GEOM::hba_subvendor",=0A= + dp->d_hba_subvendor))=0A= break;=0A= - else if (g_handleattr(bp, "GEOM::hba_subdevice",=0A= - &dp->d_hba_subdevice, 2))=0A= + else if (g_handleattr_uint16_t(bp, "GEOM::hba_subdevice",=0A= + dp->d_hba_subdevice))=0A= break;=0A= else if (!strcmp(bp->bio_attribute, "GEOM::kerneldump"))=0A= g_disk_kerneldump(bp, dp);=0A= else if (!strcmp(bp->bio_attribute, "GEOM::setstate"))=0A= g_disk_setstate(bp, sc);=0A= + else if (g_handleattr_uint16_t(bp, "GEOM::rotation_rate",=0A= + dp->d_rotation_rate))=0A= + break;=0A= else =0A= error =3D ENOIOCTL;=0A= break;=0A= Index: sys/geom/geom_disk.h=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/geom/geom_disk.h (revision 254419)=0A= +++ sys/geom/geom_disk.h (working copy)=0A= @@ -97,6 +97,7 @@=0A= uint16_t d_hba_device;=0A= uint16_t d_hba_subvendor;=0A= uint16_t d_hba_subdevice;=0A= + uint16_t d_rotation_rate;=0A= =0A= /* Fields private to the driver */=0A= void *d_drv1;=0A= @@ -121,7 +122,8 @@=0A= #define DISK_VERSION_01 0x5856105a=0A= #define DISK_VERSION_02 0x5856105b=0A= #define DISK_VERSION_03 0x5856105c=0A= -#define DISK_VERSION DISK_VERSION_03=0A= +#define DISK_VERSION_04 0x5856105d=0A= +#define DISK_VERSION DISK_VERSION_04=0A= =0A= #endif /* _KERNEL */=0A= #endif /* _GEOM_GEOM_DISK_H_ */=0A= Index: sys/geom/geom_subr.c=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/geom/geom_subr.c (revision 254419)=0A= +++ sys/geom/geom_subr.c (working copy)=0A= @@ -949,6 +949,13 @@=0A= }=0A= =0A= int=0A= +g_handleattr_uint16_t(struct bio *bp, const char *attribute, uint16_t = val)=0A= +{=0A= +=0A= + return (g_handleattr(bp, attribute, &val, sizeof val));=0A= +}=0A= +=0A= +int=0A= g_handleattr_off_t(struct bio *bp, const char *attribute, off_t val)=0A= {=0A= =0A= Index: sys/cam/ata/ata_da.c=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/cam/ata/ata_da.c (revision 254419)=0A= +++ sys/cam/ata/ata_da.c (working copy)=0A= @@ -1224,13 +1224,14 @@=0A= snprintf(announce_buf, sizeof(announce_buf),=0A= "kern.cam.ada.%d.write_cache", periph->unit_number);=0A= TUNABLE_INT_FETCH(announce_buf, &softc->write_cache);=0A= + adagetparams(periph, cgd);=0A= + softc->disk =3D disk_alloc();=0A= /* Disable queue sorting for non-rotational media by default. */=0A= - if (cgd->ident_data.media_rotation_rate =3D=3D 1)=0A= + if (cgd->ident_data.media_rotation_rate =3D=3D ATA_RATE_NON_ROTATING)=0A= softc->sort_io_queue =3D 0;=0A= else=0A= softc->sort_io_queue =3D -1;=0A= - adagetparams(periph, cgd);=0A= - softc->disk =3D disk_alloc();=0A= + softc->disk->d_rotation_rate =3D cgd->ident_data.media_rotation_rate;=0A= softc->disk->d_devstat =3D devstat_new_entry(periph->periph_name,=0A= periph->unit_number, softc->params.secsize,=0A= DEVSTAT_ALL_SUPPORTED,=0A= Index: sys/cam/scsi/scsi_all.h=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/cam/scsi/scsi_all.h (revision 254419)=0A= +++ sys/cam/scsi/scsi_all.h (working copy)=0A= @@ -1451,7 +1451,7 @@=0A= u_int8_t page_length[2];=0A= u_int8_t medium_rotation_rate[2];=0A= #define SVPD_BDC_RATE_NOT_REPORTED 0x00=0A= -#define SVPD_BDC_RATE_NONE_ROTATING 0x01=0A= +#define SVPD_BDC_RATE_NON_ROTATING 0x01=0A= u_int8_t reserved1;=0A= u_int8_t nominal_form_factor;=0A= #define SVPD_BDC_FORM_NOT_REPORTED 0x00=0A= Index: sys/cam/scsi/scsi_da.c=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/cam/scsi/scsi_da.c (revision 254419)=0A= +++ sys/cam/scsi/scsi_da.c (working copy)=0A= @@ -3386,9 +3386,17 @@=0A= * Disable queue sorting for non-rotational media=0A= * by default.=0A= */=0A= - if (scsi_2btoul(bdc->medium_rotation_rate) =3D=3D=0A= - SVPD_BDC_RATE_NONE_ROTATING)=0A= + u_int old_rate =3D softc->disk->d_rotation_rate;=0A= + softc->disk->d_rotation_rate =3D=0A= + scsi_2btoul(bdc->medium_rotation_rate);=0A= + if (softc->disk->d_rotation_rate =3D=3D=0A= + SVPD_BDC_RATE_NON_ROTATING) {=0A= softc->sort_io_queue =3D 0;=0A= + }=0A= + if (softc->disk->d_rotation_rate !=3D old_rate) {=0A= + disk_attr_changed(softc->disk,=0A= + "GEOM::rotation_rate", M_NOWAIT);=0A= + }=0A= } else {=0A= int error;=0A= error =3D daerror(done_ccb, CAM_RETRY_SELTO,=0A= @@ -3423,6 +3431,8 @@=0A= ptr =3D (uint16_t *)ata_params;=0A= =0A= if ((csio->ccb_h.status & CAM_STATUS_MASK) =3D=3D CAM_REQ_CMP) {=0A= + uint16_t old_rate;=0A= +=0A= for (i =3D 0; i < sizeof(*ata_params) / 2; i++)=0A= ptr[i] =3D le16toh(ptr[i]);=0A= if (ata_params->support_dsm & ATA_SUPPORT_DSM_TRIM) {=0A= @@ -3437,8 +3447,18 @@=0A= * Disable queue sorting for non-rotational media=0A= * by default.=0A= */=0A= - if (ata_params->media_rotation_rate =3D=3D 1)=0A= + old_rate =3D softc->disk->d_rotation_rate;=0A= + softc->disk->d_rotation_rate =3D=0A= + ata_params->media_rotation_rate;=0A= + if (softc->disk->d_rotation_rate =3D=3D=0A= + ATA_RATE_NON_ROTATING) {=0A= softc->sort_io_queue =3D 0;=0A= + }=0A= +=0A= + if (softc->disk->d_rotation_rate !=3D old_rate) {=0A= + disk_attr_changed(softc->disk,=0A= + "GEOM::rotation_rate", M_NOWAIT);=0A= + }=0A= } else {=0A= int error;=0A= error =3D daerror(done_ccb, CAM_RETRY_SELTO,=0A= ------=_NextPart_000_0088_01CE9DB5.2B47CBD0-- From owner-zfs-devel@FreeBSD.ORG Tue Aug 20 22:09:43 2013 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 4F0A14DF for ; Tue, 20 Aug 2013 22:09:43 +0000 (UTC) (envelope-from gibbs@FreeBSD.org) Received: from aslan.scsiguy.com (mail.scsiguy.com [70.89.174.89]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 00E74207A for ; Tue, 20 Aug 2013 22:09:42 +0000 (UTC) Received: from [192.168.6.147] (207-225-98-3.dia.static.qwest.net [207.225.98.3]) (authenticated bits=0) by aslan.scsiguy.com (8.14.7/8.14.5) with ESMTP id r7KM9Yvt006928 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NO); Tue, 20 Aug 2013 22:09:34 GMT (envelope-from gibbs@FreeBSD.org) Content-Type: multipart/signed; boundary="Apple-Mail=_0358433D-DC97-44B4-897D-514631ED63BE"; protocol="application/pgp-signature"; micalg=pgp-sha1 Mime-Version: 1.0 (Mac OS X Mail 6.5 \(1508\)) Subject: Re: Updated ashift optimization patch From: "Justin T. Gibbs" In-Reply-To: <52128494.2030206@gentoo.org> Date: Tue, 20 Aug 2013 16:09:19 -0600 Message-Id: References: <984A4829-FBB0-4174-83FB-06A7336F4737@FreeBSD.org> <8375F87B-F406-491F-BE75-0FAECED680E6@FreeBSD.org> <52128494.2030206@gentoo.org> To: Richard Yao X-Mailer: Apple Mail (2.1508) X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.4.3 (aslan.scsiguy.com [70.89.174.89]); Tue, 20 Aug 2013 22:09:35 +0000 (UTC) Cc: zfs-devel@freebsd.org, zfs@lists.illumos.org X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Aug 2013 22:09:43 -0000 --Apple-Mail=_0358433D-DC97-44B4-897D-514631ED63BE Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=iso-8859-1 On Aug 19, 2013, at 2:48 PM, Richard Yao wrote: > On 08/18/2013 10:26 PM, Justin T. Gibbs wrote:>> One thing we did = here, > which would be good to see in this patch, is to add >>> an overall min system ashift as thie enables admins to configure = pools >>> to be compatible with future disks they are likely to use e.g. min = ashift >>> 12 (4k compatible). This could be left at 9 by default for max > compatibility >>> but personally I'd suggest 12. >>=20 >> it would be nice to hear more consensus come out of the recent zpool > ashift discussions before doing something here. Whatever that is, I > agree that the default should be 9 until someone fixes the RAIDZ space > penalty for going to 4k on 512N drives. >>=20 >> -- >> Justin >>=20 >=20 > The default ashift is determined dynamically per top-level vdev by the > highest block_size reported by any member at creation time. There is = no > fixed default ashift. However, many drives lie, so it could appear = that way. The change Steven is proposing would set a system wide *minimum* ashift = for new pools. This would only come into play on devices reporting an = ashift smaller than the set minimum. -- Justin --Apple-Mail=_0358433D-DC97-44B4-897D-514631ED63BE Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Comment: GPGTools - http://gpgtools.org iQEcBAEBAgAGBQJSE+kYAAoJED9n8CuvaSf47BkH/2+Whh4ga35f1Fj2DnsftZ6/ RmNtv1M8HPE1snPCn/CC+cuVZYd0+Y2t6ZQ/fg3z2IirTdEfLuBz/wxmNDz/YPoi KHH5OXpcldb2yfzKh/9sSSIc6PPqtZUmwVDGUrqYVxloDMIfF2cJnAdLOzYNCdFU ZLhSXxRHkukNne6QRpC8h8k4/7+AtE9RMLxvNNOOHQ1TOmrxf9j2ikeNvEBBpx2P 3reS1Vqd4+IhF1icWe0kDITRR+VTyJ+jG9laBvyhL4pUKbWDkekszbOliDke7Oh1 WKjZVF8kGuwsRKhxyrIe9pR8F26wo6CM7FTnD+g8QGb5oN0yud5LVtV1/D1FoTY= =b0t/ -----END PGP SIGNATURE----- --Apple-Mail=_0358433D-DC97-44B4-897D-514631ED63BE-- From owner-zfs-devel@FreeBSD.ORG Tue Aug 20 22:10:50 2013 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 5C086501 for ; Tue, 20 Aug 2013 22:10:50 +0000 (UTC) (envelope-from gibbs@FreeBSD.org) Received: from aslan.scsiguy.com (mail.scsiguy.com [70.89.174.89]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 14F1220A8 for ; Tue, 20 Aug 2013 22:10:49 +0000 (UTC) Received: from [192.168.6.147] (207-225-98-3.dia.static.qwest.net [207.225.98.3]) (authenticated bits=0) by aslan.scsiguy.com (8.14.7/8.14.5) with ESMTP id r7KMAmp3006939 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NO); Tue, 20 Aug 2013 22:10:49 GMT (envelope-from gibbs@FreeBSD.org) Subject: Re: Updated ashift optimization patch Mime-Version: 1.0 (Mac OS X Mail 6.5 \(1508\)) Content-Type: text/plain; charset=us-ascii From: "Justin T. Gibbs" X-Priority: 3 In-Reply-To: <8375F87B-F406-491F-BE75-0FAECED680E6@FreeBSD.org> Date: Tue, 20 Aug 2013 16:10:43 -0600 Content-Transfer-Encoding: quoted-printable Message-Id: <88F3780B-82D7-4941-82D5-2362845F7533@FreeBSD.org> References: <984A4829-FBB0-4174-83FB-06A7336F4737@FreeBSD.org> <8375F87B-F406-491F-BE75-0FAECED680E6@FreeBSD.org> To: "zfs-devel@freebsd.org" X-Mailer: Apple Mail (2.1508) X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.4.3 (aslan.scsiguy.com [70.89.174.89]); Tue, 20 Aug 2013 22:10:49 +0000 (UTC) Cc: zfs@lists.illumos.org, Richard Yao X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Aug 2013 22:10:50 -0000 Unless I hear otherwise, I plan to commit this patch (with the EINVAL = change described below) as soon as my testing on FreeBSD/head completes. -- Justin On Aug 18, 2013, at 8:26 PM, Justin T. Gibbs wrote: > On Aug 18, 2013, at 6:23 PM, "Steven Hartland" = wrote: >=20 >> Few things I've noticed:- >>=20 >> 1. in vdev_geom.c is there a reason you use: >> + *physical_ashift =3D 0; >> + if (pp->stripesize) >> + *physical_ashift =3D highbit(pp->stripesize) - 1; >>=20 >> Instead of: >> + *physical_ashift =3D highbit(MAX(pp->stripesize, SPA_MINBLOCKSIZE)) = - 1; >=20 > 0 in ashift as in stripesize means "unset" or "don't care". This = gives more information during debugging ("Oh. The driver for this = device must not be setting this.") than if it were clamped. If we = decide it should always be set, I think "MAX(pp->sectorsize, = pp->stripesize)" would make more sense. >=20 >> 2. sysctl_vfs_zfs_max_auto_ashift should also limit the min to > >> SPA_MINBLOCKSHIFT >=20 > Setting it too low (any value less than the device's logical ashift) = just disables the feature. I don't see any value in adding code to = error out or clamp considering the most likely reason for this to occur = is an administrator trying to turn it off (e.g. set it to 0). >=20 >> 3. sysctl_vfs_zfs_max_auto_ashift should error if the value is out >> of range instead of clamping it. >=20 > Ok. I now return EINVAL if the value exceeds SPA_MAXASHIFT. >=20 >> One thing we did here, which would be good to see in this patch, is = to add >> an overall min system ashift as thie enables admins to configure = pools >> to be compatible with future disks they are likely to use e.g. min = ashift >> 12 (4k compatible). This could be left at 9 by default for max = compatibility >> but personally I'd suggest 12. >=20 > it would be nice to hear more consensus come out of the recent zpool = ashift discussions before doing something here. Whatever that is, I = agree that the default should be 9 until someone fixes the RAIDZ space = penalty for going to 4k on 512N drives. >=20 > -- > Justin > _______________________________________________ > zfs-devel@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/zfs-devel > To unsubscribe, send any mail to "zfs-devel-unsubscribe@freebsd.org" >=20 From owner-zfs-devel@FreeBSD.ORG Wed Aug 21 16:52:54 2013 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id AA8FD688 for ; Wed, 21 Aug 2013 16:52:54 +0000 (UTC) (envelope-from surya1@llnl.gov) Received: from prdiron-2.llnl.gov (prdiron-2.llnl.gov [128.15.143.172]) by mx1.freebsd.org (Postfix) with ESMTP id 84EEB297E for ; Wed, 21 Aug 2013 16:52:54 +0000 (UTC) X-Attachments: Received: from eris.llnl.gov ([128.115.7.7]) by prdiron-2.llnl.gov with ESMTP; 21 Aug 2013 09:51:46 -0700 Received: from ubuntu (spunky.chaos [192.168.1.143]) by eris.llnl.gov (Postfix) with ESMTP id 92C5F7C4A5; Wed, 21 Aug 2013 09:51:45 -0700 (PDT) Received: by ubuntu (Postfix, from userid 37859) id 6DAFC1E566; Wed, 21 Aug 2013 09:51:45 -0700 (PDT) Date: Wed, 21 Aug 2013 09:51:45 -0700 From: Prakash Surya To: zfs@lists.illumos.org Subject: Re: [zfs] Re: Updated ashift optimization patch Message-ID: <20130821165145.GB60411@llnl.gov> References: <984A4829-FBB0-4174-83FB-06A7336F4737@FreeBSD.org> <8375F87B-F406-491F-BE75-0FAECED680E6@FreeBSD.org> <88F3780B-82D7-4941-82D5-2362845F7533@FreeBSD.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <88F3780B-82D7-4941-82D5-2362845F7533@FreeBSD.org> User-Agent: Mutt/1.5.20 (2009-06-14) Cc: "zfs-devel@freebsd.org" , Richard Yao X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 Aug 2013 16:52:54 -0000 I'm looking at porting the attached patch to ZFS, and I have a question. The patch modifies the typedef of vdev_open_func_t: typedef int>---vdev_open_func_t(vdev_t *vd, uint64_t *size, uint64_t *max_size, - uint64_t *ashift); + uint64_t *logical_ashift, uint64_t *physical_ashift); But it doesn't modify the prototype for vdev_disk_open like it does for the other vdev_*_open calls (i.e. vdev_file_open, vdev_mirror_open, etc). Is that oversight, or am I misunderstanding something? I see the vdev_disk_ops structure in the illumos-gate source code, so the structure isn't Linux specific.. -- Cheers, Prakash On Tue, Aug 20, 2013 at 04:10:43PM -0600, Justin T. Gibbs wrote: > Unless I hear otherwise, I plan to commit this patch (with the EINVAL change described below) as soon as my testing on FreeBSD/head completes. > > -- > Justin > > On Aug 18, 2013, at 8:26 PM, Justin T. Gibbs wrote: > > > On Aug 18, 2013, at 6:23 PM, "Steven Hartland" wrote: > > > >> Few things I've noticed:- > >> > >> 1. in vdev_geom.c is there a reason you use: > >> + *physical_ashift = 0; > >> + if (pp->stripesize) > >> + *physical_ashift = highbit(pp->stripesize) - 1; > >> > >> Instead of: > >> + *physical_ashift = highbit(MAX(pp->stripesize, SPA_MINBLOCKSIZE)) - 1; > > > > 0 in ashift as in stripesize means "unset" or "don't care". This gives more information during debugging ("Oh. The driver for this device must not be setting this.") than if it were clamped. If we decide it should always be set, I think "MAX(pp->sectorsize, pp->stripesize)" would make more sense. > > > >> 2. sysctl_vfs_zfs_max_auto_ashift should also limit the min to > > >> SPA_MINBLOCKSHIFT > > > > Setting it too low (any value less than the device's logical ashift) just disables the feature. I don't see any value in adding code to error out or clamp considering the most likely reason for this to occur is an administrator trying to turn it off (e.g. set it to 0). > > > >> 3. sysctl_vfs_zfs_max_auto_ashift should error if the value is out > >> of range instead of clamping it. > > > > Ok. I now return EINVAL if the value exceeds SPA_MAXASHIFT. > > > >> One thing we did here, which would be good to see in this patch, is to add > >> an overall min system ashift as thie enables admins to configure pools > >> to be compatible with future disks they are likely to use e.g. min ashift > >> 12 (4k compatible). This could be left at 9 by default for max compatibility > >> but personally I'd suggest 12. > > > > it would be nice to hear more consensus come out of the recent zpool ashift discussions before doing something here. Whatever that is, I agree that the default should be 9 until someone fixes the RAIDZ space penalty for going to 4k on 512N drives. > > > > -- > > Justin > > _______________________________________________ > > zfs-devel@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/zfs-devel > > To unsubscribe, send any mail to "zfs-devel-unsubscribe@freebsd.org" > > > > > > ------------------------------------------- > illumos-zfs > Archives: https://www.listbox.com/member/archive/182191/=now > RSS Feed: https://www.listbox.com/member/archive/rss/182191/23963346-4bb55813 > Modify Your Subscription: https://www.listbox.com/member/?member_id=23963346&id_secret=23963346-89c22f02 > Powered by Listbox: http://www.listbox.com From owner-zfs-devel@FreeBSD.ORG Wed Aug 21 18:07:42 2013 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id C8CA3827; Wed, 21 Aug 2013 18:07:42 +0000 (UTC) (envelope-from ryao@gentoo.org) Received: from smtp.gentoo.org (dev.gentoo.org [IPv6:2001:470:ea4a:1:214:c2ff:fe64:b2d3]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 993912E61; Wed, 21 Aug 2013 18:07:42 +0000 (UTC) Received: from [192.168.1.2] (pool-173-52-119-53.nycmny.fios.verizon.net [173.52.119.53]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) (Authenticated sender: ryao) by smtp.gentoo.org (Postfix) with ESMTPSA id 994A833EB20; Wed, 21 Aug 2013 18:07:38 +0000 (UTC) Message-ID: <521501D6.6040003@gentoo.org> Date: Wed, 21 Aug 2013 14:07:18 -0400 From: Richard Yao User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:17.0) Gecko/20130815 Thunderbird/17.0.8 MIME-Version: 1.0 To: "Justin T. Gibbs" Subject: Re: Updated ashift optimization patch References: <984A4829-FBB0-4174-83FB-06A7336F4737@FreeBSD.org> <8375F87B-F406-491F-BE75-0FAECED680E6@FreeBSD.org> <52128494.2030206@gentoo.org> In-Reply-To: X-Enigmail-Version: 1.5.2 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="8wM1gcRvvTLg3hFNiUWe1q3muL2hpD4wo" Cc: zfs-devel@freebsd.org, zfs@lists.illumos.org X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 Aug 2013 18:07:42 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --8wM1gcRvvTLg3hFNiUWe1q3muL2hpD4wo Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable On 08/20/2013 06:09 PM, Justin T. Gibbs wrote: > The change Steven is proposing would set a system wide *minimum* ashift= for new pools. This would only come into play on devices reporting an a= shift smaller than the set minimum. If that is done, we should have a way for people to override it in place.= --8wM1gcRvvTLg3hFNiUWe1q3muL2hpD4wo Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.20 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQIcBAEBAgAGBQJSFQHYAAoJECDuEZm+6ExkTaAQAJip5FS/hDCyw3IkeM4Y76A0 vS18hIwowXUYCHZv15CSAu2rNLDmaVkjQ0gQjCsjI2poiFvZ2iU2SOlisExJG3o7 MFsvUdbLq9yKPTDdle8y7ylLRM03XKK+m5HI9iQ0ntJqmXeVkVqe79juigHAhXKq GuCUTKklAQqT5XEfGR72jTv7n8Hs/JexPro5s1PkmegVYXH0w4cdIR1xX94kipNY NOrHZmIBfQziyKIDpoxMAUXncmA2qQjXcwZuzlQLfqDItYZh/MAzYc+yprVorP8w nmaHZfRhsopyfc0X4w9OB4cTUe7PWBUVCTHCj+OqQi+CPJuqwqmDNOEiMIv3KN1f vSaqfKzwi0fkXJKzaVTfgEt5lYiNGSG9Lf/joiCbxruRe5jPHMdttd2m0Uw4iPDn 7Fr6YSGZbFzQYpXqBiSxR9/zrFnvb8dMd5eOgATaPgHidzDhP4CsBik66u4bSyN8 1cZQW15Bat/PaUfVM+s6UU9BeMOY9UCxE9K0Ctn4fbClhMozhmRpqtOmyt3j59dv WaQLOJOJxTkD9VKYu5PsErfVrAgdD1AMzx2bRk2B+dnkokOytSi+/oXaQENzPMaJ 1YMeaLChuF0f+5JwmaMw0onHcXS/60xKvW5JruOfC1c6S29r2YId/fVZjzUqXWkv GaogACiGaxCafkC+74F5 =B4BD -----END PGP SIGNATURE----- --8wM1gcRvvTLg3hFNiUWe1q3muL2hpD4wo-- From owner-zfs-devel@FreeBSD.ORG Wed Aug 21 18:23:59 2013 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id A99BA355 for ; Wed, 21 Aug 2013 18:23:59 +0000 (UTC) (envelope-from gibbs@FreeBSD.org) Received: from aslan.scsiguy.com (ns1.scsiguy.com [70.89.174.89]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 85A282FC7 for ; Wed, 21 Aug 2013 18:23:59 +0000 (UTC) Received: from [192.168.6.153] (207-225-98-3.dia.static.qwest.net [207.225.98.3]) (authenticated bits=0) by aslan.scsiguy.com (8.14.7/8.14.5) with ESMTP id r7LINvfD012489 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NO); Wed, 21 Aug 2013 18:23:57 GMT (envelope-from gibbs@FreeBSD.org) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 6.5 \(1508\)) Subject: Re: [zfs] Re: Updated ashift optimization patch From: "Justin T. Gibbs" In-Reply-To: <20130821165145.GB60411@llnl.gov> Date: Wed, 21 Aug 2013 12:23:52 -0600 Content-Transfer-Encoding: quoted-printable Message-Id: <6270646D-50B7-4106-B5DC-5AD9EACCED87@FreeBSD.org> References: <984A4829-FBB0-4174-83FB-06A7336F4737@FreeBSD.org> <8375F87B-F406-491F-BE75-0FAECED680E6@FreeBSD.org> <88F3780B-82D7-4941-82D5-2362845F7533@FreeBSD.org> <20130821165145.GB60411@llnl.gov> To: Prakash Surya X-Mailer: Apple Mail (2.1508) X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.4.3 (aslan.scsiguy.com [70.89.174.89]); Wed, 21 Aug 2013 18:23:58 +0000 (UTC) Cc: zfs@lists.illumos.org, "zfs-devel@freebsd.org" , Richard Yao X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 Aug 2013 18:23:59 -0000 FreeBSD doesn't use vdev_disk.c, so I haven't updated it. -- Justin On Aug 21, 2013, at 10:51 AM, Prakash Surya wrote: > I'm looking at porting the attached patch to ZFS, and I have a = question. >=20 > The patch modifies the typedef of vdev_open_func_t: >=20 > typedef int>---vdev_open_func_t(vdev_t *vd, uint64_t *size, = uint64_t *max_size, > - uint64_t *ashift); > + uint64_t *logical_ashift, uint64_t *physical_ashift); >=20 > But it doesn't modify the prototype for vdev_disk_open like it does = for > the other vdev_*_open calls (i.e. vdev_file_open, vdev_mirror_open, > etc). Is that oversight, or am I misunderstanding something? I see the > vdev_disk_ops structure in the illumos-gate source code, so the > structure isn't Linux specific.. >=20 > --=20 > Cheers, Prakash >=20 > On Tue, Aug 20, 2013 at 04:10:43PM -0600, Justin T. Gibbs wrote: >> Unless I hear otherwise, I plan to commit this patch (with the EINVAL = change described below) as soon as my testing on FreeBSD/head completes. >>=20 >> -- >> Justin >>=20 >> On Aug 18, 2013, at 8:26 PM, Justin T. Gibbs = wrote: >>=20 >>> On Aug 18, 2013, at 6:23 PM, "Steven Hartland" = wrote: >>>=20 >>>> Few things I've noticed:- >>>>=20 >>>> 1. in vdev_geom.c is there a reason you use: >>>> + *physical_ashift =3D 0; >>>> + if (pp->stripesize) >>>> + *physical_ashift =3D highbit(pp->stripesize) - 1; >>>>=20 >>>> Instead of: >>>> + *physical_ashift =3D highbit(MAX(pp->stripesize, = SPA_MINBLOCKSIZE)) - 1; >>>=20 >>> 0 in ashift as in stripesize means "unset" or "don't care". This = gives more information during debugging ("Oh. The driver for this = device must not be setting this.") than if it were clamped. If we = decide it should always be set, I think "MAX(pp->sectorsize, = pp->stripesize)" would make more sense. >>>=20 >>>> 2. sysctl_vfs_zfs_max_auto_ashift should also limit the min to > >>>> SPA_MINBLOCKSHIFT >>>=20 >>> Setting it too low (any value less than the device's logical ashift) = just disables the feature. I don't see any value in adding code to = error out or clamp considering the most likely reason for this to occur = is an administrator trying to turn it off (e.g. set it to 0). >>>=20 >>>> 3. sysctl_vfs_zfs_max_auto_ashift should error if the value is out >>>> of range instead of clamping it. >>>=20 >>> Ok. I now return EINVAL if the value exceeds SPA_MAXASHIFT. >>>=20 >>>> One thing we did here, which would be good to see in this patch, is = to add >>>> an overall min system ashift as thie enables admins to configure = pools >>>> to be compatible with future disks they are likely to use e.g. min = ashift >>>> 12 (4k compatible). This could be left at 9 by default for max = compatibility >>>> but personally I'd suggest 12. >>>=20 >>> it would be nice to hear more consensus come out of the recent zpool = ashift discussions before doing something here. Whatever that is, I = agree that the default should be 9 until someone fixes the RAIDZ space = penalty for going to 4k on 512N drives. >>>=20 >>> -- >>> Justin >>> _______________________________________________ >>> zfs-devel@freebsd.org mailing list >>> http://lists.freebsd.org/mailman/listinfo/zfs-devel >>> To unsubscribe, send any mail to "zfs-devel-unsubscribe@freebsd.org" >>>=20 >>=20 >>=20 >>=20 >> ------------------------------------------- >> illumos-zfs >> Archives: https://www.listbox.com/member/archive/182191/=3Dnow >> RSS Feed: = https://www.listbox.com/member/archive/rss/182191/23963346-4bb55813 >> Modify Your Subscription: = https://www.listbox.com/member/?member_id=3D23963346&id_secret=3D23963346-= 89c22f02 >> Powered by Listbox: http://www.listbox.com > _______________________________________________ > zfs-devel@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/zfs-devel > To unsubscribe, send any mail to "zfs-devel-unsubscribe@freebsd.org" >=20 From owner-zfs-devel@FreeBSD.ORG Wed Aug 21 18:42:20 2013 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 941307D5; Wed, 21 Aug 2013 18:42:20 +0000 (UTC) (envelope-from ryao@gentoo.org) Received: from smtp.gentoo.org (dev.gentoo.org [IPv6:2001:470:ea4a:1:214:c2ff:fe64:b2d3]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 5C7D92104; Wed, 21 Aug 2013 18:42:20 +0000 (UTC) Received: from [192.168.1.2] (pool-173-52-119-53.nycmny.fios.verizon.net [173.52.119.53]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) (Authenticated sender: ryao) by smtp.gentoo.org (Postfix) with ESMTPSA id 197D833EB32; Wed, 21 Aug 2013 18:42:19 +0000 (UTC) Message-ID: <521509F3.1050804@gentoo.org> Date: Wed, 21 Aug 2013 14:41:55 -0400 From: Richard Yao User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:17.0) Gecko/20130815 Thunderbird/17.0.8 MIME-Version: 1.0 To: "Justin T. Gibbs" Subject: Re: [zfs] Re: Updated ashift optimization patch References: <984A4829-FBB0-4174-83FB-06A7336F4737@FreeBSD.org> <8375F87B-F406-491F-BE75-0FAECED680E6@FreeBSD.org> <88F3780B-82D7-4941-82D5-2362845F7533@FreeBSD.org> <20130821165145.GB60411@llnl.gov> <6270646D-50B7-4106-B5DC-5AD9EACCED87@FreeBSD.org> In-Reply-To: <6270646D-50B7-4106-B5DC-5AD9EACCED87@FreeBSD.org> X-Enigmail-Version: 1.5.2 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="wtTiIi2PJvSXjwchO81e5HvOa6kaugdRx" Cc: zfs@lists.illumos.org, Prakash Surya , "zfs-devel@freebsd.org" X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 Aug 2013 18:42:20 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --wtTiIi2PJvSXjwchO81e5HvOa6kaugdRx Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable What happens to the reference to vdev_disk_ops in sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev.c on FreeBSD? Is it simply left undefined? /* * Virtual device management. */ static vdev_ops_t *vdev_ops_table[] =3D { &vdev_root_ops, &vdev_raidz_ops, &vdev_mirror_ops, &vdev_replacing_ops, &vdev_spare_ops, #ifdef _KERNEL &vdev_geom_ops, #else &vdev_disk_ops, #endif &vdev_file_ops, &vdev_missing_ops, &vdev_hole_ops, NULL }; On 08/21/2013 02:23 PM, Justin T. Gibbs wrote: > FreeBSD doesn't use vdev_disk.c, so I haven't updated it. >=20 > -- > Justin >=20 > On Aug 21, 2013, at 10:51 AM, Prakash Surya wrote: >=20 >> I'm looking at porting the attached patch to ZFS, and I have a questio= n. >> >> The patch modifies the typedef of vdev_open_func_t: >> >> typedef int>---vdev_open_func_t(vdev_t *vd, uint64_t *size, uint64= _t *max_size, >> - uint64_t *ashift); >> + uint64_t *logical_ashift, uint64_t *physical_ashift); >> >> But it doesn't modify the prototype for vdev_disk_open like it does fo= r >> the other vdev_*_open calls (i.e. vdev_file_open, vdev_mirror_open, >> etc). Is that oversight, or am I misunderstanding something? I see the= >> vdev_disk_ops structure in the illumos-gate source code, so the >> structure isn't Linux specific.. >> >> --=20 >> Cheers, Prakash >> >> On Tue, Aug 20, 2013 at 04:10:43PM -0600, Justin T. Gibbs wrote: >>> Unless I hear otherwise, I plan to commit this patch (with the EINVAL= change described below) as soon as my testing on FreeBSD/head completes.= >>> >>> -- >>> Justin >>> >>> On Aug 18, 2013, at 8:26 PM, Justin T. Gibbs wrot= e: >>> >>>> On Aug 18, 2013, at 6:23 PM, "Steven Hartland" wrote: >>>> >>>>> Few things I've noticed:- >>>>> >>>>> 1. in vdev_geom.c is there a reason you use: >>>>> + *physical_ashift =3D 0; >>>>> + if (pp->stripesize) >>>>> + *physical_ashift =3D highbit(pp->stripesize) - 1; >>>>> >>>>> Instead of: >>>>> + *physical_ashift =3D highbit(MAX(pp->stripesize, SPA_MINBLOCKSIZE= )) - 1; >>>> >>>> 0 in ashift as in stripesize means "unset" or "don't care". This gi= ves more information during debugging ("Oh. The driver for this device m= ust not be setting this.") than if it were clamped. If we decide it shou= ld always be set, I think "MAX(pp->sectorsize, pp->stripesize)" would mak= e more sense. >>>> >>>>> 2. sysctl_vfs_zfs_max_auto_ashift should also limit the min to > >>>>> SPA_MINBLOCKSHIFT >>>> >>>> Setting it too low (any value less than the device's logical ashift)= just disables the feature. I don't see any value in adding code to erro= r out or clamp considering the most likely reason for this to occur is an= administrator trying to turn it off (e.g. set it to 0). >>>> >>>>> 3. sysctl_vfs_zfs_max_auto_ashift should error if the value is out >>>>> of range instead of clamping it. >>>> >>>> Ok. I now return EINVAL if the value exceeds SPA_MAXASHIFT. >>>> >>>>> One thing we did here, which would be good to see in this patch, is= to add >>>>> an overall min system ashift as thie enables admins to configure po= ols >>>>> to be compatible with future disks they are likely to use e.g. min = ashift >>>>> 12 (4k compatible). This could be left at 9 by default for max comp= atibility >>>>> but personally I'd suggest 12. >>>> >>>> it would be nice to hear more consensus come out of the recent zpool= ashift discussions before doing something here. Whatever that is, I agr= ee that the default should be 9 until someone fixes the RAIDZ space penal= ty for going to 4k on 512N drives. >>>> >>>> -- >>>> Justin >>>> _______________________________________________ >>>> zfs-devel@freebsd.org mailing list >>>> http://lists.freebsd.org/mailman/listinfo/zfs-devel >>>> To unsubscribe, send any mail to "zfs-devel-unsubscribe@freebsd.org"= >>>> >>> >>> >>> >>> ------------------------------------------- >>> illumos-zfs >>> Archives: https://www.listbox.com/member/archive/182191/=3Dnow >>> RSS Feed: https://www.listbox.com/member/archive/rss/182191/23963346-= 4bb55813 >>> Modify Your Subscription: https://www.listbox.com/member/?member_id=3D= 23963346&id_secret=3D23963346-89c22f02 >>> Powered by Listbox: http://www.listbox.com >> _______________________________________________ >> zfs-devel@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/zfs-devel >> To unsubscribe, send any mail to "zfs-devel-unsubscribe@freebsd.org" >> >=20 --wtTiIi2PJvSXjwchO81e5HvOa6kaugdRx Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.20 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQIcBAEBAgAGBQJSFQn5AAoJECDuEZm+6Exk/NwP+QHXCyqVilcHxGhVkt5OAzA7 mpOHKu6horRQn+/D7LbL9ys0kNsz9ni1QjP7vmbUYy2/j1irb3Juuo3Ri7dKAWTn +9ZnXYqh391LWFpuuoRz6H/tLimsamRAw9+cTP3g+iksnOxpqGPVK519GMJkjjSp 4aua0rDKv0EqooTSbJ2/clCSZ/oZ90t44bnfeCBnNhKbYEn1nqDozhjOms+oDLnp Q2ojkg3Hp4R6xPEOiLBL9r7Hw8Xyro0yaWTlddoQZqLVEvJjt9l+7LyIhNXCR6zE 96yMzrQ8JrEhOxUso+96H0blhHSn3Uv1fEUmScgBrEPT4RuL+WN1OtlCoX0GWYWz /flvQcuKUXpuNmxOhap+1WBhcnA6MdOdbK5Kq3cAVRQkmg8Gpw/ak3FAg2uwmxby dtXnNIfP5683Oo+twNEH6eUll4DLEJzPXJ2um78acNiTAYlsBAVWk1IySffM5Jny LuRscYhTiNw/Xnmo7pIYjEcf0pjhdWuiUmSr9tOqDOZqH7MjTk/CkU8s0xQueeEc 1XlhFTuf+Vu+mJl+OuOgE4suCPrrL02eSzT0/BDluSUb5IUmT6KpMjLheRdpirHo 62OWbgdhMiyPnbyaA+Yn6pky0LqII2C+/izpLAHsgHOeevySbxA9UtoHFI/NvWtR /svEJirXVgetiqh1jk2u =Bqzw -----END PGP SIGNATURE----- --wtTiIi2PJvSXjwchO81e5HvOa6kaugdRx-- From owner-zfs-devel@FreeBSD.ORG Wed Aug 21 18:55:25 2013 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 737F9B76 for ; Wed, 21 Aug 2013 18:55:25 +0000 (UTC) (envelope-from gibbs@FreeBSD.org) Received: from aslan.scsiguy.com (ns1.scsiguy.com [70.89.174.89]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 49A442209 for ; Wed, 21 Aug 2013 18:55:25 +0000 (UTC) Received: from [192.168.6.153] (207-225-98-3.dia.static.qwest.net [207.225.98.3]) (authenticated bits=0) by aslan.scsiguy.com (8.14.7/8.14.5) with ESMTP id r7LItNeO012653 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NO); Wed, 21 Aug 2013 18:55:24 GMT (envelope-from gibbs@FreeBSD.org) Content-Type: multipart/signed; boundary="Apple-Mail=_D404345E-85E4-4008-A53B-AD864C627A12"; protocol="application/pgp-signature"; micalg=pgp-sha1 Mime-Version: 1.0 (Mac OS X Mail 6.5 \(1508\)) Subject: Re: [zfs] Re: Updated ashift optimization patch From: "Justin T. Gibbs" In-Reply-To: <521509F3.1050804@gentoo.org> Date: Wed, 21 Aug 2013 12:55:13 -0600 Message-Id: <925C6F98-0D37-40EA-A947-D8E7D45816E3@FreeBSD.org> References: <984A4829-FBB0-4174-83FB-06A7336F4737@FreeBSD.org> <8375F87B-F406-491F-BE75-0FAECED680E6@FreeBSD.org> <88F3780B-82D7-4941-82D5-2362845F7533@FreeBSD.org> <20130821165145.GB60411@llnl.gov> <6270646D-50B7-4106-B5DC-5AD9EACCED87@FreeBSD.org> <521509F3.1050804@gentoo.org> To: Richard Yao X-Mailer: Apple Mail (2.1508) X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.4.3 (aslan.scsiguy.com [70.89.174.89]); Wed, 21 Aug 2013 18:55:24 +0000 (UTC) Cc: zfs@lists.illumos.org, Prakash Surya , "zfs-devel@freebsd.org" X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 Aug 2013 18:55:25 -0000 --Apple-Mail=_D404345E-85E4-4008-A53B-AD864C627A12 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=iso-8859-1 Hmm. That does look a bit messy. vdev_disk.c is not included in = FreeBSD's userland build, so that op vector is left undefined in = libzpool. But, since those ops are never referenced from userland, it = works. -- Justin On Aug 21, 2013, at 12:41 PM, Richard Yao wrote: > What happens to the reference to vdev_disk_ops in > sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev.c on FreeBSD? Is = it > simply left undefined? >=20 > /* > * Virtual device management. > */ >=20 > static vdev_ops_t *vdev_ops_table[] =3D { > &vdev_root_ops, > &vdev_raidz_ops, > &vdev_mirror_ops, > &vdev_replacing_ops, > &vdev_spare_ops, > #ifdef _KERNEL > &vdev_geom_ops, > #else > &vdev_disk_ops, > #endif > &vdev_file_ops, > &vdev_missing_ops, > &vdev_hole_ops, > NULL > }; >=20 > On 08/21/2013 02:23 PM, Justin T. Gibbs wrote: >> FreeBSD doesn't use vdev_disk.c, so I haven't updated it. >>=20 >> -- >> Justin >>=20 >> On Aug 21, 2013, at 10:51 AM, Prakash Surya wrote: >>=20 >>> I'm looking at porting the attached patch to ZFS, and I have a = question. >>>=20 >>> The patch modifies the typedef of vdev_open_func_t: >>>=20 >>> typedef int>---vdev_open_func_t(vdev_t *vd, uint64_t *size, = uint64_t *max_size, >>> - uint64_t *ashift); >>> + uint64_t *logical_ashift, uint64_t *physical_ashift); >>>=20 >>> But it doesn't modify the prototype for vdev_disk_open like it does = for >>> the other vdev_*_open calls (i.e. vdev_file_open, vdev_mirror_open, >>> etc). Is that oversight, or am I misunderstanding something? I see = the >>> vdev_disk_ops structure in the illumos-gate source code, so the >>> structure isn't Linux specific.. >>>=20 >>> --=20 >>> Cheers, Prakash >>>=20 >>> On Tue, Aug 20, 2013 at 04:10:43PM -0600, Justin T. Gibbs wrote: >>>> Unless I hear otherwise, I plan to commit this patch (with the = EINVAL change described below) as soon as my testing on FreeBSD/head = completes. >>>>=20 >>>> -- >>>> Justin >>>>=20 >>>> On Aug 18, 2013, at 8:26 PM, Justin T. Gibbs = wrote: >>>>=20 >>>>> On Aug 18, 2013, at 6:23 PM, "Steven Hartland" = wrote: >>>>>=20 >>>>>> Few things I've noticed:- >>>>>>=20 >>>>>> 1. in vdev_geom.c is there a reason you use: >>>>>> + *physical_ashift =3D 0; >>>>>> + if (pp->stripesize) >>>>>> + *physical_ashift =3D highbit(pp->stripesize) - 1; >>>>>>=20 >>>>>> Instead of: >>>>>> + *physical_ashift =3D highbit(MAX(pp->stripesize, = SPA_MINBLOCKSIZE)) - 1; >>>>>=20 >>>>> 0 in ashift as in stripesize means "unset" or "don't care". This = gives more information during debugging ("Oh. The driver for this = device must not be setting this.") than if it were clamped. If we = decide it should always be set, I think "MAX(pp->sectorsize, = pp->stripesize)" would make more sense. >>>>>=20 >>>>>> 2. sysctl_vfs_zfs_max_auto_ashift should also limit the min to > >>>>>> SPA_MINBLOCKSHIFT >>>>>=20 >>>>> Setting it too low (any value less than the device's logical = ashift) just disables the feature. I don't see any value in adding code = to error out or clamp considering the most likely reason for this to = occur is an administrator trying to turn it off (e.g. set it to 0). >>>>>=20 >>>>>> 3. sysctl_vfs_zfs_max_auto_ashift should error if the value is = out >>>>>> of range instead of clamping it. >>>>>=20 >>>>> Ok. I now return EINVAL if the value exceeds SPA_MAXASHIFT. >>>>>=20 >>>>>> One thing we did here, which would be good to see in this patch, = is to add >>>>>> an overall min system ashift as thie enables admins to configure = pools >>>>>> to be compatible with future disks they are likely to use e.g. = min ashift >>>>>> 12 (4k compatible). This could be left at 9 by default for max = compatibility >>>>>> but personally I'd suggest 12. >>>>>=20 >>>>> it would be nice to hear more consensus come out of the recent = zpool ashift discussions before doing something here. Whatever that is, = I agree that the default should be 9 until someone fixes the RAIDZ space = penalty for going to 4k on 512N drives. >>>>>=20 >>>>> -- >>>>> Justin >>>>> _______________________________________________ >>>>> zfs-devel@freebsd.org mailing list >>>>> http://lists.freebsd.org/mailman/listinfo/zfs-devel >>>>> To unsubscribe, send any mail to = "zfs-devel-unsubscribe@freebsd.org" >>>>>=20 >>>>=20 >>>>=20 >>>>=20 >>>> ------------------------------------------- >>>> illumos-zfs >>>> Archives: https://www.listbox.com/member/archive/182191/=3Dnow >>>> RSS Feed: = https://www.listbox.com/member/archive/rss/182191/23963346-4bb55813 >>>> Modify Your Subscription: = https://www.listbox.com/member/?member_id=3D23963346&id_secret=3D23963346-= 89c22f02 >>>> Powered by Listbox: http://www.listbox.com >>> _______________________________________________ >>> zfs-devel@freebsd.org mailing list >>> http://lists.freebsd.org/mailman/listinfo/zfs-devel >>> To unsubscribe, send any mail to "zfs-devel-unsubscribe@freebsd.org" >>>=20 >>=20 >=20 >=20 --Apple-Mail=_D404345E-85E4-4008-A53B-AD864C627A12 Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Comment: GPGTools - http://gpgtools.org iQEcBAEBAgAGBQJSFQ0WAAoJED9n8CuvaSf4SgUIAKWMJaRx3M5y6Fhz7TImC/hR 2w54E9CJEzMeMvX0g5uXrfgxFJ8y4gGooSdWmyvYcqgKOmFrRhDVpe9Gd71puvPP s4mg5eFJsTSs9cFyJtF6zEF1iXeebUQtIg2ySCWiGzYC+fNyI/fzmQ1Xq6Zere7y euuDFsEuppYOGCRMWn3WZ7mEgT8EV6cBp93/Vpcj/l0zWZIkMnC+jM0hfC4C1T/8 NlFNxjrZpaFs7CpFvdl7EZRGxJCgNVLlHjiTiL7F5k3Ulls58wEwK5iCgbf/ZB+M UMXB6bNVt4jM8w1Ql/htf5oJcCv10bJ2lcdknQI8MvRfSl/k26CZYr5gUgHp/QI= =GoyM -----END PGP SIGNATURE----- --Apple-Mail=_D404345E-85E4-4008-A53B-AD864C627A12-- From owner-zfs-devel@FreeBSD.ORG Wed Aug 21 19:06:46 2013 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 8D06429C for ; Wed, 21 Aug 2013 19:06:46 +0000 (UTC) (envelope-from gibbs@FreeBSD.org) Received: from aslan.scsiguy.com (ns1.scsiguy.com [70.89.174.89]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 5367F22E9 for ; Wed, 21 Aug 2013 19:06:46 +0000 (UTC) Received: from [192.168.6.153] (207-225-98-3.dia.static.qwest.net [207.225.98.3]) (authenticated bits=0) by aslan.scsiguy.com (8.14.7/8.14.5) with ESMTP id r7LJ6iOY012763 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NO); Wed, 21 Aug 2013 19:06:44 GMT (envelope-from gibbs@FreeBSD.org) Content-Type: multipart/signed; boundary="Apple-Mail=_FBCEF618-8421-402A-8B31-B753F7782293"; protocol="application/pgp-signature"; micalg=pgp-sha1 Mime-Version: 1.0 (Mac OS X Mail 6.5 \(1508\)) Subject: Re: [zfs] Updated ashift optimization patch From: "Justin T. Gibbs" In-Reply-To: <925C6F98-0D37-40EA-A947-D8E7D45816E3@FreeBSD.org> Date: Wed, 21 Aug 2013 13:06:30 -0600 Message-Id: <4CA5773B-2FDB-4005-8430-9B4FD0698094@FreeBSD.org> References: <984A4829-FBB0-4174-83FB-06A7336F4737@FreeBSD.org> <8375F87B-F406-491F-BE75-0FAECED680E6@FreeBSD.org> <88F3780B-82D7-4941-82D5-2362845F7533@FreeBSD.org> <20130821165145.GB60411@llnl.gov> <6270646D-50B7-4106-B5DC-5AD9EACCED87@FreeBSD.org> <521509F3.1050804@gentoo.org> <925C6F98-0D37-40EA-A947-D8E7D45816E3@FreeBSD.org> To: Richard Yao X-Mailer: Apple Mail (2.1508) X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.4.3 (aslan.scsiguy.com [70.89.174.89]); Wed, 21 Aug 2013 19:06:45 +0000 (UTC) Cc: zfs@lists.illumos.org, Prakash Surya , "zfs-devel@freebsd.org" X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 Aug 2013 19:06:46 -0000 --Apple-Mail=_FBCEF618-8421-402A-8B31-B753F7782293 Content-Type: multipart/mixed; boundary="Apple-Mail=_CBFCFAA4-7AD2-42F4-A8DD-4BF50867D530" --Apple-Mail=_CBFCFAA4-7AD2-42F4-A8DD-4BF50867D530 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=iso-8859-1 Since you are porting the changes, please use the final version of the = change that I committed to FreeBSD. I think I fixed a few issues in = some comments since the last version I posted. -- Justin --Apple-Mail=_CBFCFAA4-7AD2-42F4-A8DD-4BF50867D530 Content-Disposition: attachment; filename=zfs_ashift.diff Content-Type: application/octet-stream; x-unix-mode=0644; name="zfs_ashift.diff" Content-Transfer-Encoding: 7bit ------------------------------------------------------------------------ r254591 | gibbs | 2013-08-20 22:10:24 -0600 (Tue, 20 Aug 2013) | 172 lines Enhance the ZFS vdev layer to maintain both a logical and a physical minimum allocation size for devices. Use this information to automatically increase ZFS's minimum allocation size for new top-level vdevs to a value that more closely matches the optimum device allocation size. Use GEOM's stripesize attribute, if set, as the physical sector size of the GEOM. Calculate the minimum blocksize of each metaslab class. Use the calculated value instead of SPA_MINBLOCKSIZE (512b) when determining the likelyhood of compression yeilding a reduction in physical space usage. Report devices with sub-optimal block size configuration in "zpool status". Also properly fail attempts to attach devices with a logical block size greater than 8kB, since this will cause corruption to ZFS's label area. Sponsored by: Spectra Logic Corporaion MFC after: 2 weeks Background ========== Many modern devices use physical allocation units that are much larger than the minimum logical allocation size accessible by external commands. Two prevalent examples of this are 512e disk drives (512b logical sector, 4K physical sector) and flash devices (512b logical sector, 4K or larger allocation block size, and 128k or larger erase block size). Operations that modify less than the physical sector size result in a costly read-modify-write or garbage collection sequence on these devices. Simply exporting the true physical sector of the device to ZFS would yield optimal performance, but has two serious drawbacks: 1) Existing pools created with devices that have different logical and physical block sizes, but were configured to use the logical block size (e.g. because the OS version used for pool construction reported the logical block size instead of the physical block size) will suddenly find that the vdev allocation size has increased. This can be easily tolerated for active members of the array, but ZFS would prevent replacement of a vdev with another identical device because it now appears that the smaller allocation size required by the pool is not supported by the new device. 2) The device's physical block size may be too large to be supported by ZFS. The optimal allocation size for the vdev may be quite large. For example, a RAID controller may export a vdev that requires read-modify-write cycles unless accessed using 64k aligned/sized requests. ZFS currently has an 8k minimum block size limit. Reporting both the logical and physical allocation sizes for vdevs solves these problems. A device may be used so long as the logical block size is compatible with the configuration. By comparing the logical and physical block sizes, new configurations can be optimized and administrators can be notified of any existing pools that are sub-optimal. sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/spa.h: Add the SPA_ASHIFT constant. ZFS currently has a hard upper limit of 13 (8k) for ashift and this constant is used to both document and enforce this limit. sys/cddl/contrib/opensolaris/uts/common/sys/fs/zfs.h: Add the VDEV_AUX_ASHIFT_TOO_BIG error code. Add fields for exporting the configured, logical, and physical ashift to the vdev_stat_t structure. Add VDEV_STAT_VALID() macro which can be used to verify the presence of required vdev_stat_t fields in nvlist data. sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev.c: Provide a SYSCTL_PROC handler for "max_auto_ashift". Since the limit is only referenced long after boot when a create operation occurs, there's no compelling need for it to be a boot time configurable tunable. This also allows the validation code for the max_auto_ashift value to be contained within the sysctl handler. Populate the new fields in the vdev_stat_t structure. Fail vdev opens if the vdev reports an ashift larger than SPA_MAXASHIFT. Propogate vdev_logical_ashift and vdev_physical_ashift between child and parent vdevs as is done for vdev_ashift. In vdev_open(), restore code that fails opens for devices where vdev_ashift grows. This can only happen now if the device's logical ashift grows, which means it really isn't safe to use the device. sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/vdev_impl.h: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev.c: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_file.c: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_geom.c: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_mirror.c: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_missing.c: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_raidz.c: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_root.c: Update the vdev_open() API so that both logical (what was just ashift before) and physical ashift are reported. sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/vdev_impl.h: Add two new fields, vdev_physical_ashift and vdev_logical_ashift, to vdev_t. sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev.c: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa_config.c: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa.c: Add vdev_ashift_optimize(). Call it anytime a new top-level vdev is allocated. cddl/contrib/opensolaris/cmd/zpool/zpool_main.c: Add text for the VDEV_AUX_ASHIFT_TOO_BIG error. For each sub-optimally configured leaf vdev, report configured and native block sizes. cddl/contrib/opensolaris/cmd/zpool/zpool_main.c: cddl/contrib/opensolaris/lib/libzfs/common/libzfs.h: cddl/contrib/opensolaris/lib/libzfs/common/libzfs_status.c: Introduce a new zpool status: ZPOOL_STATUS_NON_NATIVE_ASHIFT. This status is reported on healthy pools containing vdevs configured to use a block size smaller than their reported physical block size. cddl/contrib/opensolaris/lib/libzfs/common/libzfs_status.c: Update find_vdev_problem() and supporting functions to provide the full vdev_stat_t structure to problem checking routines, and to allow decent into replacing vdevs. Add a vdev_non_native_ashift() validator which is used on the full vdev tree to check for ZPOOL_STATUS_NON_NATIVE_ASHIFT. cddl/contrib/opensolaris/lib/libzpool/common/kernel.c: cddl/contrib/opensolaris/lib/libzpool/common/sys/zfs_context.h: Enhance sysctl userland stubs now that a SYSCTL_PROC handler is used in vdev.c. sys/cddl/contrib/opensolaris/uts/common/fs/zfs/metaslab.c: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/metaslab_impl.h: When the group membership of a metaslab class changes (i.e. when a vdev is added or removed from a pool), walk the group list to determine the smallest block size currently available and record this in the metaslab class. sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/metaslab.h: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/metaslab.c: Add the metaslab_class_get_minblocksize() accessor. sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/zio_compress.h: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio_compress.c: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c: In zio_compress_data(), take the minimum blocksize as an input parameter instead of assuming SPA_MINBLOCKSIZE. sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c: In l2arc_compress_buf(), pass SPA_MINBLOCKSIZE as the minimum blocksize of the device. The l2arc code performs has it's own code for deciding if compression is worth while, so this effectively disables zio_compress_data() from second guessing the original decision. sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c: In zio_write_bp_init(), use the minimum blocksize of the normal metaslab class when compressing data. Index: cddl/contrib/opensolaris/lib/libzfs/common/libzfs_status.c =================================================================== --- cddl/contrib/opensolaris/lib/libzfs/common/libzfs_status.c (revision 254590) +++ cddl/contrib/opensolaris/lib/libzfs/common/libzfs_status.c (revision 254591) @@ -73,57 +73,66 @@ static char *zfs_msgid_table[] = { /* ARGSUSED */ static int -vdev_missing(uint64_t state, uint64_t aux, uint64_t errs) +vdev_missing(vdev_stat_t *vs, uint_t vsc) { - return (state == VDEV_STATE_CANT_OPEN && - aux == VDEV_AUX_OPEN_FAILED); + return (vs->vs_state == VDEV_STATE_CANT_OPEN && + vs->vs_aux == VDEV_AUX_OPEN_FAILED); } /* ARGSUSED */ static int -vdev_faulted(uint64_t state, uint64_t aux, uint64_t errs) +vdev_faulted(vdev_stat_t *vs, uint_t vsc) { - return (state == VDEV_STATE_FAULTED); + return (vs->vs_state == VDEV_STATE_FAULTED); } /* ARGSUSED */ static int -vdev_errors(uint64_t state, uint64_t aux, uint64_t errs) +vdev_errors(vdev_stat_t *vs, uint_t vsc) { - return (state == VDEV_STATE_DEGRADED || errs != 0); + return (vs->vs_state == VDEV_STATE_DEGRADED || + vs->vs_read_errors != 0 || vs->vs_write_errors != 0 || + vs->vs_checksum_errors != 0); } /* ARGSUSED */ static int -vdev_broken(uint64_t state, uint64_t aux, uint64_t errs) +vdev_broken(vdev_stat_t *vs, uint_t vsc) { - return (state == VDEV_STATE_CANT_OPEN); + return (vs->vs_state == VDEV_STATE_CANT_OPEN); } /* ARGSUSED */ static int -vdev_offlined(uint64_t state, uint64_t aux, uint64_t errs) +vdev_offlined(vdev_stat_t *vs, uint_t vsc) { - return (state == VDEV_STATE_OFFLINE); + return (vs->vs_state == VDEV_STATE_OFFLINE); } /* ARGSUSED */ static int -vdev_removed(uint64_t state, uint64_t aux, uint64_t errs) +vdev_removed(vdev_stat_t *vs, uint_t vsc) { - return (state == VDEV_STATE_REMOVED); + return (vs->vs_state == VDEV_STATE_REMOVED); } +static int +vdev_non_native_ashift(vdev_stat_t *vs, uint_t vsc) +{ + return (VDEV_STAT_VALID(vs_physical_ashift, vsc) && + vs->vs_configured_ashift < vs->vs_physical_ashift); +} + /* * Detect if any leaf devices that have seen errors or could not be opened. */ static boolean_t -find_vdev_problem(nvlist_t *vdev, int (*func)(uint64_t, uint64_t, uint64_t)) +find_vdev_problem(nvlist_t *vdev, int (*func)(vdev_stat_t *, uint_t), + boolean_t ignore_replacing) { nvlist_t **child; vdev_stat_t *vs; - uint_t c, children; - char *type; + uint_t c, vsc, children; /* * Ignore problems within a 'replacing' vdev, since we're presumably in @@ -131,23 +140,25 @@ static boolean_t * out again. We'll pick up the fact that a resilver is happening * later. */ - verify(nvlist_lookup_string(vdev, ZPOOL_CONFIG_TYPE, &type) == 0); - if (strcmp(type, VDEV_TYPE_REPLACING) == 0) - return (B_FALSE); + if (ignore_replacing == B_TRUE) { + char *type; + verify(nvlist_lookup_string(vdev, ZPOOL_CONFIG_TYPE, + &type) == 0); + if (strcmp(type, VDEV_TYPE_REPLACING) == 0) + return (B_FALSE); + } + if (nvlist_lookup_nvlist_array(vdev, ZPOOL_CONFIG_CHILDREN, &child, &children) == 0) { for (c = 0; c < children; c++) - if (find_vdev_problem(child[c], func)) + if (find_vdev_problem(child[c], func, ignore_replacing)) return (B_TRUE); } else { verify(nvlist_lookup_uint64_array(vdev, ZPOOL_CONFIG_VDEV_STATS, - (uint64_t **)&vs, &c) == 0); + (uint64_t **)&vs, &vsc) == 0); - if (func(vs->vs_state, vs->vs_aux, - vs->vs_read_errors + - vs->vs_write_errors + - vs->vs_checksum_errors)) + if (func(vs, vsc) != 0) return (B_TRUE); } @@ -157,7 +168,7 @@ static boolean_t if (nvlist_lookup_nvlist_array(vdev, ZPOOL_CONFIG_L2CACHE, &child, &children) == 0) { for (c = 0; c < children; c++) - if (find_vdev_problem(child[c], func)) + if (find_vdev_problem(child[c], func, ignore_replacing)) return (B_TRUE); } @@ -270,15 +281,15 @@ check_status(nvlist_t *config, boolean_t isimport) * Bad devices in non-replicated config. */ if (vs->vs_state == VDEV_STATE_CANT_OPEN && - find_vdev_problem(nvroot, vdev_faulted)) + find_vdev_problem(nvroot, vdev_faulted, B_TRUE)) return (ZPOOL_STATUS_FAULTED_DEV_NR); if (vs->vs_state == VDEV_STATE_CANT_OPEN && - find_vdev_problem(nvroot, vdev_missing)) + find_vdev_problem(nvroot, vdev_missing, B_TRUE)) return (ZPOOL_STATUS_MISSING_DEV_NR); if (vs->vs_state == VDEV_STATE_CANT_OPEN && - find_vdev_problem(nvroot, vdev_broken)) + find_vdev_problem(nvroot, vdev_broken, B_TRUE)) return (ZPOOL_STATUS_CORRUPT_LABEL_NR); /* @@ -300,32 +311,38 @@ check_status(nvlist_t *config, boolean_t isimport) /* * Missing devices in a replicated config. */ - if (find_vdev_problem(nvroot, vdev_faulted)) + if (find_vdev_problem(nvroot, vdev_faulted, B_TRUE)) return (ZPOOL_STATUS_FAULTED_DEV_R); - if (find_vdev_problem(nvroot, vdev_missing)) + if (find_vdev_problem(nvroot, vdev_missing, B_TRUE)) return (ZPOOL_STATUS_MISSING_DEV_R); - if (find_vdev_problem(nvroot, vdev_broken)) + if (find_vdev_problem(nvroot, vdev_broken, B_TRUE)) return (ZPOOL_STATUS_CORRUPT_LABEL_R); /* * Devices with errors */ - if (!isimport && find_vdev_problem(nvroot, vdev_errors)) + if (!isimport && find_vdev_problem(nvroot, vdev_errors, B_TRUE)) return (ZPOOL_STATUS_FAILING_DEV); /* * Offlined devices */ - if (find_vdev_problem(nvroot, vdev_offlined)) + if (find_vdev_problem(nvroot, vdev_offlined, B_TRUE)) return (ZPOOL_STATUS_OFFLINE_DEV); /* * Removed device */ - if (find_vdev_problem(nvroot, vdev_removed)) + if (find_vdev_problem(nvroot, vdev_removed, B_TRUE)) return (ZPOOL_STATUS_REMOVED_DEV); /* + * Suboptimal, but usable, ashift configuration. + */ + if (find_vdev_problem(nvroot, vdev_non_native_ashift, B_FALSE)) + return (ZPOOL_STATUS_NON_NATIVE_ASHIFT); + + /* * Outdated, but usable, version */ if (SPA_VERSION_IS_SUPPORTED(version) && version != SPA_VERSION) Index: cddl/contrib/opensolaris/lib/libzfs/common/libzfs.h =================================================================== --- cddl/contrib/opensolaris/lib/libzfs/common/libzfs.h (revision 254590) +++ cddl/contrib/opensolaris/lib/libzfs/common/libzfs.h (revision 254591) @@ -326,6 +326,7 @@ typedef enum { ZPOOL_STATUS_RESILVERING, /* device being resilvered */ ZPOOL_STATUS_OFFLINE_DEV, /* device online */ ZPOOL_STATUS_REMOVED_DEV, /* removed device */ + ZPOOL_STATUS_NON_NATIVE_ASHIFT, /* (e.g. 512e dev with ashift of 9) */ /* * Finally, the following indicates a healthy pool. Index: cddl/contrib/opensolaris/lib/libzpool/common/sys/zfs_context.h =================================================================== --- cddl/contrib/opensolaris/lib/libzpool/common/sys/zfs_context.h (revision 254590) +++ cddl/contrib/opensolaris/lib/libzpool/common/sys/zfs_context.h (revision 254591) @@ -659,11 +659,55 @@ typedef uint32_t idmap_rid_t; #define SX_SYSINIT(name, lock, desc) +#define SYSCTL_HANDLER_ARGS struct sysctl_oid *oidp, void *arg1, \ + intptr_t arg2, struct sysctl_req *req + +/* + * This describes the access space for a sysctl request. This is needed + * so that we can use the interface from the kernel or from user-space. + */ +struct sysctl_req { + struct thread *td; /* used for access checking */ + int lock; /* wiring state */ + void *oldptr; + size_t oldlen; + size_t oldidx; + int (*oldfunc)(struct sysctl_req *, const void *, size_t); + void *newptr; + size_t newlen; + size_t newidx; + int (*newfunc)(struct sysctl_req *, void *, size_t); + size_t validlen; + int flags; +}; + +SLIST_HEAD(sysctl_oid_list, sysctl_oid); + +/* + * This describes one "oid" in the MIB tree. Potentially more nodes can + * be hidden behind it, expanded by the handler. + */ +struct sysctl_oid { + struct sysctl_oid_list *oid_parent; + SLIST_ENTRY(sysctl_oid) oid_link; + int oid_number; + u_int oid_kind; + void *oid_arg1; + intptr_t oid_arg2; + const char *oid_name; + int (*oid_handler)(SYSCTL_HANDLER_ARGS); + const char *oid_fmt; + int oid_refcnt; + u_int oid_running; + const char *oid_descr; +}; + #define SYSCTL_DECL(...) #define SYSCTL_NODE(...) #define SYSCTL_INT(...) #define SYSCTL_UINT(...) #define SYSCTL_ULONG(...) +#define SYSCTL_PROC(...) #define SYSCTL_QUAD(...) #define SYSCTL_UQUAD(...) #ifdef TUNABLE_INT @@ -675,6 +719,8 @@ typedef uint32_t idmap_rid_t; #define TUNABLE_ULONG(...) #define TUNABLE_QUAD(...) +int sysctl_handle_64(SYSCTL_HANDLER_ARGS); + /* Errors */ #ifndef ERESTART Index: cddl/contrib/opensolaris/lib/libzpool/common/kernel.c =================================================================== --- cddl/contrib/opensolaris/lib/libzpool/common/kernel.c (revision 254590) +++ cddl/contrib/opensolaris/lib/libzpool/common/kernel.c (revision 254591) @@ -591,6 +591,12 @@ dprintf_setup(int *argc, char **argv) dprintf_print_all = 1; } +int +sysctl_handle_64(SYSCTL_HANDLER_ARGS) +{ + return (0); +} + /* * ========================================================================= * debug printfs Index: cddl/contrib/opensolaris/cmd/zpool/zpool_main.c =================================================================== --- cddl/contrib/opensolaris/cmd/zpool/zpool_main.c (revision 254590) +++ cddl/contrib/opensolaris/cmd/zpool/zpool_main.c (revision 254591) @@ -1295,12 +1295,13 @@ print_status_config(zpool_handle_t *zhp, const cha int namewidth, int depth, boolean_t isspare) { nvlist_t **child; - uint_t c, children; + uint_t c, vsc, children; pool_scan_stat_t *ps = NULL; vdev_stat_t *vs; char rbuf[6], wbuf[6], cbuf[6]; char *vname; uint64_t notpresent; + uint64_t ashift; spare_cbdata_t cb; const char *state; @@ -1309,7 +1310,7 @@ print_status_config(zpool_handle_t *zhp, const cha children = 0; verify(nvlist_lookup_uint64_array(nv, ZPOOL_CONFIG_VDEV_STATS, - (uint64_t **)&vs, &c) == 0); + (uint64_t **)&vs, &vsc) == 0); state = zpool_state_to_name(vs->vs_state, vs->vs_aux); if (isspare) { @@ -1363,6 +1364,10 @@ print_status_config(zpool_handle_t *zhp, const cha (void) printf(gettext("unsupported feature(s)")); break; + case VDEV_AUX_ASHIFT_TOO_BIG: + (void) printf(gettext("unsupported minimum blocksize")); + break; + case VDEV_AUX_SPARED: verify(nvlist_lookup_uint64(nv, ZPOOL_CONFIG_GUID, &cb.cb_guid) == 0); @@ -1405,6 +1410,12 @@ print_status_config(zpool_handle_t *zhp, const cha (void) printf(gettext("corrupted data")); break; } + } else if (children == 0 && !isspare && + VDEV_STAT_VALID(vs_physical_ashift, vsc) && + vs->vs_configured_ashift < vs->vs_physical_ashift) { + (void) printf( + gettext(" block size: %dB configured, %dB native"), + 1 << vs->vs_configured_ashift, 1 << vs->vs_physical_ashift); } (void) nvlist_lookup_uint64_array(nv, ZPOOL_CONFIG_SCAN_STATS, @@ -4268,6 +4279,15 @@ status_callback(zpool_handle_t *zhp, void *data) "'zpool clear'.\n")); break; + case ZPOOL_STATUS_NON_NATIVE_ASHIFT: + (void) printf(gettext("status: One or more devices are " + "configured to use a non-native block size.\n" + "\tExpect reduced performance.\n")); + (void) printf(gettext("action: Replace affected devices with " + "devices that support the\n\tconfigured block size, or " + "migrate data to a properly configured\n\tpool.\n")); + break; + default: /* * The remaining errors can't actually be generated, yet. Index: sys/cddl/contrib/opensolaris/uts/common/sys/fs/zfs.h =================================================================== --- sys/cddl/contrib/opensolaris/uts/common/sys/fs/zfs.h (revision 254590) +++ sys/cddl/contrib/opensolaris/uts/common/sys/fs/zfs.h (revision 254591) @@ -621,7 +621,8 @@ typedef enum vdev_aux { VDEV_AUX_IO_FAILURE, /* experienced I/O failure */ VDEV_AUX_BAD_LOG, /* cannot read log chain(s) */ VDEV_AUX_EXTERNAL, /* external diagnosis */ - VDEV_AUX_SPLIT_POOL /* vdev was split off into another pool */ + VDEV_AUX_SPLIT_POOL, /* vdev was split off into another pool */ + VDEV_AUX_ASHIFT_TOO_BIG /* vdev's min block size is too large */ } vdev_aux_t; /* @@ -715,7 +716,13 @@ typedef struct vdev_stat { uint64_t vs_self_healed; /* self-healed bytes */ uint64_t vs_scan_removing; /* removing? */ uint64_t vs_scan_processed; /* scan processed bytes */ + uint64_t vs_configured_ashift; /* TLV vdev_ashift */ + uint64_t vs_logical_ashift; /* vdev_logical_ashift */ + uint64_t vs_physical_ashift; /* vdev_physical_ashift */ } vdev_stat_t; +#define VDEV_STAT_VALID(field, uint64_t_field_count) \ + ((uint64_t_field_count * sizeof(uint64_t)) >= \ + (offsetof(vdev_stat_t, field) + sizeof(((vdev_stat_t *)NULL)->field))) /* * DDT statistics. Note: all fields should be 64-bit because this Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/spa.h =================================================================== --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/spa.h (revision 254590) +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/spa.h (revision 254591) @@ -93,6 +93,17 @@ struct dsl_dataset; #define SPA_BLOCKSIZES (SPA_MAXBLOCKSHIFT - SPA_MINBLOCKSHIFT + 1) /* + * Maximum supported logical ashift. + * + * The current 8k allocation block size limit is due to the 8k + * aligned/sized operations performed by vdev_probe() on + * vdev_label->vl_pad2. Using another "safe region" for these tests + * would allow the limit to be raised to 16k, at the expense of + * only having 8 available uberblocks in the label area. + */ +#define SPA_MAXASHIFT 13 + +/* * Size of block to hold the configuration data (a packed nvlist) */ #define SPA_CONFIG_BLOCKSIZE (1ULL << 14) Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/vdev.h =================================================================== --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/vdev.h (revision 254590) +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/vdev.h (revision 254591) @@ -78,6 +78,7 @@ extern void vdev_rele(vdev_t *); extern int vdev_metaslab_init(vdev_t *vd, uint64_t txg); extern void vdev_metaslab_fini(vdev_t *vd); extern void vdev_metaslab_set_size(vdev_t *); +extern void vdev_ashift_optimize(vdev_t *); extern void vdev_expand(vdev_t *vd, uint64_t txg); extern void vdev_split(vdev_t *vd); extern void vdev_deadman(vdev_t *vd); Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/metaslab.h =================================================================== --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/metaslab.h (revision 254590) +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/metaslab.h (revision 254591) @@ -70,6 +70,7 @@ extern uint64_t metaslab_class_get_alloc(metaslab_ extern uint64_t metaslab_class_get_space(metaslab_class_t *mc); extern uint64_t metaslab_class_get_dspace(metaslab_class_t *mc); extern uint64_t metaslab_class_get_deferred(metaslab_class_t *mc); +extern uint64_t metaslab_class_get_minblocksize(metaslab_class_t *mc); extern metaslab_group_t *metaslab_group_create(metaslab_class_t *mc, vdev_t *vd); Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/zio_compress.h =================================================================== --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/zio_compress.h (revision 254590) +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/zio_compress.h (revision 254591) @@ -79,7 +79,7 @@ extern int lz4_decompress(void *src, void *dst, si * Compress and decompress data if necessary. */ extern size_t zio_compress_data(enum zio_compress c, void *src, void *dst, - size_t s_len); + size_t s_len, size_t minblocksize); extern int zio_decompress_data(enum zio_compress c, void *src, void *dst, size_t s_len, size_t d_len); Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/vdev_impl.h =================================================================== --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/vdev_impl.h (revision 254590) +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/vdev_impl.h (revision 254591) @@ -57,7 +57,7 @@ typedef struct vdev_cache_entry vdev_cache_entry_t * Virtual device operations */ typedef int vdev_open_func_t(vdev_t *vd, uint64_t *size, uint64_t *max_size, - uint64_t *ashift); + uint64_t *logical_ashift, uint64_t *physical_ashift); typedef void vdev_close_func_t(vdev_t *vd); typedef uint64_t vdev_asize_func_t(vdev_t *vd, uint64_t psize); typedef int vdev_io_start_func_t(zio_t *zio); @@ -123,6 +123,24 @@ struct vdev { uint64_t vdev_min_asize; /* min acceptable asize */ uint64_t vdev_max_asize; /* max acceptable asize */ uint64_t vdev_ashift; /* block alignment shift */ + /* + * Logical block alignment shift + * + * The smallest sized/aligned I/O supported by the device. + */ + uint64_t vdev_logical_ashift; + /* + * Physical block alignment shift + * + * The device supports logical I/Os with vdev_logical_ashift + * size/alignment, but optimum performance will be achieved by + * aligning/sizing requests to vdev_physical_ashift. Smaller + * requests may be inflated or incur device level read-modify-write + * operations. + * + * May be 0 to indicate no preference (i.e. use vdev_logical_ashift). + */ + uint64_t vdev_physical_ashift; uint64_t vdev_state; /* see VDEV_STATE_* #defines */ uint64_t vdev_prevstate; /* used when reopening a vdev */ vdev_ops_t *vdev_ops; /* vdev operations */ Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/metaslab_impl.h =================================================================== --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/metaslab_impl.h (revision 254590) +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/metaslab_impl.h (revision 254591) @@ -49,6 +49,7 @@ struct metaslab_class { uint64_t mc_deferred; /* total deferred frees */ uint64_t mc_space; /* total space (alloc + free) */ uint64_t mc_dspace; /* total deflated space */ + uint64_t mc_minblocksize; }; struct metaslab_group { Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_mirror.c =================================================================== --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_mirror.c (revision 254590) +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_mirror.c (revision 254591) @@ -132,7 +132,7 @@ vdev_mirror_map_alloc(zio_t *zio) static int vdev_mirror_open(vdev_t *vd, uint64_t *asize, uint64_t *max_asize, - uint64_t *ashift) + uint64_t *logical_ashift, uint64_t *physical_ashift) { int numerrors = 0; int lasterror = 0; @@ -155,7 +155,9 @@ vdev_mirror_open(vdev_t *vd, uint64_t *asize, uint *asize = MIN(*asize - 1, cvd->vdev_asize - 1) + 1; *max_asize = MIN(*max_asize - 1, cvd->vdev_max_asize - 1) + 1; - *ashift = MAX(*ashift, cvd->vdev_ashift); + *logical_ashift = MAX(*logical_ashift, cvd->vdev_ashift); + *physical_ashift = MAX(*physical_ashift, + cvd->vdev_physical_ashift); } if (numerrors == vd->vdev_children) { Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev.c =================================================================== --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev.c (revision 254590) +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev.c (revision 254591) @@ -52,6 +52,51 @@ SYSCTL_NODE(_vfs_zfs, OID_AUTO, vdev, CTLFLAG_RW, * Virtual device management. */ +/** + * The limit for ZFS to automatically increase a top-level vdev's ashift + * from logical ashift to physical ashift. + * + * Example: one or more 512B emulation child vdevs + * child->vdev_ashift = 9 (512 bytes) + * child->vdev_physical_ashift = 12 (4096 bytes) + * zfs_max_auto_ashift = 11 (2048 bytes) + * + * On pool creation or the addition of a new top-leve vdev, ZFS will + * bump the ashift of the top-level vdev to 2048. + * + * Example: one or more 512B emulation child vdevs + * child->vdev_ashift = 9 (512 bytes) + * child->vdev_physical_ashift = 12 (4096 bytes) + * zfs_max_auto_ashift = 13 (8192 bytes) + * + * On pool creation or the addition of a new top-leve vdev, ZFS will + * bump the ashift of the top-level vdev to 4096. + */ +static uint64_t zfs_max_auto_ashift = SPA_MAXASHIFT; + +static int +sysctl_vfs_zfs_max_auto_ashift(SYSCTL_HANDLER_ARGS) +{ + uint64_t val; + int err; + + val = zfs_max_auto_ashift; + err = sysctl_handle_64(oidp, &val, 0, req); + if (err != 0 || req->newptr == NULL) + return (err); + + if (val > SPA_MAXASHIFT) + val = SPA_MAXASHIFT; + + zfs_max_auto_ashift = val; + + return (0); +} +SYSCTL_PROC(_vfs_zfs, OID_AUTO, max_auto_ashift, + CTLTYPE_U64 | CTLFLAG_MPSAFE | CTLFLAG_RW, 0, sizeof(uint64_t), + sysctl_vfs_zfs_max_auto_ashift, "QU", + "Cap on logical -> physical ashift adjustment on new top-level vdevs."); + static vdev_ops_t *vdev_ops_table[] = { &vdev_root_ops, &vdev_raidz_ops, @@ -746,6 +791,8 @@ vdev_add_parent(vdev_t *cvd, vdev_ops_t *ops) mvd->vdev_min_asize = cvd->vdev_min_asize; mvd->vdev_max_asize = cvd->vdev_max_asize; mvd->vdev_ashift = cvd->vdev_ashift; + mvd->vdev_logical_ashift = cvd->vdev_logical_ashift; + mvd->vdev_physical_ashift = cvd->vdev_physical_ashift; mvd->vdev_state = cvd->vdev_state; mvd->vdev_crtxg = cvd->vdev_crtxg; @@ -777,6 +824,8 @@ vdev_remove_parent(vdev_t *cvd) mvd->vdev_ops == &vdev_replacing_ops || mvd->vdev_ops == &vdev_spare_ops); cvd->vdev_ashift = mvd->vdev_ashift; + cvd->vdev_logical_ashift = mvd->vdev_logical_ashift; + cvd->vdev_physical_ashift = mvd->vdev_physical_ashift; vdev_remove_child(mvd, cvd); vdev_remove_child(pvd, mvd); @@ -1120,7 +1169,8 @@ vdev_open(vdev_t *vd) uint64_t osize = 0; uint64_t max_osize = 0; uint64_t asize, max_asize, psize; - uint64_t ashift = 0; + uint64_t logical_ashift = 0; + uint64_t physical_ashift = 0; ASSERT(vd->vdev_open_thread == curthread || spa_config_held(spa, SCL_STATE_ALL, RW_WRITER) == SCL_STATE_ALL); @@ -1150,7 +1200,8 @@ vdev_open(vdev_t *vd) return (SET_ERROR(ENXIO)); } - error = vd->vdev_ops->vdev_op_open(vd, &osize, &max_osize, &ashift); + error = vd->vdev_ops->vdev_op_open(vd, &osize, &max_osize, + &logical_ashift, &physical_ashift); /* * Reset the vdev_reopening flag so that we actually close @@ -1248,6 +1299,17 @@ vdev_open(vdev_t *vd) return (SET_ERROR(EINVAL)); } + vd->vdev_physical_ashift = + MAX(physical_ashift, vd->vdev_physical_ashift); + vd->vdev_logical_ashift = MAX(logical_ashift, vd->vdev_logical_ashift); + vd->vdev_ashift = MAX(vd->vdev_logical_ashift, vd->vdev_ashift); + + if (vd->vdev_logical_ashift > SPA_MAXASHIFT) { + vdev_set_state(vd, B_TRUE, VDEV_STATE_CANT_OPEN, + VDEV_AUX_ASHIFT_TOO_BIG); + return (EINVAL); + } + if (vd->vdev_asize == 0) { /* * This is the first-ever open, so use the computed values. @@ -1255,19 +1317,15 @@ vdev_open(vdev_t *vd) */ vd->vdev_asize = asize; vd->vdev_max_asize = max_asize; - vd->vdev_ashift = MAX(ashift, vd->vdev_ashift); } else { /* - * Detect if the alignment requirement has increased. - * We don't want to make the pool unavailable, just - * issue a warning instead. + * Make sure the alignment requirement hasn't increased. */ - if (ashift > vd->vdev_top->vdev_ashift && + if (vd->vdev_ashift > vd->vdev_top->vdev_ashift && vd->vdev_ops->vdev_op_leaf) { - cmn_err(CE_WARN, - "Disk, '%s', has a block alignment that is " - "larger than the pool's alignment\n", - vd->vdev_path); + vdev_set_state(vd, B_TRUE, VDEV_STATE_CANT_OPEN, + VDEV_AUX_BAD_LABEL); + return (EINVAL); } vd->vdev_max_asize = max_asize; } @@ -1577,7 +1635,24 @@ vdev_metaslab_set_size(vdev_t *vd) vd->vdev_ms_shift = MAX(vd->vdev_ms_shift, SPA_MAXBLOCKSHIFT); } +/* + * Maximize performance by inflating the configured ashift for + * top level vdevs to be as close to the physical ashift as + * possible without exceeding the administrator specified + * limit. + */ void +vdev_ashift_optimize(vdev_t *vd) +{ + if (vd == vd->vdev_top && + (vd->vdev_ashift < vd->vdev_physical_ashift) && + (vd->vdev_ashift < zfs_max_auto_ashift)) { + vd->vdev_ashift = MIN(zfs_max_auto_ashift, + vd->vdev_physical_ashift); + } +} + +void vdev_dirty(vdev_t *vd, int flags, void *arg, uint64_t txg) { ASSERT(vd == vd->vdev_top); @@ -2595,6 +2670,10 @@ vdev_get_stats(vdev_t *vd, vdev_stat_t *vs) if (vd->vdev_ops->vdev_op_leaf) vs->vs_rsize += VDEV_LABEL_START_SIZE + VDEV_LABEL_END_SIZE; vs->vs_esize = vd->vdev_max_asize - vd->vdev_asize; + vs->vs_configured_ashift = vd->vdev_top != NULL + ? vd->vdev_top->vdev_ashift : vd->vdev_ashift; + vs->vs_logical_ashift = vd->vdev_logical_ashift; + vs->vs_physical_ashift = vd->vdev_physical_ashift; mutex_exit(&vd->vdev_stat_lock); /* Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c =================================================================== --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c (revision 254590) +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c (revision 254591) @@ -5147,7 +5147,7 @@ l2arc_compress_buf(l2arc_buf_hdr_t *l2hdr) len = l2hdr->b_asize; cdata = zio_data_buf_alloc(len); csize = zio_compress_data(ZIO_COMPRESS_LZ4, l2hdr->b_tmp_cdata, - cdata, l2hdr->b_asize); + cdata, l2hdr->b_asize, (size_t)SPA_MINBLOCKSIZE); if (csize == 0) { /* zero block, indicate that there's nothing to write */ Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa_config.c =================================================================== --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa_config.c (revision 254590) +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa_config.c (revision 254591) @@ -519,8 +519,10 @@ spa_config_update(spa_t *spa, int what) */ for (c = 0; c < rvd->vdev_children; c++) { vdev_t *tvd = rvd->vdev_child[c]; - if (tvd->vdev_ms_array == 0) + if (tvd->vdev_ms_array == 0) { + vdev_ashift_optimize(tvd); vdev_metaslab_set_size(tvd); + } vdev_expand(tvd, txg); } } Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_geom.c =================================================================== --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_geom.c (revision 254590) +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_geom.c (revision 254591) @@ -576,7 +576,7 @@ vdev_geom_open_by_path(vdev_t *vd, int check_guid) static int vdev_geom_open(vdev_t *vd, uint64_t *psize, uint64_t *max_psize, - uint64_t *ashift) + uint64_t *logical_ashift, uint64_t *physical_ashift) { struct g_provider *pp; struct g_consumer *cp; @@ -662,9 +662,13 @@ vdev_geom_open(vdev_t *vd, uint64_t *psize, uint64 *max_psize = *psize = pp->mediasize; /* - * Determine the device's minimum transfer size. + * Determine the device's minimum transfer size and preferred + * transfer size. */ - *ashift = highbit(MAX(pp->sectorsize, SPA_MINBLOCKSIZE)) - 1; + *logical_ashift = highbit(MAX(pp->sectorsize, SPA_MINBLOCKSIZE)) - 1; + *physical_ashift = 0; + if (pp->stripesize) + *physical_ashift = highbit(pp->stripesize) - 1; /* * Clear the nowritecache settings, so that on a vdev_reopen() Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/metaslab.c =================================================================== --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/metaslab.c (revision 254590) +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/metaslab.c (revision 254591) @@ -180,6 +180,27 @@ metaslab_class_space_update(metaslab_class_t *mc, atomic_add_64(&mc->mc_dspace, dspace_delta); } +void +metaslab_class_minblocksize_update(metaslab_class_t *mc) +{ + metaslab_group_t *mg; + vdev_t *vd; + uint64_t minashift = UINT64_MAX; + + if ((mg = mc->mc_rotor) == NULL) { + mc->mc_minblocksize = SPA_MINBLOCKSIZE; + return; + } + + do { + vd = mg->mg_vd; + if (vd->vdev_ashift < minashift) + minashift = vd->vdev_ashift; + } while ((mg = mg->mg_next) != mc->mc_rotor); + + mc->mc_minblocksize = 1ULL << minashift; +} + uint64_t metaslab_class_get_alloc(metaslab_class_t *mc) { @@ -204,6 +225,12 @@ metaslab_class_get_dspace(metaslab_class_t *mc) return (spa_deflate(mc->mc_spa) ? mc->mc_dspace : mc->mc_space); } +uint64_t +metaslab_class_get_minblocksize(metaslab_class_t *mc) +{ + return (mc->mc_minblocksize); +} + /* * ========================================================================== * Metaslab groups @@ -295,6 +322,7 @@ metaslab_group_activate(metaslab_group_t *mg) mgnext->mg_prev = mg; } mc->mc_rotor = mg; + metaslab_class_minblocksize_update(mc); } void @@ -326,6 +354,7 @@ metaslab_group_passivate(metaslab_group_t *mg) mg->mg_prev = NULL; mg->mg_next = NULL; + metaslab_class_minblocksize_update(mc); } static void Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_raidz.c =================================================================== --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_raidz.c (revision 254590) +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_raidz.c (revision 254591) @@ -1478,7 +1478,7 @@ vdev_raidz_reconstruct(raidz_map_t *rm, int *t, in static int vdev_raidz_open(vdev_t *vd, uint64_t *asize, uint64_t *max_asize, - uint64_t *ashift) + uint64_t *logical_ashift, uint64_t *physical_ashift) { vdev_t *cvd; uint64_t nparity = vd->vdev_nparity; @@ -1507,7 +1507,9 @@ vdev_raidz_open(vdev_t *vd, uint64_t *asize, uint6 *asize = MIN(*asize - 1, cvd->vdev_asize - 1) + 1; *max_asize = MIN(*max_asize - 1, cvd->vdev_max_asize - 1) + 1; - *ashift = MAX(*ashift, cvd->vdev_ashift); + *logical_ashift = MAX(*logical_ashift, cvd->vdev_ashift); + *physical_ashift = MAX(*physical_ashift, + cvd->vdev_physical_ashift); } *asize *= vd->vdev_children; Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_missing.c =================================================================== --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_missing.c (revision 254590) +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_missing.c (revision 254591) @@ -45,7 +45,7 @@ /* ARGSUSED */ static int vdev_missing_open(vdev_t *vd, uint64_t *psize, uint64_t *max_psize, - uint64_t *ashift) + uint64_t *logical_ashift, uint64_t *physical_ashift) { /* * Really this should just fail. But then the root vdev will be in the @@ -55,7 +55,8 @@ vdev_missing_open(vdev_t *vd, uint64_t *psize, uin */ *psize = 0; *max_psize = 0; - *ashift = 0; + *logical_ashift = 0; + *physical_ashift = 0; return (0); } Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c =================================================================== --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c (revision 254590) +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c (revision 254591) @@ -1137,8 +1137,10 @@ zio_write_bp_init(zio_t *zio) } if (compress != ZIO_COMPRESS_OFF) { + metaslab_class_t *mc = spa_normal_class(spa); void *cbuf = zio_buf_alloc(lsize); - psize = zio_compress_data(compress, zio->io_data, cbuf, lsize); + psize = zio_compress_data(compress, zio->io_data, cbuf, lsize, + (size_t)metaslab_class_get_minblocksize(mc)); if (psize == 0 || psize == lsize) { compress = ZIO_COMPRESS_OFF; zio_buf_free(cbuf, lsize); Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_file.c =================================================================== --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_file.c (revision 254590) +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_file.c (revision 254591) @@ -49,7 +49,7 @@ vdev_file_rele(vdev_t *vd) static int vdev_file_open(vdev_t *vd, uint64_t *psize, uint64_t *max_psize, - uint64_t *ashift) + uint64_t *logical_ashift, uint64_t *physical_ashift) { vdev_file_t *vf; vnode_t *vp; @@ -130,7 +130,8 @@ skip_open: } *max_psize = *psize = vattr.va_size; - *ashift = SPA_MINBLOCKSHIFT; + *logical_ashift = SPA_MINBLOCKSHIFT; + *physical_ashift = SPA_MINBLOCKSHIFT; return (0); } Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_root.c =================================================================== --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_root.c (revision 254590) +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_root.c (revision 254591) @@ -55,7 +55,7 @@ too_many_errors(vdev_t *vd, int numerrors) static int vdev_root_open(vdev_t *vd, uint64_t *asize, uint64_t *max_asize, - uint64_t *ashift) + uint64_t *logical_ashift, uint64_t *physical_ashift) { int lasterror = 0; int numerrors = 0; @@ -83,7 +83,8 @@ vdev_root_open(vdev_t *vd, uint64_t *asize, uint64 *asize = 0; *max_asize = 0; - *ashift = 0; + *logical_ashift = 0; + *physical_ashift = 0; return (0); } Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio_compress.c =================================================================== --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio_compress.c (revision 254590) +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio_compress.c (revision 254591) @@ -77,7 +77,8 @@ zio_compress_select(enum zio_compress child, enum } size_t -zio_compress_data(enum zio_compress c, void *src, void *dst, size_t s_len) +zio_compress_data(enum zio_compress c, void *src, void *dst, size_t s_len, + size_t minblocksize) { uint64_t *word, *word_end; size_t c_len, d_len, r_len; @@ -102,7 +103,7 @@ size_t return (s_len); /* Compress at least 12.5% */ - d_len = P2ALIGN(s_len - (s_len >> 3), (size_t)SPA_MINBLOCKSIZE); + d_len = P2ALIGN(s_len - (s_len >> 3), minblocksize); if (d_len == 0) return (s_len); @@ -115,14 +116,14 @@ size_t * Cool. We compressed at least as much as we were hoping to. * For both security and repeatability, pad out the last sector. */ - r_len = P2ROUNDUP(c_len, (size_t)SPA_MINBLOCKSIZE); + r_len = P2ROUNDUP(c_len, minblocksize); if (r_len > c_len) { bzero((char *)dst + c_len, r_len - c_len); c_len = r_len; } ASSERT3U(c_len, <=, d_len); - ASSERT(P2PHASE(c_len, (size_t)SPA_MINBLOCKSIZE) == 0); + ASSERT(P2PHASE(c_len, minblocksize) == 0); return (c_len); } Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa.c =================================================================== --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa.c (revision 254590) +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa.c (revision 254591) @@ -3424,6 +3424,7 @@ spa_create(const char *pool, nvlist_t *nvroot, nvl (error = spa_validate_aux(spa, nvroot, txg, VDEV_ALLOC_ADD)) == 0) { for (int c = 0; c < rvd->vdev_children; c++) { + vdev_ashift_optimize(rvd->vdev_child[c]); vdev_metaslab_set_size(rvd->vdev_child[c]); vdev_expand(rvd->vdev_child[c], txg); } ------------------------------------------------------------------------ --Apple-Mail=_CBFCFAA4-7AD2-42F4-A8DD-4BF50867D530 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=iso-8859-1 On Aug 21, 2013, at 12:55 PM, Justin T. Gibbs wrote: > Hmm. That does look a bit messy. vdev_disk.c is not included in = FreeBSD's userland build, so that op vector is left undefined in = libzpool. But, since those ops are never referenced from userland, it = works. >=20 > -- > Justin >=20 > On Aug 21, 2013, at 12:41 PM, Richard Yao wrote: >=20 >> What happens to the reference to vdev_disk_ops in >> sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev.c on FreeBSD? Is = it >> simply left undefined? >>=20 >> /* >> * Virtual device management. >> */ >>=20 >> static vdev_ops_t *vdev_ops_table[] =3D { >> &vdev_root_ops, >> &vdev_raidz_ops, >> &vdev_mirror_ops, >> &vdev_replacing_ops, >> &vdev_spare_ops, >> #ifdef _KERNEL >> &vdev_geom_ops, >> #else >> &vdev_disk_ops, >> #endif >> &vdev_file_ops, >> &vdev_missing_ops, >> &vdev_hole_ops, >> NULL >> }; >>=20 >> On 08/21/2013 02:23 PM, Justin T. Gibbs wrote: >>> FreeBSD doesn't use vdev_disk.c, so I haven't updated it. >>>=20 >>> -- >>> Justin >>>=20 >>> On Aug 21, 2013, at 10:51 AM, Prakash Surya wrote: >>>=20 >>>> I'm looking at porting the attached patch to ZFS, and I have a = question. >>>>=20 >>>> The patch modifies the typedef of vdev_open_func_t: >>>>=20 >>>> typedef int>---vdev_open_func_t(vdev_t *vd, uint64_t *size, = uint64_t *max_size, >>>> - uint64_t *ashift); >>>> + uint64_t *logical_ashift, uint64_t *physical_ashift); >>>>=20 >>>> But it doesn't modify the prototype for vdev_disk_open like it does = for >>>> the other vdev_*_open calls (i.e. vdev_file_open, vdev_mirror_open, >>>> etc). Is that oversight, or am I misunderstanding something? I see = the >>>> vdev_disk_ops structure in the illumos-gate source code, so the >>>> structure isn't Linux specific.. >>>>=20 >>>> --=20 >>>> Cheers, Prakash >>>>=20 >>>> On Tue, Aug 20, 2013 at 04:10:43PM -0600, Justin T. Gibbs wrote: >>>>> Unless I hear otherwise, I plan to commit this patch (with the = EINVAL change described below) as soon as my testing on FreeBSD/head = completes. >>>>>=20 >>>>> -- >>>>> Justin >>>>>=20 >>>>> On Aug 18, 2013, at 8:26 PM, Justin T. Gibbs = wrote: >>>>>=20 >>>>>> On Aug 18, 2013, at 6:23 PM, "Steven Hartland" = wrote: >>>>>>=20 >>>>>>> Few things I've noticed:- >>>>>>>=20 >>>>>>> 1. in vdev_geom.c is there a reason you use: >>>>>>> + *physical_ashift =3D 0; >>>>>>> + if (pp->stripesize) >>>>>>> + *physical_ashift =3D highbit(pp->stripesize) - 1; >>>>>>>=20 >>>>>>> Instead of: >>>>>>> + *physical_ashift =3D highbit(MAX(pp->stripesize, = SPA_MINBLOCKSIZE)) - 1; >>>>>>=20 >>>>>> 0 in ashift as in stripesize means "unset" or "don't care". This = gives more information during debugging ("Oh. The driver for this = device must not be setting this.") than if it were clamped. If we = decide it should always be set, I think "MAX(pp->sectorsize, = pp->stripesize)" would make more sense. >>>>>>=20 >>>>>>> 2. sysctl_vfs_zfs_max_auto_ashift should also limit the min to > >>>>>>> SPA_MINBLOCKSHIFT >>>>>>=20 >>>>>> Setting it too low (any value less than the device's logical = ashift) just disables the feature. I don't see any value in adding code = to error out or clamp considering the most likely reason for this to = occur is an administrator trying to turn it off (e.g. set it to 0). >>>>>>=20 >>>>>>> 3. sysctl_vfs_zfs_max_auto_ashift should error if the value is = out >>>>>>> of range instead of clamping it. >>>>>>=20 >>>>>> Ok. I now return EINVAL if the value exceeds SPA_MAXASHIFT. >>>>>>=20 >>>>>>> One thing we did here, which would be good to see in this patch, = is to add >>>>>>> an overall min system ashift as thie enables admins to configure = pools >>>>>>> to be compatible with future disks they are likely to use e.g. = min ashift >>>>>>> 12 (4k compatible). This could be left at 9 by default for max = compatibility >>>>>>> but personally I'd suggest 12. >>>>>>=20 >>>>>> it would be nice to hear more consensus come out of the recent = zpool ashift discussions before doing something here. Whatever that is, = I agree that the default should be 9 until someone fixes the RAIDZ space = penalty for going to 4k on 512N drives. >>>>>>=20 >>>>>> -- >>>>>> Justin >>>>>> _______________________________________________ >>>>>> zfs-devel@freebsd.org mailing list >>>>>> http://lists.freebsd.org/mailman/listinfo/zfs-devel >>>>>> To unsubscribe, send any mail to = "zfs-devel-unsubscribe@freebsd.org" >>>>>>=20 >>>>>=20 >>>>>=20 >>>>>=20 >>>>> ------------------------------------------- >>>>> illumos-zfs >>>>> Archives: https://www.listbox.com/member/archive/182191/=3Dnow >>>>> RSS Feed: = https://www.listbox.com/member/archive/rss/182191/23963346-4bb55813 >>>>> Modify Your Subscription: = https://www.listbox.com/member/?member_id=3D23963346&id_secret=3D23963346-= 89c22f02 >>>>> Powered by Listbox: http://www.listbox.com >>>> _______________________________________________ >>>> zfs-devel@freebsd.org mailing list >>>> http://lists.freebsd.org/mailman/listinfo/zfs-devel >>>> To unsubscribe, send any mail to = "zfs-devel-unsubscribe@freebsd.org" >>>>=20 >>>=20 >>=20 >>=20 >=20 --Apple-Mail=_CBFCFAA4-7AD2-42F4-A8DD-4BF50867D530-- --Apple-Mail=_FBCEF618-8421-402A-8B31-B753F7782293 Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Comment: GPGTools - http://gpgtools.org iQEcBAEBAgAGBQJSFQ+/AAoJED9n8CuvaSf4U34IAIAtEYJnbJGwMie8i/Pwoae8 DF8O1OzRRw49J7P4yvk6lcekatemQEuiZPWC8WZWHvWEW6kFyDx38rdoGbpGK9YE X/wdsOaouhIOvNxYBkG0qyi2UYPuMTuf50jcVB/v07wMy0MiygNVVba6zHu637My jR7jcM13j43YRrLmR4DJiqfBcY1Cnm0dUaqZu/Uqe6EXYh4IRw5nfAQ6NuOXuwJi W3H3nYm3J2nBYUj8+RQlWAp2JrHPVYEGf2ymVQYO7nGlQUPxt/ScvbHfEo2Jkle4 xvOsFbsdS4L8J3j+8YJ1h9f2a/VwrH0YIZKWCzSJEsFqXqHAGwnwZ5fpUe7/4to= =xGiK -----END PGP SIGNATURE----- --Apple-Mail=_FBCEF618-8421-402A-8B31-B753F7782293-- From owner-zfs-devel@FreeBSD.ORG Wed Aug 21 19:19:43 2013 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 9C8CB6EF; Wed, 21 Aug 2013 19:19:43 +0000 (UTC) (envelope-from surya1@llnl.gov) Received: from prdiron-3.llnl.gov (prdiron-3.llnl.gov [128.15.143.173]) by mx1.freebsd.org (Postfix) with ESMTP id 813CC23D7; Wed, 21 Aug 2013 19:19:43 +0000 (UTC) X-Attachments: Received: from eris.llnl.gov ([128.115.7.7]) by prdiron-3.llnl.gov with ESMTP; 21 Aug 2013 12:18:36 -0700 Received: from ubuntu (spunky.chaos [192.168.1.143]) by eris.llnl.gov (Postfix) with ESMTP id EDF277C4A5; Wed, 21 Aug 2013 12:18:35 -0700 (PDT) Received: by ubuntu (Postfix, from userid 37859) id D7C9A1E2ED; Wed, 21 Aug 2013 12:18:35 -0700 (PDT) Date: Wed, 21 Aug 2013 12:18:35 -0700 From: Prakash Surya To: "Justin T. Gibbs" Subject: Re: [zfs] Updated ashift optimization patch Message-ID: <20130821191835.GG60411@llnl.gov> References: <984A4829-FBB0-4174-83FB-06A7336F4737@FreeBSD.org> <8375F87B-F406-491F-BE75-0FAECED680E6@FreeBSD.org> <88F3780B-82D7-4941-82D5-2362845F7533@FreeBSD.org> <20130821165145.GB60411@llnl.gov> <6270646D-50B7-4106-B5DC-5AD9EACCED87@FreeBSD.org> <521509F3.1050804@gentoo.org> <925C6F98-0D37-40EA-A947-D8E7D45816E3@FreeBSD.org> <4CA5773B-2FDB-4005-8430-9B4FD0698094@FreeBSD.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4CA5773B-2FDB-4005-8430-9B4FD0698094@FreeBSD.org> User-Agent: Mutt/1.5.20 (2009-06-14) Cc: Richard Yao , "zfs-devel@freebsd.org" , zfs@lists.illumos.org X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 Aug 2013 19:19:43 -0000 Sure, where is the upstream repo for FreeBSD kept? -- Cheers, Prakash On Wed, Aug 21, 2013 at 01:06:30PM -0600, Justin T. Gibbs wrote: > Since you are porting the changes, please use the final version of the change that I committed to FreeBSD. I think I fixed a few issues in some comments since the last version I posted. > > -- > Justin > > > > On Aug 21, 2013, at 12:55 PM, Justin T. Gibbs wrote: > > > Hmm. That does look a bit messy. vdev_disk.c is not included in FreeBSD's userland build, so that op vector is left undefined in libzpool. But, since those ops are never referenced from userland, it works. > > > > -- > > Justin > > > > On Aug 21, 2013, at 12:41 PM, Richard Yao wrote: > > > >> What happens to the reference to vdev_disk_ops in > >> sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev.c on FreeBSD? Is it > >> simply left undefined? > >> > >> /* > >> * Virtual device management. > >> */ > >> > >> static vdev_ops_t *vdev_ops_table[] = { > >> &vdev_root_ops, > >> &vdev_raidz_ops, > >> &vdev_mirror_ops, > >> &vdev_replacing_ops, > >> &vdev_spare_ops, > >> #ifdef _KERNEL > >> &vdev_geom_ops, > >> #else > >> &vdev_disk_ops, > >> #endif > >> &vdev_file_ops, > >> &vdev_missing_ops, > >> &vdev_hole_ops, > >> NULL > >> }; > >> > >> On 08/21/2013 02:23 PM, Justin T. Gibbs wrote: > >>> FreeBSD doesn't use vdev_disk.c, so I haven't updated it. > >>> > >>> -- > >>> Justin > >>> > >>> On Aug 21, 2013, at 10:51 AM, Prakash Surya wrote: > >>> > >>>> I'm looking at porting the attached patch to ZFS, and I have a question. > >>>> > >>>> The patch modifies the typedef of vdev_open_func_t: > >>>> > >>>> typedef int>---vdev_open_func_t(vdev_t *vd, uint64_t *size, uint64_t *max_size, > >>>> - uint64_t *ashift); > >>>> + uint64_t *logical_ashift, uint64_t *physical_ashift); > >>>> > >>>> But it doesn't modify the prototype for vdev_disk_open like it does for > >>>> the other vdev_*_open calls (i.e. vdev_file_open, vdev_mirror_open, > >>>> etc). Is that oversight, or am I misunderstanding something? I see the > >>>> vdev_disk_ops structure in the illumos-gate source code, so the > >>>> structure isn't Linux specific.. > >>>> > >>>> -- > >>>> Cheers, Prakash > >>>> > >>>> On Tue, Aug 20, 2013 at 04:10:43PM -0600, Justin T. Gibbs wrote: > >>>>> Unless I hear otherwise, I plan to commit this patch (with the EINVAL change described below) as soon as my testing on FreeBSD/head completes. > >>>>> > >>>>> -- > >>>>> Justin > >>>>> > >>>>> On Aug 18, 2013, at 8:26 PM, Justin T. Gibbs wrote: > >>>>> > >>>>>> On Aug 18, 2013, at 6:23 PM, "Steven Hartland" wrote: > >>>>>> > >>>>>>> Few things I've noticed:- > >>>>>>> > >>>>>>> 1. in vdev_geom.c is there a reason you use: > >>>>>>> + *physical_ashift = 0; > >>>>>>> + if (pp->stripesize) > >>>>>>> + *physical_ashift = highbit(pp->stripesize) - 1; > >>>>>>> > >>>>>>> Instead of: > >>>>>>> + *physical_ashift = highbit(MAX(pp->stripesize, SPA_MINBLOCKSIZE)) - 1; > >>>>>> > >>>>>> 0 in ashift as in stripesize means "unset" or "don't care". This gives more information during debugging ("Oh. The driver for this device must not be setting this.") than if it were clamped. If we decide it should always be set, I think "MAX(pp->sectorsize, pp->stripesize)" would make more sense. > >>>>>> > >>>>>>> 2. sysctl_vfs_zfs_max_auto_ashift should also limit the min to > > >>>>>>> SPA_MINBLOCKSHIFT > >>>>>> > >>>>>> Setting it too low (any value less than the device's logical ashift) just disables the feature. I don't see any value in adding code to error out or clamp considering the most likely reason for this to occur is an administrator trying to turn it off (e.g. set it to 0). > >>>>>> > >>>>>>> 3. sysctl_vfs_zfs_max_auto_ashift should error if the value is out > >>>>>>> of range instead of clamping it. > >>>>>> > >>>>>> Ok. I now return EINVAL if the value exceeds SPA_MAXASHIFT. > >>>>>> > >>>>>>> One thing we did here, which would be good to see in this patch, is to add > >>>>>>> an overall min system ashift as thie enables admins to configure pools > >>>>>>> to be compatible with future disks they are likely to use e.g. min ashift > >>>>>>> 12 (4k compatible). This could be left at 9 by default for max compatibility > >>>>>>> but personally I'd suggest 12. > >>>>>> > >>>>>> it would be nice to hear more consensus come out of the recent zpool ashift discussions before doing something here. Whatever that is, I agree that the default should be 9 until someone fixes the RAIDZ space penalty for going to 4k on 512N drives. > >>>>>> > >>>>>> -- > >>>>>> Justin > >>>>>> _______________________________________________ > >>>>>> zfs-devel@freebsd.org mailing list > >>>>>> http://lists.freebsd.org/mailman/listinfo/zfs-devel > >>>>>> To unsubscribe, send any mail to "zfs-devel-unsubscribe@freebsd.org" > >>>>>> > >>>>> > >>>>> > >>>>> > >>>>> ------------------------------------------- > >>>>> illumos-zfs > >>>>> Archives: https://www.listbox.com/member/archive/182191/=now > >>>>> RSS Feed: https://www.listbox.com/member/archive/rss/182191/23963346-4bb55813 > >>>>> Modify Your Subscription: https://www.listbox.com/member/?member_id=23963346&id_secret=23963346-89c22f02 > >>>>> Powered by Listbox: http://www.listbox.com > >>>> _______________________________________________ > >>>> zfs-devel@freebsd.org mailing list > >>>> http://lists.freebsd.org/mailman/listinfo/zfs-devel > >>>> To unsubscribe, send any mail to "zfs-devel-unsubscribe@freebsd.org" > >>>> > >>> > >> > >> > > > From owner-zfs-devel@FreeBSD.ORG Wed Aug 21 19:30:23 2013 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id AA6CDADA for ; Wed, 21 Aug 2013 19:30:23 +0000 (UTC) (envelope-from gibbs@FreeBSD.org) Received: from aslan.scsiguy.com (aslan.scsiguy.com [70.89.174.89]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 834FB24E1 for ; Wed, 21 Aug 2013 19:30:23 +0000 (UTC) Received: from [192.168.6.153] (207-225-98-3.dia.static.qwest.net [207.225.98.3]) (authenticated bits=0) by aslan.scsiguy.com (8.14.7/8.14.5) with ESMTP id r7LJUL7g012911 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NO); Wed, 21 Aug 2013 19:30:22 GMT (envelope-from gibbs@FreeBSD.org) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 6.5 \(1508\)) Subject: Re: [zfs] Updated ashift optimization patch From: "Justin T. Gibbs" In-Reply-To: <20130821191835.GG60411@llnl.gov> Date: Wed, 21 Aug 2013 13:30:16 -0600 Content-Transfer-Encoding: quoted-printable Message-Id: <81826551-0DF2-46E4-BCA1-174A002449D5@FreeBSD.org> References: <984A4829-FBB0-4174-83FB-06A7336F4737@FreeBSD.org> <8375F87B-F406-491F-BE75-0FAECED680E6@FreeBSD.org> <88F3780B-82D7-4941-82D5-2362845F7533@FreeBSD.org> <20130821165145.GB60411@llnl.gov> <6270646D-50B7-4106-B5DC-5AD9EACCED87@FreeBSD.org> <521509F3.1050804@gentoo.org> <925C6F98-0D37-40EA-A947-D8E7D45816E3@FreeBSD.org> <4CA5773B-2FDB-4005-8430-9B4FD0698094@FreeBSD.org> <20130821191835.GG60411@llnl.gov> To: zfs@lists.illumos.org X-Mailer: Apple Mail (2.1508) X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.4.3 (aslan.scsiguy.com [70.89.174.89]); Wed, 21 Aug 2013 19:30:22 +0000 (UTC) Cc: Richard Yao , "zfs-devel@freebsd.org" X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 Aug 2013 19:30:23 -0000 The revision was included in my previous email, but you can pull = directly from the source if you'd like. FreeBSD's master repo is in subversion at svn://svn.freebsd.org/base. = You'll want the "head" branch. The git mirror repo at git://github.com/freebsd/freebsd.git is = unofficial but seems to work. -- Justin On Aug 21, 2013, at 1:18 PM, Prakash Surya wrote: > Sure, where is the upstream repo for FreeBSD kept? >=20 > --=20 > Cheers, Prakash >=20 > On Wed, Aug 21, 2013 at 01:06:30PM -0600, Justin T. Gibbs wrote: >> Since you are porting the changes, please use the final version of = the change that I committed to FreeBSD. I think I fixed a few issues in = some comments since the last version I posted. >>=20 >> -- >> Justin >>=20 >=20 >=20 >>=20 >>=20 >> On Aug 21, 2013, at 12:55 PM, Justin T. Gibbs = wrote: >>=20 >>> Hmm. That does look a bit messy. vdev_disk.c is not included in = FreeBSD's userland build, so that op vector is left undefined in = libzpool. But, since those ops are never referenced from userland, it = works. >>>=20 >>> -- >>> Justin >>>=20 >>> On Aug 21, 2013, at 12:41 PM, Richard Yao wrote: >>>=20 >>>> What happens to the reference to vdev_disk_ops in >>>> sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev.c on FreeBSD? = Is it >>>> simply left undefined? >>>>=20 >>>> /* >>>> * Virtual device management. >>>> */ >>>>=20 >>>> static vdev_ops_t *vdev_ops_table[] =3D { >>>> &vdev_root_ops, >>>> &vdev_raidz_ops, >>>> &vdev_mirror_ops, >>>> &vdev_replacing_ops, >>>> &vdev_spare_ops, >>>> #ifdef _KERNEL >>>> &vdev_geom_ops, >>>> #else >>>> &vdev_disk_ops, >>>> #endif >>>> &vdev_file_ops, >>>> &vdev_missing_ops, >>>> &vdev_hole_ops, >>>> NULL >>>> }; >>>>=20 >>>> On 08/21/2013 02:23 PM, Justin T. Gibbs wrote: >>>>> FreeBSD doesn't use vdev_disk.c, so I haven't updated it. >>>>>=20 >>>>> -- >>>>> Justin >>>>>=20 >>>>> On Aug 21, 2013, at 10:51 AM, Prakash Surya = wrote: >>>>>=20 >>>>>> I'm looking at porting the attached patch to ZFS, and I have a = question. >>>>>>=20 >>>>>> The patch modifies the typedef of vdev_open_func_t: >>>>>>=20 >>>>>> typedef int>---vdev_open_func_t(vdev_t *vd, uint64_t *size, = uint64_t *max_size, >>>>>> - uint64_t *ashift); >>>>>> + uint64_t *logical_ashift, uint64_t *physical_ashift); >>>>>>=20 >>>>>> But it doesn't modify the prototype for vdev_disk_open like it = does for >>>>>> the other vdev_*_open calls (i.e. vdev_file_open, = vdev_mirror_open, >>>>>> etc). Is that oversight, or am I misunderstanding something? I = see the >>>>>> vdev_disk_ops structure in the illumos-gate source code, so the >>>>>> structure isn't Linux specific.. >>>>>>=20 >>>>>> --=20 >>>>>> Cheers, Prakash >>>>>>=20 >>>>>> On Tue, Aug 20, 2013 at 04:10:43PM -0600, Justin T. Gibbs wrote: >>>>>>> Unless I hear otherwise, I plan to commit this patch (with the = EINVAL change described below) as soon as my testing on FreeBSD/head = completes. >>>>>>>=20 >>>>>>> -- >>>>>>> Justin >>>>>>>=20 >>>>>>> On Aug 18, 2013, at 8:26 PM, Justin T. Gibbs = wrote: >>>>>>>=20 >>>>>>>> On Aug 18, 2013, at 6:23 PM, "Steven Hartland" = wrote: >>>>>>>>=20 >>>>>>>>> Few things I've noticed:- >>>>>>>>>=20 >>>>>>>>> 1. in vdev_geom.c is there a reason you use: >>>>>>>>> + *physical_ashift =3D 0; >>>>>>>>> + if (pp->stripesize) >>>>>>>>> + *physical_ashift =3D highbit(pp->stripesize) - 1; >>>>>>>>>=20 >>>>>>>>> Instead of: >>>>>>>>> + *physical_ashift =3D highbit(MAX(pp->stripesize, = SPA_MINBLOCKSIZE)) - 1; >>>>>>>>=20 >>>>>>>> 0 in ashift as in stripesize means "unset" or "don't care". = This gives more information during debugging ("Oh. The driver for this = device must not be setting this.") than if it were clamped. If we = decide it should always be set, I think "MAX(pp->sectorsize, = pp->stripesize)" would make more sense. >>>>>>>>=20 >>>>>>>>> 2. sysctl_vfs_zfs_max_auto_ashift should also limit the min to = > >>>>>>>>> SPA_MINBLOCKSHIFT >>>>>>>>=20 >>>>>>>> Setting it too low (any value less than the device's logical = ashift) just disables the feature. I don't see any value in adding code = to error out or clamp considering the most likely reason for this to = occur is an administrator trying to turn it off (e.g. set it to 0). >>>>>>>>=20 >>>>>>>>> 3. sysctl_vfs_zfs_max_auto_ashift should error if the value is = out >>>>>>>>> of range instead of clamping it. >>>>>>>>=20 >>>>>>>> Ok. I now return EINVAL if the value exceeds SPA_MAXASHIFT. >>>>>>>>=20 >>>>>>>>> One thing we did here, which would be good to see in this = patch, is to add >>>>>>>>> an overall min system ashift as thie enables admins to = configure pools >>>>>>>>> to be compatible with future disks they are likely to use e.g. = min ashift >>>>>>>>> 12 (4k compatible). This could be left at 9 by default for max = compatibility >>>>>>>>> but personally I'd suggest 12. >>>>>>>>=20 >>>>>>>> it would be nice to hear more consensus come out of the recent = zpool ashift discussions before doing something here. Whatever that is, = I agree that the default should be 9 until someone fixes the RAIDZ space = penalty for going to 4k on 512N drives. >>>>>>>>=20 >>>>>>>> -- >>>>>>>> Justin >>>>>>>> _______________________________________________ >>>>>>>> zfs-devel@freebsd.org mailing list >>>>>>>> http://lists.freebsd.org/mailman/listinfo/zfs-devel >>>>>>>> To unsubscribe, send any mail to = "zfs-devel-unsubscribe@freebsd.org" >>>>>>>>=20 >>>>>>>=20 >>>>>>>=20 >>>>>>>=20 >>>>>>> ------------------------------------------- >>>>>>> illumos-zfs >>>>>>> Archives: https://www.listbox.com/member/archive/182191/=3Dnow >>>>>>> RSS Feed: = https://www.listbox.com/member/archive/rss/182191/23963346-4bb55813 >>>>>>> Modify Your Subscription: https://www.listbox.com/member/?& >>>>>>> Powered by Listbox: http://www.listbox.com >>>>>> _______________________________________________ >>>>>> zfs-devel@freebsd.org mailing list >>>>>> http://lists.freebsd.org/mailman/listinfo/zfs-devel >>>>>> To unsubscribe, send any mail to = "zfs-devel-unsubscribe@freebsd.org" >>>>>>=20 >>>>>=20 >>>>=20 >>>>=20 >>>=20 >>=20 >=20 >=20 >=20 >=20 >=20 > ------------------------------------------- > illumos-zfs > Archives: https://www.listbox.com/member/archive/182191/=3Dnow > RSS Feed: = https://www.listbox.com/member/archive/rss/182191/23052111-5da6ea43 > Modify Your Subscription: = https://www.listbox.com/member/?member_id=3D23052111&id_secret=3D23052111-= e1f18a8a > Powered by Listbox: http://www.listbox.com >=20 From owner-zfs-devel@FreeBSD.ORG Thu Aug 22 15:06:29 2013 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 6E80C5ED for ; Thu, 22 Aug 2013 15:06:29 +0000 (UTC) (envelope-from surya1@llnl.gov) Received: from prdiron-1.llnl.gov (prdiron-1.llnl.gov [128.15.143.171]) by mx1.freebsd.org (Postfix) with ESMTP id 5346523DD for ; Thu, 22 Aug 2013 15:06:29 +0000 (UTC) X-Attachments: Received: from eris.llnl.gov ([128.115.7.7]) by prdiron-1.llnl.gov with ESMTP; 22 Aug 2013 08:05:20 -0700 Received: from ubuntu (spunky.chaos [192.168.1.143]) by eris.llnl.gov (Postfix) with ESMTP id 9135F7C4A5; Thu, 22 Aug 2013 08:05:20 -0700 (PDT) Received: by ubuntu (Postfix, from userid 37859) id 4AE12207ED; Thu, 22 Aug 2013 08:05:20 -0700 (PDT) Date: Thu, 22 Aug 2013 08:05:20 -0700 From: Prakash Surya To: zfs@lists.illumos.org Subject: Re: [zfs] Updated ashift optimization patch Message-ID: <20130822150520.GH60411@llnl.gov> References: <8375F87B-F406-491F-BE75-0FAECED680E6@FreeBSD.org> <88F3780B-82D7-4941-82D5-2362845F7533@FreeBSD.org> <20130821165145.GB60411@llnl.gov> <6270646D-50B7-4106-B5DC-5AD9EACCED87@FreeBSD.org> <521509F3.1050804@gentoo.org> <925C6F98-0D37-40EA-A947-D8E7D45816E3@FreeBSD.org> <4CA5773B-2FDB-4005-8430-9B4FD0698094@FreeBSD.org> <20130821191835.GG60411@llnl.gov> <81826551-0DF2-46E4-BCA1-174A002449D5@FreeBSD.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <81826551-0DF2-46E4-BCA1-174A002449D5@FreeBSD.org> User-Agent: Mutt/1.5.20 (2009-06-14) Cc: Richard Yao , "zfs-devel@freebsd.org" X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Aug 2013 15:06:29 -0000 Thanks. The github repo is easier for me to work with, so I pulled from there. I pushed a pull request to ZFS on Linux: https://github.com/zfsonlinux/zfs/pull/1671 The patch applied relatively clean. The main changes I made for the port were removing the GEOM and SYSCTL diffs. The SYSCTL changes should probably be moved to a module parameter for Linux, but I haven't done that yet. -- Cheers, Prakash On Wed, Aug 21, 2013 at 01:30:16PM -0600, Justin T. Gibbs wrote: > The revision was included in my previous email, but you can pull directly from the source if you'd like. > > FreeBSD's master repo is in subversion at svn://svn.freebsd.org/base. You'll want the "head" branch. > > The git mirror repo at git://github.com/freebsd/freebsd.git is unofficial but seems to work. > > -- > Justin > > On Aug 21, 2013, at 1:18 PM, Prakash Surya wrote: > > > Sure, where is the upstream repo for FreeBSD kept? > > > > -- > > Cheers, Prakash > > > > On Wed, Aug 21, 2013 at 01:06:30PM -0600, Justin T. Gibbs wrote: > >> Since you are porting the changes, please use the final version of the change that I committed to FreeBSD. I think I fixed a few issues in some comments since the last version I posted. > >> > >> -- > >> Justin > >> > > > > > >> > >> > >> On Aug 21, 2013, at 12:55 PM, Justin T. Gibbs wrote: > >> > >>> Hmm. That does look a bit messy. vdev_disk.c is not included in FreeBSD's userland build, so that op vector is left undefined in libzpool. But, since those ops are never referenced from userland, it works. > >>> > >>> -- > >>> Justin > >>> > >>> On Aug 21, 2013, at 12:41 PM, Richard Yao wrote: > >>> > >>>> What happens to the reference to vdev_disk_ops in > >>>> sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev.c on FreeBSD? Is it > >>>> simply left undefined? > >>>> > >>>> /* > >>>> * Virtual device management. > >>>> */ > >>>> > >>>> static vdev_ops_t *vdev_ops_table[] = { > >>>> &vdev_root_ops, > >>>> &vdev_raidz_ops, > >>>> &vdev_mirror_ops, > >>>> &vdev_replacing_ops, > >>>> &vdev_spare_ops, > >>>> #ifdef _KERNEL > >>>> &vdev_geom_ops, > >>>> #else > >>>> &vdev_disk_ops, > >>>> #endif > >>>> &vdev_file_ops, > >>>> &vdev_missing_ops, > >>>> &vdev_hole_ops, > >>>> NULL > >>>> }; > >>>> > >>>> On 08/21/2013 02:23 PM, Justin T. Gibbs wrote: > >>>>> FreeBSD doesn't use vdev_disk.c, so I haven't updated it. > >>>>> > >>>>> -- > >>>>> Justin > >>>>> > >>>>> On Aug 21, 2013, at 10:51 AM, Prakash Surya wrote: > >>>>> > >>>>>> I'm looking at porting the attached patch to ZFS, and I have a question. > >>>>>> > >>>>>> The patch modifies the typedef of vdev_open_func_t: > >>>>>> > >>>>>> typedef int>---vdev_open_func_t(vdev_t *vd, uint64_t *size, uint64_t *max_size, > >>>>>> - uint64_t *ashift); > >>>>>> + uint64_t *logical_ashift, uint64_t *physical_ashift); > >>>>>> > >>>>>> But it doesn't modify the prototype for vdev_disk_open like it does for > >>>>>> the other vdev_*_open calls (i.e. vdev_file_open, vdev_mirror_open, > >>>>>> etc). Is that oversight, or am I misunderstanding something? I see the > >>>>>> vdev_disk_ops structure in the illumos-gate source code, so the > >>>>>> structure isn't Linux specific.. > >>>>>> > >>>>>> -- > >>>>>> Cheers, Prakash > >>>>>> > >>>>>> On Tue, Aug 20, 2013 at 04:10:43PM -0600, Justin T. Gibbs wrote: > >>>>>>> Unless I hear otherwise, I plan to commit this patch (with the EINVAL change described below) as soon as my testing on FreeBSD/head completes. > >>>>>>> > >>>>>>> -- > >>>>>>> Justin > >>>>>>> > >>>>>>> On Aug 18, 2013, at 8:26 PM, Justin T. Gibbs wrote: > >>>>>>> > >>>>>>>> On Aug 18, 2013, at 6:23 PM, "Steven Hartland" wrote: > >>>>>>>> > >>>>>>>>> Few things I've noticed:- > >>>>>>>>> > >>>>>>>>> 1. in vdev_geom.c is there a reason you use: > >>>>>>>>> + *physical_ashift = 0; > >>>>>>>>> + if (pp->stripesize) > >>>>>>>>> + *physical_ashift = highbit(pp->stripesize) - 1; > >>>>>>>>> > >>>>>>>>> Instead of: > >>>>>>>>> + *physical_ashift = highbit(MAX(pp->stripesize, SPA_MINBLOCKSIZE)) - 1; > >>>>>>>> > >>>>>>>> 0 in ashift as in stripesize means "unset" or "don't care". This gives more information during debugging ("Oh. The driver for this device must not be setting this.") than if it were clamped. If we decide it should always be set, I think "MAX(pp->sectorsize, pp->stripesize)" would make more sense. > >>>>>>>> > >>>>>>>>> 2. sysctl_vfs_zfs_max_auto_ashift should also limit the min to > > >>>>>>>>> SPA_MINBLOCKSHIFT > >>>>>>>> > >>>>>>>> Setting it too low (any value less than the device's logical ashift) just disables the feature. I don't see any value in adding code to error out or clamp considering the most likely reason for this to occur is an administrator trying to turn it off (e.g. set it to 0). > >>>>>>>> > >>>>>>>>> 3. sysctl_vfs_zfs_max_auto_ashift should error if the value is out > >>>>>>>>> of range instead of clamping it. > >>>>>>>> > >>>>>>>> Ok. I now return EINVAL if the value exceeds SPA_MAXASHIFT. > >>>>>>>> > >>>>>>>>> One thing we did here, which would be good to see in this patch, is to add > >>>>>>>>> an overall min system ashift as thie enables admins to configure pools > >>>>>>>>> to be compatible with future disks they are likely to use e.g. min ashift > >>>>>>>>> 12 (4k compatible). This could be left at 9 by default for max compatibility > >>>>>>>>> but personally I'd suggest 12. > >>>>>>>> > >>>>>>>> it would be nice to hear more consensus come out of the recent zpool ashift discussions before doing something here. Whatever that is, I agree that the default should be 9 until someone fixes the RAIDZ space penalty for going to 4k on 512N drives. > >>>>>>>> > >>>>>>>> -- > >>>>>>>> Justin > >>>>>>>> _______________________________________________ > >>>>>>>> zfs-devel@freebsd.org mailing list > >>>>>>>> http://lists.freebsd.org/mailman/listinfo/zfs-devel > >>>>>>>> To unsubscribe, send any mail to "zfs-devel-unsubscribe@freebsd.org" > >>>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> ------------------------------------------- > >>>>>>> illumos-zfs > >>>>>>> Archives: https://www.listbox.com/member/archive/182191/=now > >>>>>>> RSS Feed: https://www.listbox.com/member/archive/rss/182191/23963346-4bb55813 > >>>>>>> Modify Your Subscription: https://www.listbox.com/member/?& > >>>>>>> Powered by Listbox: http://www.listbox.com > >>>>>> _______________________________________________ > >>>>>> zfs-devel@freebsd.org mailing list > >>>>>> http://lists.freebsd.org/mailman/listinfo/zfs-devel > >>>>>> To unsubscribe, send any mail to "zfs-devel-unsubscribe@freebsd.org" > >>>>>> > >>>>> > >>>> > >>>> > >>> > >> > > > > > > > > > > > > ------------------------------------------- > > illumos-zfs > > Archives: https://www.listbox.com/member/archive/182191/=now > > RSS Feed: https://www.listbox.com/member/archive/rss/182191/23052111-5da6ea43 > > Modify Your Subscription: https://www.listbox.com/member/?& > > Powered by Listbox: http://www.listbox.com > > > > > > ------------------------------------------- > illumos-zfs > Archives: https://www.listbox.com/member/archive/182191/=now > RSS Feed: https://www.listbox.com/member/archive/rss/182191/23963346-4bb55813 > Modify Your Subscription: https://www.listbox.com/member/?member_id=23963346&id_secret=23963346-89c22f02 > Powered by Listbox: http://www.listbox.com From owner-zfs-devel@FreeBSD.ORG Fri Aug 30 14:27:48 2013 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id B4C1DB5C for ; Fri, 30 Aug 2013 14:27:48 +0000 (UTC) (envelope-from smh@freebsd.org) Received: from smtp1.multiplay.co.uk (smtp1.multiplay.co.uk [85.236.96.35]) by mx1.freebsd.org (Postfix) with ESMTP id 655702916 for ; Fri, 30 Aug 2013 14:27:48 +0000 (UTC) Received: by smtp1.multiplay.co.uk (Postfix, from userid 65534) id 8467D20E7088A; Fri, 30 Aug 2013 14:21:52 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.multiplay.co.uk X-Spam-Level: X-Spam-Status: No, score=-2.5 required=8.0 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham version=3.3.1 Received: from r2d2 (82-69-141-170.dsl.in-addr.zen.co.uk [82.69.141.170]) by smtp1.multiplay.co.uk (Postfix) with ESMTPA id 9021420E70847 for ; Fri, 30 Aug 2013 14:21:50 +0000 (UTC) Message-ID: <5CBC43F8D341437D9811A3117615F314@multiplay.co.uk> From: "Steven Hartland" To: Subject: Improving zvol support Date: Fri, 30 Aug 2013 15:22:24 +0100 MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="----=_NextPart_000_0FCA_01CEA594.BA51F800" X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.5931 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.6157 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 30 Aug 2013 14:27:48 -0000 This is a multi-part message in MIME format. ------=_NextPart_000_0FCA_01CEA594.BA51F800 Content-Type: text/plain; format=flowed; charset="iso-8859-1"; reply-type=original Content-Transfer-Encoding: 7bit There's a few outstanding PR's for zvol issues such as:- http://www.freebsd.org/cgi/query-pr.cgi?pr=178999 I've attached a patch to this PR (also to this email) which helps clear quite a few issues highlighted by this PR and by tests in STF. The locking side of this patch while not 100% ideal does prevent the common issues such as zvol renames so I think worth getting this committing until more time can be spent on it. What do people think? Regards Steve ------=_NextPart_000_0FCA_01CEA594.BA51F800 Content-Type: application/octet-stream; name="zvol-fixes.patch" Content-Transfer-Encoding: quoted-printable Content-Disposition: attachment; filename="zvol-fixes.patch" Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_ioctl.c=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_ioctl.c (revision = 253623)=0A= +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_ioctl.c (working = copy)=0A= @@ -3296,6 +3296,10 @@=0A= if (error !=3D 0)=0A= (void) dsl_destroy_head(fsname);=0A= }=0A= +#ifdef __FreeBSD__=0A= + if (error =3D=3D 0)=0A= + zvol_create_minors(fsname);=0A= +#endif=0A= return (error);=0A= }=0A= =0A= Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dsl_dataset.c=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dsl_dataset.c = (revision 253623)=0A= +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dsl_dataset.c = (working copy)=0A= @@ -1289,8 +1289,7 @@=0A= fnvlist_free(suspended);=0A= }=0A= =0A= -#ifdef __FreeBSD__=0A= -#ifdef _KERNEL=0A= +#if defined(__FreeBSD__) && defined(_KERNEL)=0A= if (error =3D=3D 0) {=0A= for (pair =3D nvlist_next_nvpair(snaps, NULL); pair !=3D NULL;=0A= pair =3D nvlist_next_nvpair(snaps, pair)) {=0A= @@ -1299,7 +1298,6 @@=0A= }=0A= }=0A= #endif=0A= -#endif=0A= return (error);=0A= }=0A= =0A= @@ -1671,11 +1669,9 @@=0A= dsl_dataset_rename_snapshot_sync_impl(dsl_pool_t *dp,=0A= dsl_dataset_t *hds, void *arg)=0A= {=0A= -#ifdef __FreeBSD__=0A= -#ifdef _KERNEL=0A= +#if defined(__FreeBSD__) && defined(_KERNEL)=0A= char *oldname, *newname;=0A= #endif=0A= -#endif=0A= dsl_dataset_rename_snapshot_arg_t *ddrsa =3D arg;=0A= dsl_dataset_t *ds;=0A= uint64_t val;=0A= @@ -1697,25 +1693,25 @@=0A= =0A= VERIFY0(dsl_dataset_snap_remove(hds, ddrsa->ddrsa_oldsnapname, tx));=0A= mutex_enter(&ds->ds_lock);=0A= +#if defined(__FreeBSD__) && defined(_KERNEL)=0A= + oldname =3D kmem_alloc(MAXPATHLEN, KM_SLEEP);=0A= + newname =3D kmem_alloc(MAXPATHLEN, KM_SLEEP);=0A= + dsl_dataset_name(ds, oldname);=0A= +#endif=0A= (void) strcpy(ds->ds_snapname, ddrsa->ddrsa_newsnapname);=0A= +#if defined(__FreeBSD__) && defined(_KERNEL)=0A= + dsl_dataset_name(ds, newname);=0A= +#endif=0A= mutex_exit(&ds->ds_lock);=0A= VERIFY0(zap_add(dp->dp_meta_objset, hds->ds_phys->ds_snapnames_zapobj,=0A= ds->ds_snapname, 8, 1, &ds->ds_object, tx));=0A= =0A= -#ifdef __FreeBSD__=0A= -#ifdef _KERNEL=0A= - oldname =3D kmem_alloc(MAXPATHLEN, KM_SLEEP);=0A= - newname =3D kmem_alloc(MAXPATHLEN, KM_SLEEP);=0A= - snprintf(oldname, MAXPATHLEN, "%s@%s", ddrsa->ddrsa_fsname,=0A= - ddrsa->ddrsa_oldsnapname);=0A= - snprintf(newname, MAXPATHLEN, "%s@%s", ddrsa->ddrsa_fsname,=0A= - ddrsa->ddrsa_newsnapname);=0A= +#if defined(__FreeBSD__) && defined(_KERNEL)=0A= zfsvfs_update_fromname(oldname, newname);=0A= zvol_rename_minors(oldname, newname);=0A= kmem_free(newname, MAXPATHLEN);=0A= kmem_free(oldname, MAXPATHLEN);=0A= #endif=0A= -#endif=0A= dsl_dataset_rele(ds, FTAG);=0A= =0A= return (0);=0A= @@ -1731,9 +1727,14 @@=0A= VERIFY0(dsl_dataset_hold(dp, ddrsa->ddrsa_fsname, FTAG, &hds));=0A= ddrsa->ddrsa_tx =3D tx;=0A= if (ddrsa->ddrsa_recursive) {=0A= + /* Take the spa_namespace_lock so zvol renames don't deadlock */=0A= + mutex_enter(&spa_namespace_lock);=0A= +=0A= VERIFY0(dmu_objset_find_dp(dp, hds->ds_dir->dd_object,=0A= dsl_dataset_rename_snapshot_sync_impl, ddrsa,=0A= DS_FIND_CHILDREN));=0A= +=0A= + mutex_exit(&spa_namespace_lock);=0A= } else {=0A= VERIFY0(dsl_dataset_rename_snapshot_sync_impl(dp, hds, ddrsa));=0A= }=0A= @@ -2031,6 +2032,9 @@=0A= dsl_dir_t *odd =3D NULL;=0A= uint64_t oldnext_obj;=0A= int64_t delta;=0A= +#if defined(__FreeBSD__) && defined(_KERNEL)=0A= + char *oldname, *newname;=0A= +#endif=0A= =0A= VERIFY0(promote_hold(ddpa, dp, FTAG));=0A= hds =3D ddpa->ddpa_clone;=0A= @@ -2096,6 +2100,13 @@=0A= dd->dd_phys->dd_clones, origin_head->ds_object, tx));=0A= }=0A= =0A= +#if defined(__FreeBSD__) && defined(_KERNEL)=0A= + /* Take the spa_namespace_lock so zvol renames don't livelock */=0A= + mutex_enter(&spa_namespace_lock);=0A= +=0A= + oldname =3D kmem_alloc(MAXPATHLEN, KM_SLEEP);=0A= + newname =3D kmem_alloc(MAXPATHLEN, KM_SLEEP);=0A= +#endif=0A= /* move snapshots to this dir */=0A= for (snap =3D list_head(&ddpa->shared_snaps); snap;=0A= snap =3D list_next(&ddpa->shared_snaps, snap)) {=0A= @@ -2115,6 +2126,9 @@=0A= VERIFY0(dsl_dataset_get_snapname(ds));=0A= VERIFY0(dsl_dataset_snap_remove(origin_head,=0A= ds->ds_snapname, tx));=0A= +#if defined(__FreeBSD__) && defined(_KERNEL)=0A= + dsl_dataset_name(ds, oldname);=0A= +#endif=0A= VERIFY0(zap_add(dp->dp_meta_objset,=0A= hds->ds_phys->ds_snapnames_zapobj, ds->ds_snapname,=0A= 8, 1, &ds->ds_object, tx));=0A= @@ -2127,6 +2141,11 @@=0A= dsl_dir_rele(ds->ds_dir, ds);=0A= VERIFY0(dsl_dir_hold_obj(dp, dd->dd_object,=0A= NULL, ds, &ds->ds_dir));=0A= +#if defined(__FreeBSD__) && defined(_KERNEL)=0A= + dsl_dataset_name(ds, newname);=0A= + zfsvfs_update_fromname(oldname, newname);=0A= + zvol_rename_minors(oldname, newname);=0A= +#endif=0A= =0A= /* move any clone references */=0A= if (ds->ds_phys->ds_next_clones_obj &&=0A= @@ -2165,6 +2184,12 @@=0A= ASSERT(!dsl_prop_hascb(ds));=0A= }=0A= =0A= +#if defined(__FreeBSD__) && defined(_KERNEL)=0A= + mutex_exit(&spa_namespace_lock);=0A= +=0A= + kmem_free(newname, MAXPATHLEN);=0A= + kmem_free(oldname, MAXPATHLEN);=0A= +#endif=0A= /*=0A= * Change space accounting.=0A= * Note, pa->*usedsnap and dd_used_breakdown[SNAP] will either=0A= Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zvol.c=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zvol.c (revision = 253623)=0A= +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zvol.c (working copy)=0A= @@ -2302,9 +2302,10 @@=0A= if (dmu_objset_type(os) =3D=3D DMU_OST_ZVOL) {=0A= dsl_dataset_long_hold(os->os_dsl_dataset, FTAG);=0A= dsl_pool_rele(dmu_objset_pool(os), FTAG);=0A= - if ((error =3D zvol_create_minor(name)) =3D=3D 0)=0A= + error =3D zvol_create_minor(name);=0A= + if (error =3D=3D 0 || error =3D=3D EEXIST) {=0A= error =3D zvol_create_snapshots(os, name);=0A= - else {=0A= + } else {=0A= printf("ZFS WARNING: Unable to create ZVOL %s (error=3D%d).\n",=0A= name, error);=0A= }=0A= @@ -2387,12 +2388,16 @@=0A= size_t oldnamelen, newnamelen;=0A= zvol_state_t *zv;=0A= char *namebuf;=0A= + boolean_t locked =3D B_FALSE;=0A= =0A= oldnamelen =3D strlen(oldname);=0A= newnamelen =3D strlen(newname);=0A= =0A= DROP_GIANT();=0A= - mutex_enter(&spa_namespace_lock);=0A= + if (!MUTEX_HELD(&spa_namespace_lock)) {=0A= + mutex_enter(&spa_namespace_lock);=0A= + locked =3D B_TRUE;=0A= + }=0A= g_topology_lock();=0A= =0A= LIST_FOREACH(gp, &zfs_zvol_class.geom, geom) {=0A= @@ -2415,6 +2420,7 @@=0A= }=0A= =0A= g_topology_unlock();=0A= - mutex_exit(&spa_namespace_lock);=0A= + if (locked)=0A= + mutex_exit(&spa_namespace_lock);=0A= PICKUP_GIANT();=0A= }=0A= ------=_NextPart_000_0FCA_01CEA594.BA51F800-- From owner-zfs-devel@FreeBSD.ORG Fri Aug 30 21:29:35 2013 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id BAE957A2 for ; Fri, 30 Aug 2013 21:29:35 +0000 (UTC) (envelope-from prvs=195486f407=killing@multiplay.co.uk) Received: from mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) (using TLSv1 with cipher RC4-MD5 (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 59D272094 for ; Fri, 30 Aug 2013 21:29:34 +0000 (UTC) Received: from r2d2 ([82.69.141.170]) by mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) (MDaemon PRO v10.0.4) with ESMTP id md50005795106.msg for ; Fri, 30 Aug 2013 22:29:25 +0100 X-Spam-Processed: mail1.multiplay.co.uk, Fri, 30 Aug 2013 22:29:25 +0100 (not processed: message from valid local sender) X-MDDKIM-Result: neutral (mail1.multiplay.co.uk) X-MDRemoteIP: 82.69.141.170 X-Return-Path: prvs=195486f407=killing@multiplay.co.uk X-Envelope-From: killing@multiplay.co.uk X-MDaemon-Deliver-To: zfs-devel@FreeBSD.org Message-ID: From: "Steven Hartland" To: Subject: STF version available compatible with HEAD? Date: Fri, 30 Aug 2013 22:30:05 +0100 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="iso-8859-1"; reply-type=original Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.5931 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.6157 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 30 Aug 2013 21:29:35 -0000 I raised this on another thread, but not had any replies so creating a dedicated one so it doesn't get missed. Is there any chance of getting a version of ZFS STF which is compatible with HEAD? Regards Steve ================================================ This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmaster@multiplay.co.uk. From owner-zfs-devel@FreeBSD.ORG Fri Aug 30 21:35:46 2013 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 678B98CD for ; Fri, 30 Aug 2013 21:35:46 +0000 (UTC) (envelope-from will@firepipe.net) Received: from mail-ve0-f174.google.com (mail-ve0-f174.google.com [209.85.128.174]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 2919A20FF for ; Fri, 30 Aug 2013 21:35:45 +0000 (UTC) Received: by mail-ve0-f174.google.com with SMTP id d10so1734464vea.33 for ; Fri, 30 Aug 2013 14:35:44 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=X3NUSBqwYq27pFuP8eGBeRJP1KXTkmybBQBHrcAFcBM=; b=VaanMxbc8Cq/PNdF4y6/3hPcSJMGjn/rMVRDHkYKgz/CRTYIJdYv0j8NV7HKhRXQpx RDumcwd8bNeAiEa6VDdZ60iU7a7HgGBeEC3GYNGeuVAmYgwDr1m+PUIGAFD8ouhXg23S GcjC92o5MJCGTWgtieark1zehTKKFxoOXZgNKOBtB9cUTnCmwoAT3l+wnqzhSILWEuPo 9PnLj0yrrRbXYAMrP8ODZSPKLfHik1WGf1+7pWz9v+7fK8KdyL8/lzJcvzykRd+9K5pK BZjrOluEIgw5uGhYsYKfKYYW1UbY62/C9A/2kAd2I4OScyLcC18Oa+I8mMz3/RzYVTou G40A== X-Gm-Message-State: ALoCoQkr77T6edBdD98i+kiQ729bQONU65FdQgNX46cETyYf3TaKp73Cm+t/0oU7QpokQgctkiZM MIME-Version: 1.0 X-Received: by 10.220.164.202 with SMTP id f10mr3453975vcy.25.1377898544767; Fri, 30 Aug 2013 14:35:44 -0700 (PDT) Received: by 10.58.226.66 with HTTP; Fri, 30 Aug 2013 14:35:44 -0700 (PDT) In-Reply-To: References: Date: Fri, 30 Aug 2013 15:35:44 -0600 Message-ID: Subject: Re: STF version available compatible with HEAD? From: Will Andrews To: Steven Hartland Content-Type: text/plain; charset=UTF-8 Cc: "zfs-devel@freebsd.org" X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 30 Aug 2013 21:35:46 -0000 On Fri, Aug 30, 2013 at 3:30 PM, Steven Hartland wrote: > I raised this on another thread, but not had any replies so creating a > dedicated one so it doesn't get missed. > > Is there any chance of getting a version of ZFS STF which is > compatible with HEAD? Hi Steve, I worked on this issue earlier this week. It turns out that the problem was caused by HEAD switching from fmake to bmake, which doesn't fully support the Solaris build system used. It still works fine on stable/9. I submitted a fix for one issue to Simon Gerraty, which has been committed to NetBSD: http://cvsweb.netbsd.org/bsdweb.cgi/src/usr.bin/make/parse.c However, there are other bmake incompatibilities that I haven't been able to sort out yet. That said, my plan is to: (0) Fix builds on HEAD (1) Check in the original Solaris Test Collection from OpenSolaris to /vendor/opensolaris (2) Branch from there to /base/head/cddl/tools/regression/stc (3) Commit the current version we have at Spectra Logic. (4) Push/pull updates as they occur. If anyone wishes to help, I'm happy to tar up what we have right now. --Will. From owner-zfs-devel@FreeBSD.ORG Fri Aug 30 22:10:31 2013 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 6FB49248 for ; Fri, 30 Aug 2013 22:10:31 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-we0-x22b.google.com (mail-we0-x22b.google.com [IPv6:2a00:1450:400c:c03::22b]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 074022317 for ; Fri, 30 Aug 2013 22:10:30 +0000 (UTC) Received: by mail-we0-f171.google.com with SMTP id p57so2052012wes.16 for ; Fri, 30 Aug 2013 15:10:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=aeOi0c34MDDXsconVv9+3GwBrrn1PBRqHuOHjph6cTk=; b=J7SSknw0iLp2SarnWll1A6n0hSSuW9Ay15PGvqLdFH3vD6+DmIgMcM+j2tFoObP7Wt UdZqbJHPudhVVo9K+un18vhL4Ithd2tqnb8qauELrIR5VRwPcakkNmzTZ4F+MxuYM/9T PyI5uKsqjgbkkDuxOk9+2eXmhWflDReCrLZIovQxaswZDmDS6aqpEwYn7QG6nfXPfveJ IxmMwz/9PEMWBgU/PVrZ4lbZcyHu63EKXKkT7njdNQvJZ0gUrxjrpLTM8OdGVeQ/eiNW u2wDLhd2X9dW8QKUIJgw+Pd3h8rC49mKbYvmCRo1APtv/TAvUukRQLG4nSsgadNa+8Of JQgg== MIME-Version: 1.0 X-Received: by 10.180.198.44 with SMTP id iz12mr4181997wic.32.1377900629349; Fri, 30 Aug 2013 15:10:29 -0700 (PDT) Sender: asomers@gmail.com Received: by 10.194.171.35 with HTTP; Fri, 30 Aug 2013 15:10:29 -0700 (PDT) In-Reply-To: References: Date: Fri, 30 Aug 2013 16:10:29 -0600 X-Google-Sender-Auth: jLOLXhl2CmYrE_WndrpaiFCCRuU Message-ID: Subject: Re: STF version available compatible with HEAD? From: Alan Somers To: Will Andrews Content-Type: text/plain; charset=ISO-8859-1 Cc: "zfs-devel@freebsd.org" X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 30 Aug 2013 22:10:31 -0000 On Fri, Aug 30, 2013 at 3:35 PM, Will Andrews wrote: > On Fri, Aug 30, 2013 at 3:30 PM, Steven Hartland > wrote: >> I raised this on another thread, but not had any replies so creating a >> dedicated one so it doesn't get missed. >> >> Is there any chance of getting a version of ZFS STF which is >> compatible with HEAD? > > Hi Steve, > > I worked on this issue earlier this week. It turns out that the > problem was caused by HEAD switching from fmake to bmake, which > doesn't fully support the Solaris build system used. It still works > fine on stable/9. > > I submitted a fix for one issue to Simon Gerraty, which has been > committed to NetBSD: > http://cvsweb.netbsd.org/bsdweb.cgi/src/usr.bin/make/parse.c > > However, there are other bmake incompatibilities that I haven't been > able to sort out yet. > > That said, my plan is to: > (0) Fix builds on HEAD > (1) Check in the original Solaris Test Collection from OpenSolaris to > /vendor/opensolaris > (2) Branch from there to /base/head/cddl/tools/regression/stc > (3) Commit the current version we have at Spectra Logic. > (4) Push/pull updates as they occur. I plan to do a step 5) Port the STF's build system to bsd.test.mk. Then we can ditch the crufty STF build system. I haven't done this yet because Garrett has made dramatic changes in bsd.test.mk and related files between stable/9 and head, and we're only using stable/9 over here. But we'll probably switch to 10 soon. > > If anyone wishes to help, I'm happy to tar up what we have right now. > > --Will. > _______________________________________________ > zfs-devel@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/zfs-devel > To unsubscribe, send any mail to "zfs-devel-unsubscribe@freebsd.org" From owner-zfs-devel@FreeBSD.ORG Sat Aug 31 14:41:32 2013 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id ED5242C8; Sat, 31 Aug 2013 14:41:32 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 183BE255C; Sat, 31 Aug 2013 14:41:31 +0000 (UTC) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id RAA26179; Sat, 31 Aug 2013 17:41:30 +0300 (EEST) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1VFmMk-000DbK-Ab; Sat, 31 Aug 2013 17:41:30 +0300 Message-ID: <52220062.3040301@FreeBSD.org> Date: Sat, 31 Aug 2013 17:40:34 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:17.0) Gecko/20130810 Thunderbird/17.0.8 MIME-Version: 1.0 To: Steven Hartland , zfs-devel@FreeBSD.org Subject: Re: Improving zvol support References: <5CBC43F8D341437D9811A3117615F314@multiplay.co.uk> In-Reply-To: <5CBC43F8D341437D9811A3117615F314@multiplay.co.uk> X-Enigmail-Version: 1.5.1 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 31 Aug 2013 14:41:33 -0000 on 30/08/2013 17:22 Steven Hartland said the following: > There's a few outstanding PR's for zvol issues such as:- > http://www.freebsd.org/cgi/query-pr.cgi?pr=178999 > > I've attached a patch to this PR (also to this email) > which helps clear quite a few issues highlighted by this > PR and by tests in STF. > > The locking side of this patch while not 100% ideal does > prevent the common issues such as zvol renames so I think > worth getting this committing until more time can be spent > on it. > > What do people think? A few general comments: - it's a good idea to describe the changes and possibly to provide a patch per problem - this looks like an svn diff output, for reviewers' convenience please use -x -p options - both I and Spectra Logic have independent patches to zvol code that significantly change locking in it and completely remove spa_namespace_lock problem I have never seen the Spectra's code although I did have a chance. Will Andrews convinced me that their approach is superior to mine and they were willing to upstream their changes. I think that it would be a good idea to take a look at all proposed approaches, select or fuse the best and finally commit something. BTW, the current zvol code has another major GEOM-related bug - it drops and picks up g_topology_lock in zvol_geom_access, which must not be done. P.S. message-id of my proposal is <510E9FFC.6000303@FreeBSD.org>, it was posted to this list. -- Andriy Gapon From owner-zfs-devel@FreeBSD.ORG Sat Aug 31 16:40:41 2013 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 4607EE19; Sat, 31 Aug 2013 16:40:41 +0000 (UTC) (envelope-from prvs=195558002b=killing@multiplay.co.uk) Received: from mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) (using TLSv1 with cipher RC4-MD5 (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id B75C12A5C; Sat, 31 Aug 2013 16:40:40 +0000 (UTC) Received: from r2d2 ([82.69.141.170]) by mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) (MDaemon PRO v10.0.4) with ESMTP id md50005800621.msg; Sat, 31 Aug 2013 17:40:37 +0100 X-Spam-Processed: mail1.multiplay.co.uk, Sat, 31 Aug 2013 17:40:37 +0100 (not processed: message from valid local sender) X-MDDKIM-Result: neutral (mail1.multiplay.co.uk) X-MDRemoteIP: 82.69.141.170 X-Return-Path: prvs=195558002b=killing@multiplay.co.uk X-Envelope-From: killing@multiplay.co.uk Message-ID: From: "Steven Hartland" To: "Andriy Gapon" , References: <5CBC43F8D341437D9811A3117615F314@multiplay.co.uk> <52220062.3040301@FreeBSD.org> Subject: Re: Improving zvol support Date: Sat, 31 Aug 2013 17:41:27 +0100 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="iso-8859-1"; reply-type=original Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.5931 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.6157 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 31 Aug 2013 16:40:41 -0000 ----- Original Message ----- From: "Andriy Gapon" To: "Steven Hartland" ; Sent: Saturday, August 31, 2013 3:40 PM Subject: Re: Improving zvol support > on 30/08/2013 17:22 Steven Hartland said the following: >> There's a few outstanding PR's for zvol issues such as:- >> http://www.freebsd.org/cgi/query-pr.cgi?pr=178999 >> >> I've attached a patch to this PR (also to this email) >> which helps clear quite a few issues highlighted by this >> PR and by tests in STF. >> >> The locking side of this patch while not 100% ideal does >> prevent the common issues such as zvol renames so I think >> worth getting this committing until more time can be spent >> on it. >> >> What do people think? > > A few general comments: > - it's a good idea to describe the changes and possibly to provide a patch per > problem > - this looks like an svn diff output, for reviewers' convenience please use -x > -p options > - both I and Spectra Logic have independent patches to zvol code that > significantly change locking in it and completely remove spa_namespace_lock problem > > I have never seen the Spectra's code although I did have a chance. Will Andrews > convinced me that their approach is superior to mine and they were willing to > upstream their changes. I think that it would be a good idea to take a look at > all proposed approaches, select or fuse the best and finally commit something. > > BTW, the current zvol code has another major GEOM-related bug - it drops and > picks up g_topology_lock in zvol_geom_access, which must not be done. > > P.S. message-id of my proposal is <510E9FFC.6000303@FreeBSD.org>, it was posted > to this list. I think we need a concerted effort to make what everyones working on or has worked on but hasnt committed to the tree more visible. It seems where's a lot of duplication of effort recently, which is just wasteing everyones time. It seems the guys over at Spectra have loads of great fixes and features which are taking significant effort to develop yet are unknown till others like myself look to actually get a fix into the tree which I'm sure everyone would agree is very frustrating. I hence propose that everyone creates a list of features / fixes that they have locally and we start a new thread to collate these so we don't waste valuable time duplicating effort. Regards Steve ================================================ This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmaster@multiplay.co.uk. From owner-zfs-devel@FreeBSD.ORG Mon Sep 2 08:58:39 2013 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id A84C921F; Mon, 2 Sep 2013 08:58:39 +0000 (UTC) (envelope-from prvs=195726abe6=killing@multiplay.co.uk) Received: from mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) (using TLSv1 with cipher RC4-MD5 (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 962522952; Mon, 2 Sep 2013 08:58:38 +0000 (UTC) Received: from r2d2 ([82.69.141.170]) by mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) (MDaemon PRO v10.0.4) with ESMTP id md50005817074.msg; Mon, 02 Sep 2013 09:58:29 +0100 X-Spam-Processed: mail1.multiplay.co.uk, Mon, 02 Sep 2013 09:58:29 +0100 (not processed: message from valid local sender) X-MDDKIM-Result: neutral (mail1.multiplay.co.uk) X-MDRemoteIP: 82.69.141.170 X-Return-Path: prvs=195726abe6=killing@multiplay.co.uk X-Envelope-From: killing@multiplay.co.uk Message-ID: <84622B7475804BD78C56DBC47DA3988D@multiplay.co.uk> From: "Steven Hartland" To: "Xin Li" , "Will Andrews" , , "Matthew Ahrens" , "Justin T. Gibbs" References: <3F20CA2BA01E4B07899B4CB82EB0E220@multiplay.co! .uk> <37A9F30CF2AB4445BFAE1402BA705B9B@multiplay.co.uk> <1A5316AB-4E00-4C2B-B023-86ED6F1E6CC1@FreeBSD.org> <16CCE71F932643ABBC05F800E9956950@multiplay.co.uk> Subject: Re: N-way mirror read speedup in zfsonlinux Date: Mon, 2 Sep 2013 09:58:37 +0100 MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="----=_NextPart_000_0052_01CEA7C2.FE5E9AF0" X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.5931 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.6157 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Sep 2013 08:58:39 -0000 This is a multi-part message in MIME format. ------=_NextPart_000_0052_01CEA7C2.FE5E9AF0 Content-Type: text/plain; format=flowed; charset="iso-8859-1"; reply-type=original Content-Transfer-Encoding: 7bit ----- Original Message ----- From: "Steven Hartland" > I've updated my version with your changes along with extracting our > the mm structure to a simple inline used in both routes. > > It would be good to understand why we cant set the offset in vdev_queue_io as > having call vdev_queue_register_lastoffset multiple times is a bit messy. > > FYI: I'm away for a week now so may not be able to read emails as much as usual > so if there's a delay replying thats why. Latest version of the patch attached. Justin was looking to see if he can make the update of the vq_lastoffset generic without impacting performance, but appart from that there are no outstanding feedback which hasn't been addressed. @Justin any joy? If not, do you think your going to have any time soon to investigate or should I look to commit this as is? In terms of testing we put this through its paces on our steam proxy box last weekend and pushed well over 10TB through it with no issues :) Regards Steve ================================================ This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmaster@multiplay.co.uk. ------=_NextPart_000_0052_01CEA7C2.FE5E9AF0 Content-Type: application/octet-stream; name="zfs-mirror-load.patch" Content-Transfer-Encoding: quoted-printable Content-Disposition: attachment; filename="zfs-mirror-load.patch" Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/vdev.h=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/vdev.h (revision = 254419)=0A= +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/vdev.h (working = copy)=0A= @@ -119,6 +119,9 @@=0A= extern void vdev_queue_fini(vdev_t *vd);=0A= extern zio_t *vdev_queue_io(zio_t *zio);=0A= extern void vdev_queue_io_done(zio_t *zio);=0A= +extern int vdev_queue_length(vdev_t *vd);=0A= +extern uint64_t vdev_queue_lastoffset(vdev_t *vd);=0A= +extern void vdev_queue_register_lastoffset(vdev_t *vd, zio_t *zio);=0A= =0A= extern void vdev_config_dirty(vdev_t *vd);=0A= extern void vdev_config_clean(vdev_t *vd);=0A= Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/vdev_impl.h=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/vdev_impl.h = (revision 254419)=0A= +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/vdev_impl.h = (working copy)=0A= @@ -106,6 +106,7 @@=0A= avl_tree_t vq_pending_tree;=0A= hrtime_t vq_io_complete_ts;=0A= kmutex_t vq_lock;=0A= + uint64_t vq_lastoffset;=0A= };=0A= =0A= /*=0A= @@ -199,7 +200,10 @@=0A= spa_aux_vdev_t *vdev_aux; /* for l2cache vdevs */=0A= zio_t *vdev_probe_zio; /* root of current probe */=0A= vdev_aux_t vdev_label_aux; /* on-disk aux state */=0A= - struct trim_map *vdev_trimmap;=0A= + struct trim_map *vdev_trimmap; /* map on outstanding trims */ =0A= + uint16_t vdev_rotation_rate; /* rotational rate of the media */=0A= +#define VDEV_RATE_UNKNOWN 0=0A= +#define VDEV_RATE_NON_ROTATING 1=0A= =0A= /*=0A= * For DTrace to work in userland (libzpool) context, these fields must=0A= Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_geom.c=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_geom.c (revision = 254419)=0A= +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_geom.c (working = copy)=0A= @@ -42,9 +42,11 @@=0A= * Virtual device vector for GEOM.=0A= */=0A= =0A= +static g_attrchanged_t vdev_geom_attrchanged;=0A= struct g_class zfs_vdev_class =3D {=0A= .name =3D "ZFS::VDEV",=0A= .version =3D G_VERSION,=0A= + .attrchanged =3D vdev_geom_attrchanged,=0A= };=0A= =0A= DECLARE_GEOM_CLASS(zfs_vdev_class, zfs_vdev);=0A= @@ -62,6 +64,34 @@=0A= &vdev_geom_bio_delete_disable, 0, "Disable BIO_DELETE");=0A= =0A= static void=0A= +vdev_geom_set_rotation_rate(vdev_t *vd, struct g_consumer *cp)=0A= +{ =0A= + int error;=0A= + uint16_t rate;=0A= +=0A= + error =3D g_getattr("GEOM::rotation_rate", cp, &rate);=0A= + if (error =3D=3D 0)=0A= + vd->vdev_rotation_rate =3D rate;=0A= + else=0A= + vd->vdev_rotation_rate =3D VDEV_RATE_UNKNOWN;=0A= +}=0A= +=0A= +static void=0A= +vdev_geom_attrchanged(struct g_consumer *cp, const char *attr)=0A= +{=0A= + vdev_t *vd;=0A= +=0A= + vd =3D cp->private;=0A= + if (vd =3D=3D NULL)=0A= + return;=0A= +=0A= + if (strcmp(attr, "GEOM::rotation_rate") =3D=3D 0) {=0A= + vdev_geom_set_rotation_rate(vd, cp);=0A= + return;=0A= + }=0A= +}=0A= +=0A= +static void=0A= vdev_geom_orphan(struct g_consumer *cp)=0A= {=0A= vdev_t *vd;=0A= @@ -678,6 +708,11 @@=0A= vd->vdev_physpath =3D kmem_alloc(bufsize, KM_SLEEP);=0A= snprintf(vd->vdev_physpath, bufsize, "/dev/%s", pp->name);=0A= =0A= + /*=0A= + * Determine the device's rotation rate.=0A= + */=0A= + vdev_geom_set_rotation_rate(vd, cp);=0A= +=0A= return (0);=0A= }=0A= =0A= Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_mirror.c=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_mirror.c = (revision 254419)=0A= +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_mirror.c = (working copy)=0A= @@ -41,27 +41,97 @@=0A= vdev_t *mc_vd;=0A= uint64_t mc_offset;=0A= int mc_error;=0A= + int mc_load;=0A= uint8_t mc_tried;=0A= uint8_t mc_skipped;=0A= uint8_t mc_speculative;=0A= } mirror_child_t;=0A= =0A= typedef struct mirror_map {=0A= + int *mm_preferred;=0A= + int mm_preferred_cnt;=0A= int mm_children;=0A= int mm_replacing;=0A= - int mm_preferred;=0A= int mm_root;=0A= - mirror_child_t mm_child[1];=0A= + mirror_child_t mm_child[];=0A= } mirror_map_t;=0A= =0A= -int vdev_mirror_shift =3D 21;=0A= +static int vdev_mirror_shift =3D 21;=0A= =0A= +SYSCTL_DECL(_vfs_zfs_vdev);=0A= +static SYSCTL_NODE(_vfs_zfs_vdev, OID_AUTO, mirror, CTLFLAG_RD, 0,=0A= + "ZFS VDEV Mirror");=0A= +=0A= +/*=0A= + * The load configuration settings below are tuned by default for=0A= + * the case where all devices are of the same rotational type.=0A= + *=0A= + * If there is a mixture of rotating and non-rotating media, setting=0A= + * non_rotating_seek_inc to 0 may well provide better results as it=0A= + * will direct more reads to the non-rotating vdevs which are more=0A= + * likely to have a higher performance.=0A= + */=0A= +=0A= +/* Rotating media load calculation configuration. */=0A= +static int rotating_inc =3D 0;=0A= +TUNABLE_INT("vfs.zfs.vdev.mirror.rotating_inc", &rotating_inc);=0A= +SYSCTL_INT(_vfs_zfs_vdev_mirror, OID_AUTO, rotating_inc, CTLFLAG_RW,=0A= + &rotating_inc, 0, "Rotating media load increment for non-seeking = I/O's");=0A= +=0A= +static int rotating_seek_inc =3D 5;=0A= +TUNABLE_INT("vfs.zfs.vdev.mirror.rotating_seek_inc", = &rotating_seek_inc);=0A= +SYSCTL_INT(_vfs_zfs_vdev_mirror, OID_AUTO, rotating_seek_inc, = CTLFLAG_RW,=0A= + &rotating_seek_inc, 0, "Rotating media load increment for seeking = I/O's");=0A= +=0A= +static int rotating_seek_offset =3D 1 * 1024 * 1024;=0A= +TUNABLE_INT("vfs.zfs.vdev.mirror.rotating_seek_offset", = &rotating_seek_offset);=0A= +SYSCTL_INT(_vfs_zfs_vdev_mirror, OID_AUTO, rotating_seek_offset, = CTLFLAG_RW,=0A= + &rotating_seek_offset, 0, "Offset in bytes from the last I/O which "=0A= + "triggers a reduced rotating media seek increment");=0A= +=0A= +/* Non-rotating media load calculation configuration. */=0A= +static int non_rotating_inc =3D 0;=0A= +TUNABLE_INT("vfs.zfs.vdev.mirror.non_rotating_inc", &non_rotating_inc);=0A= +SYSCTL_INT(_vfs_zfs_vdev_mirror, OID_AUTO, non_rotating_inc, CTLFLAG_RW,=0A= + &non_rotating_inc, 0,=0A= + "Non-rotating media load increment for non-seeking I/O's");=0A= +=0A= +static int non_rotating_seek_inc =3D 1;=0A= +TUNABLE_INT("vfs.zfs.vdev.mirror.non_rotating_seek_inc",=0A= + &non_rotating_seek_inc);=0A= +SYSCTL_INT(_vfs_zfs_vdev_mirror, OID_AUTO, non_rotating_seek_inc, = CTLFLAG_RW,=0A= + &non_rotating_seek_inc, 0,=0A= + "Non-rotating media load increment for seeking I/O's");=0A= +=0A= +=0A= +static inline size_t=0A= +vdev_mirror_map_size(int children)=0A= +{=0A= + return (offsetof(mirror_map_t, mm_child[children]) +=0A= + sizeof(int) * children);=0A= +}=0A= +=0A= +static inline mirror_map_t *=0A= +vdev_mirror_map_alloc(int children, boolean_t replacing, boolean_t root)=0A= +{=0A= + mirror_map_t *mm;=0A= +=0A= + mm =3D kmem_zalloc(vdev_mirror_map_size(children), KM_SLEEP);=0A= + mm->mm_children =3D children;=0A= + mm->mm_replacing =3D replacing;=0A= + mm->mm_root =3D root;=0A= + mm->mm_preferred =3D (int *)((uintptr_t)mm + =0A= + offsetof(mirror_map_t, mm_child[children]));=0A= +=0A= + return mm;=0A= +}=0A= +=0A= static void=0A= vdev_mirror_map_free(zio_t *zio)=0A= {=0A= mirror_map_t *mm =3D zio->io_vsd;=0A= =0A= - kmem_free(mm, offsetof(mirror_map_t, mm_child[mm->mm_children]));=0A= + kmem_free(mm, vdev_mirror_map_size(mm->mm_children));=0A= }=0A= =0A= static const zio_vsd_ops_t vdev_mirror_vsd_ops =3D {=0A= @@ -69,55 +139,80 @@=0A= zio_vsd_default_cksum_report=0A= };=0A= =0A= +static int=0A= +vdev_mirror_load(mirror_map_t *mm, vdev_t *vd, uint64_t zio_offset)=0A= +{=0A= + uint64_t lastoffset;=0A= + int load;=0A= +=0A= + /* All DVAs have equal weight at the root. */=0A= + if (mm->mm_root =3D=3D B_TRUE)=0A= + return (INT_MAX);=0A= +=0A= + /*=0A= + * We don't return INT_MAX if the device is resilvering i.e.=0A= + * vdev_resilver_txg !=3D 0 as when tested performance was slightly=0A= + * worse overall when resilvering with compared to without.=0A= + */=0A= +=0A= + /* Standard load based on pending queue length. */=0A= + load =3D vdev_queue_length(vd);=0A= + lastoffset =3D vdev_queue_lastoffset(vd);=0A= +=0A= + /* Non-rotating media. */=0A= + if (vd->vdev_rotation_rate =3D=3D VDEV_RATE_NON_ROTATING) {=0A= + if (lastoffset =3D=3D zio_offset)=0A= + return (load + non_rotating_inc);=0A= +=0A= + /*=0A= + * Apply a seek penalty even for non-rotating devices as=0A= + * sequential I/O'a can be aggregated into fewer operations=0A= + * on the device, thus avoiding unnecessary per-command=0A= + * overhead and boosting performance.=0A= + */=0A= + return (load + non_rotating_seek_inc);=0A= + }=0A= +=0A= + /* I/O's which directly follow the last I/O. */=0A= + if (lastoffset =3D=3D zio_offset)=0A= + return (load + rotating_inc);=0A= +=0A= + /*=0A= + * Apply half the seek increment to I/O's within seek offset=0A= + * of the last I/O queued to this vdev as they should incure less=0A= + * of a seek increment.=0A= + */=0A= + if (ABS(lastoffset - zio_offset) < rotating_seek_offset)=0A= + return (load + (rotating_seek_inc / 2));=0A= +=0A= + /* Apply the full seek increment to all other I/O's. */=0A= + return (load + rotating_seek_inc);=0A= +}=0A= +=0A= +=0A= static mirror_map_t *=0A= -vdev_mirror_map_alloc(zio_t *zio)=0A= +vdev_mirror_map_init(zio_t *zio)=0A= {=0A= mirror_map_t *mm =3D NULL;=0A= mirror_child_t *mc;=0A= vdev_t *vd =3D zio->io_vd;=0A= - int c, d;=0A= + int c;=0A= =0A= if (vd =3D=3D NULL) {=0A= dva_t *dva =3D zio->io_bp->blk_dva;=0A= spa_t *spa =3D zio->io_spa;=0A= =0A= - c =3D BP_GET_NDVAS(zio->io_bp);=0A= -=0A= - mm =3D kmem_zalloc(offsetof(mirror_map_t, mm_child[c]), KM_SLEEP);=0A= - mm->mm_children =3D c;=0A= - mm->mm_replacing =3D B_FALSE;=0A= - mm->mm_preferred =3D spa_get_random(c);=0A= - mm->mm_root =3D B_TRUE;=0A= -=0A= - /*=0A= - * Check the other, lower-index DVAs to see if they're on=0A= - * the same vdev as the child we picked. If they are, use=0A= - * them since they are likely to have been allocated from=0A= - * the primary metaslab in use at the time, and hence are=0A= - * more likely to have locality with single-copy data.=0A= - */=0A= - for (c =3D mm->mm_preferred, d =3D c - 1; d >=3D 0; d--) {=0A= - if (DVA_GET_VDEV(&dva[d]) =3D=3D DVA_GET_VDEV(&dva[c]))=0A= - mm->mm_preferred =3D d;=0A= - }=0A= -=0A= + mm =3D vdev_mirror_map_alloc(BP_GET_NDVAS(zio->io_bp), B_FALSE,=0A= + B_TRUE);=0A= for (c =3D 0; c < mm->mm_children; c++) {=0A= mc =3D &mm->mm_child[c];=0A= -=0A= mc->mc_vd =3D vdev_lookup_top(spa, DVA_GET_VDEV(&dva[c]));=0A= mc->mc_offset =3D DVA_GET_OFFSET(&dva[c]);=0A= }=0A= } else {=0A= - c =3D vd->vdev_children;=0A= -=0A= - mm =3D kmem_zalloc(offsetof(mirror_map_t, mm_child[c]), KM_SLEEP);=0A= - mm->mm_children =3D c;=0A= - mm->mm_replacing =3D (vd->vdev_ops =3D=3D &vdev_replacing_ops ||=0A= - vd->vdev_ops =3D=3D &vdev_spare_ops);=0A= - mm->mm_preferred =3D mm->mm_replacing ? 0 :=0A= - (zio->io_offset >> vdev_mirror_shift) % c;=0A= - mm->mm_root =3D B_FALSE;=0A= -=0A= + mm =3D vdev_mirror_map_alloc(vd->vdev_children,=0A= + (vd->vdev_ops =3D=3D &vdev_replacing_ops ||=0A= + vd->vdev_ops =3D=3D &vdev_spare_ops), B_FALSE);=0A= for (c =3D 0; c < mm->mm_children; c++) {=0A= mc =3D &mm->mm_child[c];=0A= mc->mc_vd =3D vd->vdev_child[c];=0A= @@ -209,6 +304,48 @@=0A= }=0A= =0A= /*=0A= + * Check the other, lower-index DVAs to see if they're on the same=0A= + * vdev as the child we picked. If they are, use them since they=0A= + * are likely to have been allocated from the primary metaslab in=0A= + * use at the time, and hence are more likely to have locality with=0A= + * single-copy data.=0A= + */=0A= +static int=0A= +vdev_mirror_dva_select(zio_t *zio, int preferred)=0A= +{=0A= + dva_t *dva =3D zio->io_bp->blk_dva;=0A= + mirror_map_t *mm =3D zio->io_vsd;=0A= + int c;=0A= +=0A= + for (c =3D preferred - 1; c >=3D 0; c--) {=0A= + if (DVA_GET_VDEV(&dva[c]) =3D=3D DVA_GET_VDEV(&dva[preferred]))=0A= + preferred =3D c;=0A= + }=0A= + return (preferred);=0A= +}=0A= +=0A= +static int=0A= +vdev_mirror_preferred_child_randomize(zio_t *zio)=0A= +{=0A= + mirror_map_t *mm =3D zio->io_vsd;=0A= + int p;=0A= +=0A= + if (mm->mm_root =3D=3D B_TRUE) {=0A= + p =3D spa_get_random(mm->mm_preferred_cnt);=0A= + return (vdev_mirror_dva_select(zio, mm->mm_preferred[p]));=0A= + }=0A= +=0A= + /*=0A= + * To ensure we don't always favour the first matching vdev,=0A= + * which could lead to wear leveling issues on SSD's, we=0A= + * use the I/O offset as a sudo random seed into the vdevs=0A= + * which have the lowest load.=0A= + */=0A= + p =3D (zio->io_offset >> vdev_mirror_shift) % mm->mm_preferred_cnt;=0A= + return (mm->mm_preferred[p]);=0A= +}=0A= +=0A= +/*=0A= * Try to find a child whose DTL doesn't contain the block we want to = read.=0A= * If we can't, try the read on any vdev we haven't already tried.=0A= */=0A= @@ -216,9 +353,8 @@=0A= vdev_mirror_child_select(zio_t *zio)=0A= {=0A= mirror_map_t *mm =3D zio->io_vsd;=0A= - mirror_child_t *mc;=0A= uint64_t txg =3D zio->io_txg;=0A= - int i, c;=0A= + int c, lowest_load;=0A= =0A= ASSERT(zio->io_bp =3D=3D NULL || BP_PHYSICAL_BIRTH(zio->io_bp) =3D=3D = txg);=0A= =0A= @@ -227,32 +363,65 @@=0A= * If a child is known to be completely inaccessible (indicated by=0A= * vdev_readable() returning B_FALSE), don't even try.=0A= */=0A= - for (i =3D 0, c =3D mm->mm_preferred; i < mm->mm_children; i++, c++) {=0A= - if (c >=3D mm->mm_children)=0A= - c =3D 0;=0A= + lowest_load =3D INT_MAX;=0A= + mm->mm_preferred_cnt =3D 0;=0A= + for (c =3D 0; c < mm->mm_children; c++) {=0A= + mirror_child_t *mc;=0A= +=0A= mc =3D &mm->mm_child[c];=0A= if (mc->mc_tried || mc->mc_skipped)=0A= continue;=0A= +=0A= if (!vdev_readable(mc->mc_vd)) {=0A= mc->mc_error =3D SET_ERROR(ENXIO);=0A= mc->mc_tried =3D 1; /* don't even try */=0A= mc->mc_skipped =3D 1;=0A= continue;=0A= }=0A= - if (!vdev_dtl_contains(mc->mc_vd, DTL_MISSING, txg, 1))=0A= - return (c);=0A= - mc->mc_error =3D SET_ERROR(ESTALE);=0A= - mc->mc_skipped =3D 1;=0A= - mc->mc_speculative =3D 1;=0A= +=0A= + if (vdev_dtl_contains(mc->mc_vd, DTL_MISSING, txg, 1)) {=0A= + mc->mc_error =3D SET_ERROR(ESTALE);=0A= + mc->mc_skipped =3D 1;=0A= + mc->mc_speculative =3D 1;=0A= + continue;=0A= + }=0A= +=0A= + mc->mc_load =3D vdev_mirror_load(mm, mc->mc_vd, mc->mc_offset);=0A= + if (mc->mc_load > lowest_load)=0A= + continue;=0A= +=0A= + if (mc->mc_load < lowest_load) {=0A= + lowest_load =3D mc->mc_load;=0A= + mm->mm_preferred_cnt =3D 0;=0A= + }=0A= + mm->mm_preferred[mm->mm_preferred_cnt] =3D c;=0A= + mm->mm_preferred_cnt++;=0A= }=0A= =0A= + if (mm->mm_preferred_cnt =3D=3D 1) {=0A= + vdev_queue_register_lastoffset(=0A= + mm->mm_child[mm->mm_preferred[0]].mc_vd, zio);=0A= + return (mm->mm_preferred[0]);=0A= + }=0A= +=0A= + if (mm->mm_preferred_cnt > 1) {=0A= + int c =3D vdev_mirror_preferred_child_randomize(zio);=0A= +=0A= + vdev_queue_register_lastoffset(mm->mm_child[c].mc_vd, zio);=0A= + return (c);=0A= + }=0A= +=0A= /*=0A= * Every device is either missing or has this txg in its DTL.=0A= * Look for any child we haven't already tried before giving up.=0A= */=0A= - for (c =3D 0; c < mm->mm_children; c++)=0A= - if (!mm->mm_child[c].mc_tried)=0A= + for (c =3D 0; c < mm->mm_children; c++) {=0A= + if (!mm->mm_child[c].mc_tried) {=0A= + vdev_queue_register_lastoffset(mm->mm_child[c].mc_vd,=0A= + zio);=0A= return (c);=0A= + }=0A= + }=0A= =0A= /*=0A= * Every child failed. There's no place left to look.=0A= @@ -267,7 +436,7 @@=0A= mirror_child_t *mc;=0A= int c, children;=0A= =0A= - mm =3D vdev_mirror_map_alloc(zio);=0A= + mm =3D vdev_mirror_map_init(zio);=0A= =0A= if (zio->io_type =3D=3D ZIO_TYPE_READ) {=0A= if ((zio->io_flags & ZIO_FLAG_SCRUB) && !mm->mm_replacing) {=0A= Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c = (revision 254419)=0A= +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c (working = copy)=0A= @@ -155,6 +155,8 @@=0A= =0A= avl_create(&vq->vq_pending_tree, vdev_queue_offset_compare,=0A= sizeof (zio_t), offsetof(struct zio, io_offset_node));=0A= +=0A= + vq->vq_lastoffset =3D 0;=0A= }=0A= =0A= void=0A= @@ -446,3 +448,26 @@=0A= =0A= mutex_exit(&vq->vq_lock);=0A= }=0A= +=0A= +/*=0A= + * As these three methods are only used for load calculations we're not = concerned=0A= + * if we get an incorrect value on 32bit platforms due to lack of = vq_lock mutex=0A= + * uses here. Instead we prefer to keep it lock free for the = performance.=0A= + */ =0A= +int=0A= +vdev_queue_length(vdev_t *vd)=0A= +{=0A= + return (avl_numnodes(&vd->vdev_queue.vq_pending_tree));=0A= +}=0A= +=0A= +uint64_t=0A= +vdev_queue_lastoffset(vdev_t *vd)=0A= +{=0A= + return (vd->vdev_queue.vq_lastoffset);=0A= +}=0A= +=0A= +void=0A= +vdev_queue_register_lastoffset(vdev_t *vd, zio_t *zio)=0A= +{=0A= + vd->vdev_queue.vq_lastoffset =3D zio->io_offset + zio->io_size;=0A= +}=0A= Index: sys/geom/geom.h=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/geom/geom.h (revision 254419)=0A= +++ sys/geom/geom.h (working copy)=0A= @@ -270,6 +270,7 @@=0A= int len);=0A= int g_handleattr_int(struct bio *bp, const char *attribute, int val);=0A= int g_handleattr_off_t(struct bio *bp, const char *attribute, off_t = val);=0A= +int g_handleattr_uint16_t(struct bio *bp, const char *attribute, = uint16_t val);=0A= int g_handleattr_str(struct bio *bp, const char *attribute, const char = *str);=0A= struct g_consumer * g_new_consumer(struct g_geom *gp);=0A= struct g_geom * g_new_geomf(struct g_class *mp, const char *fmt, ...)=0A= Index: sys/geom/geom_disk.c=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/geom/geom_disk.c (revision 254419)=0A= +++ sys/geom/geom_disk.c (working copy)=0A= @@ -376,22 +376,25 @@=0A= break;=0A= else if (g_handleattr_str(bp, "GEOM::ident", dp->d_ident))=0A= break;=0A= - else if (g_handleattr(bp, "GEOM::hba_vendor",=0A= - &dp->d_hba_vendor, 2))=0A= + else if (g_handleattr_uint16_t(bp, "GEOM::hba_vendor",=0A= + dp->d_hba_vendor))=0A= break;=0A= - else if (g_handleattr(bp, "GEOM::hba_device",=0A= - &dp->d_hba_device, 2))=0A= + else if (g_handleattr_uint16_t(bp, "GEOM::hba_device",=0A= + dp->d_hba_device))=0A= break;=0A= - else if (g_handleattr(bp, "GEOM::hba_subvendor",=0A= - &dp->d_hba_subvendor, 2))=0A= + else if (g_handleattr_uint16_t(bp, "GEOM::hba_subvendor",=0A= + dp->d_hba_subvendor))=0A= break;=0A= - else if (g_handleattr(bp, "GEOM::hba_subdevice",=0A= - &dp->d_hba_subdevice, 2))=0A= + else if (g_handleattr_uint16_t(bp, "GEOM::hba_subdevice",=0A= + dp->d_hba_subdevice))=0A= break;=0A= else if (!strcmp(bp->bio_attribute, "GEOM::kerneldump"))=0A= g_disk_kerneldump(bp, dp);=0A= else if (!strcmp(bp->bio_attribute, "GEOM::setstate"))=0A= g_disk_setstate(bp, sc);=0A= + else if (g_handleattr_uint16_t(bp, "GEOM::rotation_rate",=0A= + dp->d_rotation_rate))=0A= + break;=0A= else =0A= error =3D ENOIOCTL;=0A= break;=0A= Index: sys/geom/geom_disk.h=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/geom/geom_disk.h (revision 254419)=0A= +++ sys/geom/geom_disk.h (working copy)=0A= @@ -97,6 +97,7 @@=0A= uint16_t d_hba_device;=0A= uint16_t d_hba_subvendor;=0A= uint16_t d_hba_subdevice;=0A= + uint16_t d_rotation_rate;=0A= =0A= /* Fields private to the driver */=0A= void *d_drv1;=0A= @@ -121,7 +122,8 @@=0A= #define DISK_VERSION_01 0x5856105a=0A= #define DISK_VERSION_02 0x5856105b=0A= #define DISK_VERSION_03 0x5856105c=0A= -#define DISK_VERSION DISK_VERSION_03=0A= +#define DISK_VERSION_04 0x5856105d=0A= +#define DISK_VERSION DISK_VERSION_04=0A= =0A= #endif /* _KERNEL */=0A= #endif /* _GEOM_GEOM_DISK_H_ */=0A= Index: sys/geom/geom_subr.c=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/geom/geom_subr.c (revision 254419)=0A= +++ sys/geom/geom_subr.c (working copy)=0A= @@ -949,6 +949,13 @@=0A= }=0A= =0A= int=0A= +g_handleattr_uint16_t(struct bio *bp, const char *attribute, uint16_t = val)=0A= +{=0A= +=0A= + return (g_handleattr(bp, attribute, &val, sizeof val));=0A= +}=0A= +=0A= +int=0A= g_handleattr_off_t(struct bio *bp, const char *attribute, off_t val)=0A= {=0A= =0A= Index: sys/cam/ata/ata_da.c=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/cam/ata/ata_da.c (revision 254419)=0A= +++ sys/cam/ata/ata_da.c (working copy)=0A= @@ -1224,13 +1224,14 @@=0A= snprintf(announce_buf, sizeof(announce_buf),=0A= "kern.cam.ada.%d.write_cache", periph->unit_number);=0A= TUNABLE_INT_FETCH(announce_buf, &softc->write_cache);=0A= + adagetparams(periph, cgd);=0A= + softc->disk =3D disk_alloc();=0A= /* Disable queue sorting for non-rotational media by default. */=0A= - if (cgd->ident_data.media_rotation_rate =3D=3D 1)=0A= + if (cgd->ident_data.media_rotation_rate =3D=3D ATA_RATE_NON_ROTATING)=0A= softc->sort_io_queue =3D 0;=0A= else=0A= softc->sort_io_queue =3D -1;=0A= - adagetparams(periph, cgd);=0A= - softc->disk =3D disk_alloc();=0A= + softc->disk->d_rotation_rate =3D cgd->ident_data.media_rotation_rate;=0A= softc->disk->d_devstat =3D devstat_new_entry(periph->periph_name,=0A= periph->unit_number, softc->params.secsize,=0A= DEVSTAT_ALL_SUPPORTED,=0A= Index: sys/cam/scsi/scsi_all.h=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/cam/scsi/scsi_all.h (revision 254419)=0A= +++ sys/cam/scsi/scsi_all.h (working copy)=0A= @@ -1451,7 +1451,7 @@=0A= u_int8_t page_length[2];=0A= u_int8_t medium_rotation_rate[2];=0A= #define SVPD_BDC_RATE_NOT_REPORTED 0x00=0A= -#define SVPD_BDC_RATE_NONE_ROTATING 0x01=0A= +#define SVPD_BDC_RATE_NON_ROTATING 0x01=0A= u_int8_t reserved1;=0A= u_int8_t nominal_form_factor;=0A= #define SVPD_BDC_FORM_NOT_REPORTED 0x00=0A= Index: sys/cam/scsi/scsi_da.c=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/cam/scsi/scsi_da.c (revision 254419)=0A= +++ sys/cam/scsi/scsi_da.c (working copy)=0A= @@ -3386,9 +3386,17 @@=0A= * Disable queue sorting for non-rotational media=0A= * by default.=0A= */=0A= - if (scsi_2btoul(bdc->medium_rotation_rate) =3D=3D=0A= - SVPD_BDC_RATE_NONE_ROTATING)=0A= + u_int old_rate =3D softc->disk->d_rotation_rate;=0A= + softc->disk->d_rotation_rate =3D=0A= + scsi_2btoul(bdc->medium_rotation_rate);=0A= + if (softc->disk->d_rotation_rate =3D=3D=0A= + SVPD_BDC_RATE_NON_ROTATING) {=0A= softc->sort_io_queue =3D 0;=0A= + }=0A= + if (softc->disk->d_rotation_rate !=3D old_rate) {=0A= + disk_attr_changed(softc->disk,=0A= + "GEOM::rotation_rate", M_NOWAIT);=0A= + }=0A= } else {=0A= int error;=0A= error =3D daerror(done_ccb, CAM_RETRY_SELTO,=0A= @@ -3423,6 +3431,8 @@=0A= ptr =3D (uint16_t *)ata_params;=0A= =0A= if ((csio->ccb_h.status & CAM_STATUS_MASK) =3D=3D CAM_REQ_CMP) {=0A= + uint16_t old_rate;=0A= +=0A= for (i =3D 0; i < sizeof(*ata_params) / 2; i++)=0A= ptr[i] =3D le16toh(ptr[i]);=0A= if (ata_params->support_dsm & ATA_SUPPORT_DSM_TRIM) {=0A= @@ -3437,8 +3447,18 @@=0A= * Disable queue sorting for non-rotational media=0A= * by default.=0A= */=0A= - if (ata_params->media_rotation_rate =3D=3D 1)=0A= + old_rate =3D softc->disk->d_rotation_rate;=0A= + softc->disk->d_rotation_rate =3D=0A= + ata_params->media_rotation_rate;=0A= + if (softc->disk->d_rotation_rate =3D=3D=0A= + ATA_RATE_NON_ROTATING) {=0A= softc->sort_io_queue =3D 0;=0A= + }=0A= +=0A= + if (softc->disk->d_rotation_rate !=3D old_rate) {=0A= + disk_attr_changed(softc->disk,=0A= + "GEOM::rotation_rate", M_NOWAIT);=0A= + }=0A= } else {=0A= int error;=0A= error =3D daerror(done_ccb, CAM_RETRY_SELTO,=0A= ------=_NextPart_000_0052_01CEA7C2.FE5E9AF0-- From owner-zfs-devel@FreeBSD.ORG Tue Sep 3 16:33:25 2013 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 25DCB460 for ; Tue, 3 Sep 2013 16:33:25 +0000 (UTC) (envelope-from will@firepipe.net) Received: from mail-vc0-f169.google.com (mail-vc0-f169.google.com [209.85.220.169]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id DD6A12A71 for ; Tue, 3 Sep 2013 16:33:24 +0000 (UTC) Received: by mail-vc0-f169.google.com with SMTP id ib11so4264773vcb.0 for ; Tue, 03 Sep 2013 09:33:23 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:date:message-id:subject:from:to :content-type; bh=0fM0VhcJ6g19KC/I6SGpz0MYamlyc84oIaTlkgVs0fw=; b=Lh9HEgtMp3/RkHgWN0fn7s8etBxX9/UkZRO1U9uvej9ziYRRMQ7C3+f6yc3jWQ4is6 qPmdKAXEYY5tCYpg7Tj7igY6YCiFO2y5GfuVzk/RH6Lapd+IuGlEVvhGz36zfpcWAm/1 OwRkviCUizU3n71LH26HioeumYwiGx/RycBIfUmlSH5HfbOUo0o48Mi6/38H48xrbeQM MdGVN23G5Wbg67krhd8Cz+8LmDB7sLd5MYMXTPvojPMl8r2xe1JAHpKAQHHXGRQu3WmD 9n2KE4PvRMakkrUw8qtl0pJCe0h9JDFpS9ronxoDAG9FPxWObaw7Rj8kWNuPvbz7UrGw 93ZQ== X-Gm-Message-State: ALoCoQnmzc9TfATmqZl0xtj3Q9ueZjPfDYAEtdAcNtYKprpU2DRfaXM/2O7GbEyvasDrnToOA3Ah MIME-Version: 1.0 X-Received: by 10.52.27.106 with SMTP id s10mr9920085vdg.24.1378226003750; Tue, 03 Sep 2013 09:33:23 -0700 (PDT) Received: by 10.58.226.66 with HTTP; Tue, 3 Sep 2013 09:33:23 -0700 (PDT) Date: Tue, 3 Sep 2013 10:33:23 -0600 Message-ID: Subject: Spectra Logic's ZFS changes From: Will Andrews To: "zfs-devel@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 03 Sep 2013 16:33:25 -0000 Hi, In the past, I have posted Spectra Logic's ZFS work to my GitHub account, primarily for Illumos developers to review. We have a number of changes that are FreeBSD-specific, but the vast majority (as % of diff size) are not. For this reason, my repository is a clone of illumos-gate, and I wrote a "transfer" script that copies changes from FreeBSD & SpectraBSD. The goal was to create baselines for diffs between the three implementations, to ease migration of specific pieces. Some of our changes are finished. Others need a little polish. Still others need more testing or feedback from others to integrate upstream. Many are over a year or two old, mostly because of the time-consuming nature of pushing them upstream, and because product development & bugfixes keep us busy. We are happy to work with others to integrate our changes upstream, where there is interest in doing so. Several of our smaller changes have already been pushed into illumos-gate and merged down to FreeBSD head & stable/9, but there are many more (generally larger & more complex) that have yet to be merged. Here is a list of some of the features & infrastructure changes: * Asynchronous copy-on-write: Instead of synchronously resolving COW faults, handle them with asynchronous reads in the background. This makes it feasible to use large recordsizes without paying much of a performance penalty for partial writes. Which in turn yields ideal results for RAIDZ, which works best with large recordsizes. * Asynchronous DMU: Provide a means of performing DMU calls asynchronously. This was intended primarily to give ZVOLs a chance to handle multiple outstanding I/Os and thus maintain more simultaneous I/O and provide ZFS prefetch with additional context. As part of this project, we also minimized dnode holds performed during DMU I/Os, centralized chunking of DMU I/Os, and got rid of most code duplication in dmu.c. * DMU buffer user eviction improvements: reduces lock order reversal opportunity, improves verifiability of the mechanism's operations. * Clean up ZVOL locking; it now uses the objset/dataset user mechanism to atomically manage ownership and object lifetime, instead of the global spa_namespace_lock. This mechanism is already used for filesystems & snapshots. * Separate ZVOL OS-independent and OS-dependent layers. On FreeBSD, geoms are no longer directly connected to the underlying zvol & objset just by being created and destroyed; they now behave more like Illumos' zvol vfsops. Fix many GEOM-related locking & object lifetime issues by separating control and I/O paths. * Add generalized hooks for the SPA to manage ZVOL device nodes in the appropriate places, instead of #ifdef'd hooks. * FreeBSD ZFS FUID/ACL & system/user flags implementation, to support storing/retrieving & honoring of native CIFS ACLs. * Expansion of event notifications, particularly for pool/dataset events. * Detect & handle device blocksize optimally, already committed to FreeBSD/head by Justin. We gave a presentation at BSDCan 2012 about the Async COW & Async DMU work, as well as some related bits. My GitHub URL for this work is: http://github.com/wca/illumos-gate/ 'master' is a mirror of illumos/illumos-gate/master. The most recent branches are freebsd-20130820-r254568, which is the ZFS/DTrace-only bits from FreeBSD/stable/9 as of r254568. Branched from that is spectrabsd-20130820, which contains the SpectraBSD version of the same files using the same baseline. Both branches were updated as of August 20th. The diffstat summary for freebsd-20130820-r254568 vs spectrabsd-20130820 is: 121 files changed, 10127 insertions(+), 4973 deletions(-). I'm curious: How much divergence from Illumos are other FreeBSD ZFS developers willing to accept, and for how long? I have been assuming that simply committing most of the non-FreeBSD-specific changes to FreeBSD/head would be unacceptable in terms of impact on merging from illumos-gate. :-) Thanks, --Will. From owner-zfs-devel@FreeBSD.ORG Wed Sep 4 01:08:41 2013 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 9B47298B for ; Wed, 4 Sep 2013 01:08:41 +0000 (UTC) (envelope-from araujobsdport@gmail.com) Received: from mail-pb0-x22a.google.com (mail-pb0-x22a.google.com [IPv6:2607:f8b0:400e:c01::22a]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 746792BC0 for ; Wed, 4 Sep 2013 01:08:41 +0000 (UTC) Received: by mail-pb0-f42.google.com with SMTP id un15so6746050pbc.29 for ; Tue, 03 Sep 2013 18:08:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=references:mime-version:in-reply-to:content-type :content-transfer-encoding:message-id:cc:from:subject:date:to; bh=99BsoksUmtpgqE6MBq/S3wsfwQ1s8gNnu2FXKrmLubQ=; b=xNxgmqepRU8yFkEBD+LdVpFq8lDMXAH6yBNm1X7qh2Np/MuxkNaemZNxo6WL9Fw7g1 Mk+2DjJW1cFqUrQPs1T/T0IFxt838TDmuoSDT+/jzXKGKz2tCGQPOY7qZENwl4rYrhlu OWRp3dbwP2XXYUPNcG+2e1UZArGBNgQnBbVBzuaw0rzQhFk2sgTH/X7MPymRmvLSwaF1 lEtaMTANzKh0jKj4TmOfPpIKfxOUosSaXJ68pqfAKz3VkJeIlp9GTRUoisoIvPPu9zSx MKoiN/6EZE+p+Cv40rz8vzpWXKGBwzg42MfHfI0v4QfMEc1y6+njLO9NmwWjCb5/9gij Opfg== X-Received: by 10.68.191.36 with SMTP id gv4mr235287pbc.167.1378256921160; Tue, 03 Sep 2013 18:08:41 -0700 (PDT) Received: from [100.113.127.139] ([27.244.25.131]) by mx.google.com with ESMTPSA id nj9sm25375645pbc.13.1969.12.31.16.00.00 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 03 Sep 2013 18:08:40 -0700 (PDT) References: Mime-Version: 1.0 (1.0) In-Reply-To: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-Id: <76DE5A61-6AB4-4DCC-93BD-00BBC572F355@gmail.com> X-Mailer: iPhone Mail (10B146) From: araujobsdport@gmail.com Subject: Re: Spectra Logic's ZFS changes Date: Wed, 4 Sep 2013 09:08:31 +0800 To: Will Andrews Cc: "zfs-devel@freebsd.org" X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 04 Sep 2013 01:08:41 -0000 Sent from my iPhone On 2013/9/4, at 0:33, Will Andrews wrote: > Hi, > > In the past, I have posted Spectra Logic's ZFS work to my GitHub > account, primarily for Illumos developers to review. > > We have a number of changes that are FreeBSD-specific, but the vast > majority (as % of diff size) are not. For this reason, my repository > is a clone of illumos-gate, and I wrote a "transfer" script that > copies changes from FreeBSD & SpectraBSD. The goal was to create > baselines for diffs between the three implementations, to ease > migration of specific pieces. > > Some of our changes are finished. Others need a little polish. Still > others need more testing or feedback from others to integrate > upstream. Many are over a year or two old, mostly because of the > time-consuming nature of pushing them upstream, and because product > development & bugfixes keep us busy. We are happy to work with others > to integrate our changes upstream, where there is interest in doing > so. > > Several of our smaller changes have already been pushed into > illumos-gate and merged down to FreeBSD head & stable/9, but there are > many more (generally larger & more complex) that have yet to be > merged. > > > Here is a list of some of the features & infrastructure changes: > > * Asynchronous copy-on-write: Instead of synchronously resolving COW > faults, handle them with asynchronous reads in the background. This > makes it feasible to use large recordsizes without paying much of a > performance penalty for partial writes. Which in turn yields ideal > results for RAIDZ, which works best with large recordsizes. > > * Asynchronous DMU: Provide a means of performing DMU calls > asynchronously. This was intended primarily to give ZVOLs a chance to > handle multiple outstanding I/Os and thus maintain more simultaneous > I/O and provide ZFS prefetch with additional context. As part of this > project, we also minimized dnode holds performed during DMU I/Os, > centralized chunking of DMU I/Os, and got rid of most code duplication > in dmu.c. > > * DMU buffer user eviction improvements: reduces lock order reversal > opportunity, improves verifiability of the mechanism's operations. > > * Clean up ZVOL locking; it now uses the objset/dataset user mechanism > to atomically manage ownership and object lifetime, instead of the > global spa_namespace_lock. This mechanism is already used for > filesystems & snapshots. > > * Separate ZVOL OS-independent and OS-dependent layers. On FreeBSD, > geoms are no longer directly connected to the underlying zvol & objset > just by being created and destroyed; they now behave more like > Illumos' zvol vfsops. Fix many GEOM-related locking & object lifetime > issues by separating control and I/O paths. > > * Add generalized hooks for the SPA to manage ZVOL device nodes in the > appropriate places, instead of #ifdef'd hooks. > > * FreeBSD ZFS FUID/ACL & system/user flags implementation, to support > storing/retrieving & honoring of native CIFS ACLs. > > * Expansion of event notifications, particularly for pool/dataset events. > > * Detect & handle device blocksize optimally, already committed to > FreeBSD/head by Justin. > > > We gave a presentation at BSDCan 2012 about the Async COW & Async DMU > work, as well as some related bits. > > My GitHub URL for this work is: http://github.com/wca/illumos-gate/ > > 'master' is a mirror of illumos/illumos-gate/master. > > The most recent branches are freebsd-20130820-r254568, which is the > ZFS/DTrace-only bits from FreeBSD/stable/9 as of r254568. Branched > from that is spectrabsd-20130820, which contains the SpectraBSD > version of the same files using the same baseline. Both branches were > updated as of August 20th. > > The diffstat summary for freebsd-20130820-r254568 vs > spectrabsd-20130820 is: 121 files changed, 10127 insertions(+), 4973 > deletions(-). > > I'm curious: How much divergence from Illumos are other FreeBSD ZFS > developers willing to accept, and for how long? I have been assuming > that simply committing most of the non-FreeBSD-specific changes to > FreeBSD/head would be unacceptable in terms of impact on merging from > illumos-gate. :-) > Amazing work! Thanks so much to share it with us. > Thanks, > --Will. > _______________________________________________ > zfs-devel@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/zfs-devel > To unsubscribe, send any mail to "zfs-devel-unsubscribe@freebsd.org" From owner-zfs-devel@FreeBSD.ORG Sun Sep 15 17:09:14 2013 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 96871F3B for ; Sun, 15 Sep 2013 17:09:14 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id E51682B58 for ; Sun, 15 Sep 2013 17:09:13 +0000 (UTC) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id UAA07937 for ; Sun, 15 Sep 2013 20:09:12 +0300 (EEST) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1VLFot-0009au-VA for zfs-devel@FreeBSD.org; Sun, 15 Sep 2013 20:09:12 +0300 Message-ID: <5235E97F.6050801@FreeBSD.org> Date: Sun, 15 Sep 2013 20:08:15 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:17.0) Gecko/20130810 Thunderbird/17.0.8 MIME-Version: 1.0 To: zfs-devel@FreeBSD.org Subject: 4045 zfs write throttle & i/o scheduler performance work X-Enigmail-Version: 1.5.1 Content-Type: text/plain; charset=X-VIET-VPS Content-Transfer-Encoding: 7bit X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 15 Sep 2013 17:09:14 -0000 I wonder if anyone already has a patch for integrating "4045 zfs write throttle & i/o scheduler performance work" from Illumos to FreeBSD. Or if anyone is working on this. We would like to test this change in our environment as soon as possible. So I could start working on it if needed. BTW, it seems that "3581 spa_zio_taskq[ZIO_TYPE_FREE][ZIO_TASKQ_ISSUE]->tq_lock is piping hot" was not integrated into FreeBSD despite being committed to Illumos back in February. I am not sure if we need the main change from this commit, but, as it is frequent with Illumos commits, it contains some changes to definitions of some macros, enums and structs. So I think it is worth having that change merged, at least partially. That would help with merge the future changing including "4045 zfs write throttle & i/o scheduler performance work". P.S. Do we have a FreeBSD ZFS IRC channel? Should we? -- Andriy Gapon From owner-zfs-devel@FreeBSD.ORG Sun Sep 15 17:38:48 2013 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 1199F408; Sun, 15 Sep 2013 17:38:48 +0000 (UTC) (envelope-from pawel@dawidek.net) Received: from mail.dawidek.net (garage.dawidek.net [91.121.88.72]) by mx1.freebsd.org (Postfix) with ESMTP id CE2422CA2; Sun, 15 Sep 2013 17:38:46 +0000 (UTC) Received: from localhost (89-73-195-149.dynamic.chello.pl [89.73.195.149]) by mail.dawidek.net (Postfix) with ESMTPSA id 9253AF25; Sun, 15 Sep 2013 19:33:10 +0200 (CEST) Date: Sun, 15 Sep 2013 19:38:56 +0200 From: Pawel Jakub Dawidek To: Andriy Gapon Subject: Re: 4045 zfs write throttle & i/o scheduler performance work Message-ID: <20130915173856.GB1421@garage.freebsd.pl> References: <5235E97F.6050801@FreeBSD.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="WhfpMioaduB5tiZL" Content-Disposition: inline In-Reply-To: <5235E97F.6050801@FreeBSD.org> X-OS: FreeBSD 10.0-CURRENT amd64 User-Agent: Mutt/1.5.21 (2010-09-15) Cc: zfs-devel@FreeBSD.org X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 15 Sep 2013 17:38:48 -0000 --WhfpMioaduB5tiZL Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sun, Sep 15, 2013 at 08:08:15PM +0300, Andriy Gapon wrote: > P.S. > Do we have a FreeBSD ZFS IRC channel? Should we? I'd suggest just using the #OpenZFS channel on freenode. --=20 Pawel Jakub Dawidek http://www.wheelsystems.com FreeBSD committer http://www.FreeBSD.org Am I Evil? Yes, I Am! http://mobter.com --WhfpMioaduB5tiZL Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.20 (FreeBSD) iEYEARECAAYFAlI18LAACgkQForvXbEpPzT4QACg1/wL9bcuMgSPwhrnxizmNtON b/sAn0g7yo9GVRBjuXSbDCsLl7Kfe5xS =8GVX -----END PGP SIGNATURE----- --WhfpMioaduB5tiZL-- From owner-zfs-devel@FreeBSD.ORG Wed Sep 18 11:09:55 2013 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 32432B1E for ; Wed, 18 Sep 2013 11:09:55 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 83E1421DE for ; Wed, 18 Sep 2013 11:09:54 +0000 (UTC) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id OAA25178 for ; Wed, 18 Sep 2013 14:09:47 +0300 (EEST) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1VMFdi-000Nmm-Rm for zfs-devel@FreeBSD.org; Wed, 18 Sep 2013 14:09:46 +0300 Message-ID: <523989C2.9070706@FreeBSD.org> Date: Wed, 18 Sep 2013 14:08:50 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:17.0) Gecko/20130810 Thunderbird/17.0.8 MIME-Version: 1.0 To: zfs-devel@FreeBSD.org Subject: r205231 X-Enigmail-Version: 1.5.1 Content-Type: text/plain; charset=X-VIET-VPS Content-Transfer-Encoding: 7bit X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Sep 2013 11:09:55 -0000 Guys, I would like to ask your opinion of r205231, specifically the part that splits the state lists. The change is quite large. I can admit that I do not fully understand it. And it introduces many differences to the upstream code which makes merges quite a bit harder. Unfortunately, Kip is not available for a discussion and I am not sure when he will be. Is there anybody who is in the know about this change? Has anyone besides Kip evaluated significance of its impact on performance? Should we perhaps consider reverting this change for the sake of staying closer to the upstream? Or, conversely, propose a refinement of this change to the upstream? And finally, is there anyone who can help with merging an upstream change that touches several blocks that were modified by this change? Thank you! -- Andriy Gapon From owner-zfs-devel@FreeBSD.ORG Wed Sep 18 13:51:54 2013 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id BC572F1E; Wed, 18 Sep 2013 13:51:54 +0000 (UTC) (envelope-from pawel@dawidek.net) Received: from mail.dawidek.net (garage.dawidek.net [91.121.88.72]) by mx1.freebsd.org (Postfix) with ESMTP id 8501F2BF4; Wed, 18 Sep 2013 13:51:54 +0000 (UTC) Received: from localhost (58.wheelsystems.com [83.12.187.58]) by mail.dawidek.net (Postfix) with ESMTPSA id 0F6277BA; Wed, 18 Sep 2013 15:46:14 +0200 (CEST) Date: Wed, 18 Sep 2013 15:52:12 +0200 From: Pawel Jakub Dawidek To: Andriy Gapon Subject: Re: r205231 Message-ID: <20130918135212.GA1370@garage.freebsd.pl> References: <523989C2.9070706@FreeBSD.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="G4iJoqBmSsgzjUCe" Content-Disposition: inline In-Reply-To: <523989C2.9070706@FreeBSD.org> X-OS: FreeBSD 10.0-CURRENT amd64 User-Agent: Mutt/1.5.21 (2010-09-15) Cc: zfs-devel@FreeBSD.org X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Sep 2013 13:51:54 -0000 --G4iJoqBmSsgzjUCe Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, Sep 18, 2013 at 02:08:50PM +0300, Andriy Gapon wrote: >=20 > Guys, >=20 > I would like to ask your opinion of r205231, specifically the part that s= plits > the state lists. > The change is quite large. I can admit that I do not fully understand it= =2E And > it introduces many differences to the upstream code which makes merges qu= ite a > bit harder. AFAIR Kip observed this contention for workload with multiple sequential streams, like video streaming or something. If it really reduces contention and we can prove it then I think it would be nice to upstream the change. My guess would be that it does help, the hard part might be to find a case where it is clearly visible. I'm also not fun of hardcoding numer of locks to 16. It should scale with the number of cores at least. --=20 Pawel Jakub Dawidek http://www.wheelsystems.com FreeBSD committer http://www.FreeBSD.org Am I Evil? Yes, I Am! http://mobter.com --G4iJoqBmSsgzjUCe Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.20 (FreeBSD) iEYEARECAAYFAlI5sAwACgkQForvXbEpPzRf7gCfSTsIA6Wksbny25XVThR0Z9A3 Ix0An2ZG3320cKndYPER3cKKPhxhucVa =rutL -----END PGP SIGNATURE----- --G4iJoqBmSsgzjUCe-- From owner-zfs-devel@FreeBSD.ORG Wed Sep 18 15:43:49 2013 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 4E7385B9; Wed, 18 Sep 2013 15:43:49 +0000 (UTC) (envelope-from mavbsd@gmail.com) Received: from mail-bk0-f53.google.com (mail-bk0-f53.google.com [209.85.214.53]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id B174D22FE; Wed, 18 Sep 2013 15:43:47 +0000 (UTC) Received: by mail-bk0-f53.google.com with SMTP id d7so2851287bkh.40 for ; Wed, 18 Sep 2013 08:43:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:content-type:content-transfer-encoding; bh=ivpx0Z9CTfRhgqq8rY2pntkU+7OPv164XgFtp+FoR3k=; b=PkxgFxZPZxGCLvsAI/4G7OsVQR8BhJbKcQPhxMWiW8EGVgc61DxKnGePMdFhFqvaad S7/sZXx8miQo2JGKIaiSaoeXVwrdCu1d52dkHs4jEl4N/z8uJpaA9W8LfOd4M3K14P+s qCguSBZiVysDJsPupyji7K+bv4AbR6cyu2HkcSF7bgFf2IblNgk3xpV8UcAlTNtBtfpF wedJonxGSKz/zSwoHWdDpcxTyCxqai7iAQpLJokGy94/Sc2DFrz5Td+t3oKZrud0qLQG SqmuwCOruJaU2GyOCi6qSyB80GKtHFqPYd5b5ZgyXcN23wbsX5//+4ldbaD0PuBfdCCE tD0A== X-Received: by 10.205.68.137 with SMTP id xy9mr12452737bkb.28.1379519010953; Wed, 18 Sep 2013 08:43:30 -0700 (PDT) Received: from mavbook.mavhome.dp.ua ([5.248.115.71]) by mx.google.com with ESMTPSA id l5sm1220018bko.7.1969.12.31.16.00.00 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Wed, 18 Sep 2013 08:43:29 -0700 (PDT) Sender: Alexander Motin Message-ID: <5239CA1F.9010104@FreeBSD.org> Date: Wed, 18 Sep 2013 18:43:27 +0300 From: Alexander Motin User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:17.0) Gecko/20130616 Thunderbird/17.0.6 MIME-Version: 1.0 To: Andriy Gapon Subject: Re: r205231 References: <523989C2.9070706@FreeBSD.org> In-Reply-To: <523989C2.9070706@FreeBSD.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: zfs-devel@FreeBSD.org X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Sep 2013 15:43:49 -0000 On 18.09.2013 14:08, Andriy Gapon wrote: > Guys, > > I would like to ask your opinion of r205231, specifically the part that splits > the state lists. > The change is quite large. I can admit that I do not fully understand it. And > it introduces many differences to the upstream code which makes merges quite a > bit harder. > > Unfortunately, Kip is not available for a discussion and I am not sure when he > will be. > > Is there anybody who is in the know about this change? > Has anyone besides Kip evaluated significance of its impact on performance? > Should we perhaps consider reverting this change for the sake of staying closer > to the upstream? Or, conversely, propose a refinement of this change to the > upstream? Not explicitly about this case and almost flaming, but my internal (non-scientific) feeling is that every lock in ZFS in congested and affects performance. While optimizing block level I was able to reach almost a million IOPS, in no way I can push more then 100K random read disk IOPS through the ZFS. And unless my memory is leaking exactly ARC-related locks were standing on the way according to profiling. If you wish to play with it, we could manage you access to my 1M IOPS iron. -- Alexander Motin From owner-zfs-devel@FreeBSD.ORG Fri Sep 20 18:25:13 2013 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 2AC448B6; Fri, 20 Sep 2013 18:25:13 +0000 (UTC) (envelope-from olivier777a7@gmail.com) Received: from mail-la0-x22d.google.com (mail-la0-x22d.google.com [IPv6:2a00:1450:4010:c03::22d]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 395A128AD; Fri, 20 Sep 2013 18:25:12 +0000 (UTC) Received: by mail-la0-f45.google.com with SMTP id eh20so645907lab.32 for ; Fri, 20 Sep 2013 11:25:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=778fNebCgKXrC2nOvAUX3nG2SC5QyO48uzLZH9egt1Q=; b=EKyrrkboPJSDa1xvDZ+YKMhz4iomlO5S9qlO+rmBkh8nss7cdLlenKSU7kkDlH9EtD F4xgGphc+zmR9PTby9V2fXJ+h1Dy1mVHVWsukWNS+yx1hfV+4pwkG/JLEZS0jLYDVtaN qsq7I9yxgfTvd2xPLaU5C0z+s8hKCkWKuAoIyBB4ymKno+7ttgj06u/pRIRCXtJth3wr InlwLF3/Zccjzei2Wi2FQe+xEUR/SfKlmnaW7x4CAkOKojIHz1BeVYiyhtVS9Z+2pmrV yYiZiPlte6jka0RLvl7UMN3Fc4c7To5wupaPtzs360NRka0/875AsVWklD7tMPQEpIS6 n4hQ== MIME-Version: 1.0 X-Received: by 10.112.51.101 with SMTP id j5mr7309599lbo.17.1379701510213; Fri, 20 Sep 2013 11:25:10 -0700 (PDT) Received: by 10.114.176.69 with HTTP; Fri, 20 Sep 2013 11:25:10 -0700 (PDT) In-Reply-To: <51E944B0.5080409@gmail.com> References: <51E944B0.5080409@gmail.com> Date: Fri, 20 Sep 2013 11:25:10 -0700 Message-ID: Subject: Re: 9.2PRERELEASE ZFS panic in lzjb_compress From: olivier To: Volodymyr Kostyrko Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: "freebsd-stable@freebsd.org" , zfs-devel@freebsd.org X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Sep 2013 18:25:13 -0000 Got another, very similar panic again on recent 9-STABLE (r255602); I assume the latest 9.2 release candidate is affected too. Anybody have any idea of what could be causing this, and of a workaround other than turning compression off? Unlike the last panic I reported, this one did not occur during a zfs send/receive operation. There were just a number of processes potentially writing to disk at the same time. All hardware is healthy as far as I can tell (memory is ECC and no errors in logs; zpool status and smartctl show no problems). Fatal trap 12: page fault while in kernel mode cpuid = 4; apic id = 24 cpuid = 51; apic id = 83 fault virtual address = 0xffffff8700a9cc65 fault virtual address = 0xffffff8700ab0ea9 fault code = supervisor read data, page not present instruction pointer = 0x20:0xffffffff8195ff47 fault code = supervisor read data, page not present stack pointer = 0x28:0xffffffcf951390a0 Fatal trap 12: page fault while in kernel mode frame pointer = 0x28:0xffffffcf951398f0 Fatal trap 12: page fault while in kernel mode code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, long 1, def32 0, gran 1 instruction pointer = 0x20:0xffffffff8195ffa4 stack pointer = 0x28:0xffffffcf951250a0 processor eflags = frame pointer = 0x28:0xffffffcf951258f0 interrupt enabled, code segment = base 0x0, limit 0xfffff, type 0x1b resume, IOPL = 0 cpuid = 28; apic id = 4c Fatal trap 12: page fault while in kernel mode = DPL 0, pres 1, long 1, def32 0, gran 1 current process = 0 (zio_write_issue_hig) processor eflags = fault virtual address = 0xffffff8700aa22ac interrupt enabled, fault code = supervisor read data, page not present resume, IOPL = 0 trap number = 12 instruction pointer = 0x20:0xffffffff8195ffa4 current process = 0 (zio_write_issue_hig) panic: page fault cpuid = 4 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2a/frame 0xffffffcf95138b30 kdb_backtrace() at kdb_backtrace+0x37/frame 0xffffffcf95138bf0 panic() at panic+0x1ce/frame 0xffffffcf95138cf0 trap_fatal() at trap_fatal+0x290/frame 0xffffffcf95138d50 trap_pfault() at trap_pfault+0x211/frame 0xffffffcf95138de0 trap() at trap+0x344/frame 0xffffffcf95138fe0 calltrap() at calltrap+0x8/frame 0xffffffcf95138fe0 --- trap 0xc, rip = 0xffffffff8195ff47, rsp = 0xffffffcf951390a0, rbp = 0xffffffcf951398f0 --- lzjb_compress() at lzjb_compress+0xa7/frame 0xffffffcf951398f0 zio_compress_data() at zio_compress_data+0x92/frame 0xffffffcf95139920 zio_write_bp_init() at zio_write_bp_init+0x24b/frame 0xffffffcf95139970 zio_execute() at zio_execute+0xc3/frame 0xffffffcf951399b0 taskqueue_run_locked() at taskqueue_run_locked+0x74/frame 0xffffffcf95139a00 taskqueue_thread_loop() at taskqueue_thread_loop+0x46/frame 0xffffffcf95139a20 fork_exit() at fork_exit+0x11f/frame 0xffffffcf95139a70 fork_trampoline() at fork_trampoline+0xe/frame 0xffffffcf95139a70 --- trap 0, rip = 0, rsp = 0xffffffcf95139b30, rbp = 0 --- 0x51f47 is in lzjb_compress (/usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/lzjb.c:74). 69 } 70 if (src > (uchar_t *)s_start + s_len - MATCH_MAX) { 71 *dst++ = *src++; 72 continue; 73 } 74 hash = (src[0] << 16) + (src[1] << 8) + src[2]; 75 hash += hash >> 9; 76 hash += hash >> 5; 77 hp = &lempel[hash & (LEMPEL_SIZE - 1)]; 78 offset = (intptr_t)(src - *hp) & OFFSET_MASK; dmesg output is at http://pastebin.com/U34fwJ5f kernel config is at http://pastebin.com/c9HKfcsz I can provide more information if useful. Thanks On Fri, Jul 19, 2013 at 6:52 AM, Volodymyr Kostyrko wrote: > 19.07.2013 07:04, olivier wrote: > >> Hi, >> Running 9.2-PRERELEASE #19 r253313 I got the following panic >> >> Fatal trap 12: page fault while in kernel mode >> cpuid = 22; apic id = 46 >> fault virtual address = 0xffffff827ebca30c >> fault code = supervisor read data, page not present >> instruction pointer = 0x20:0xffffffff81983055 >> stack pointer = 0x28:0xffffffcf75bd60a0 >> frame pointer = 0x28:0xffffffcf75bd68f0 >> code segment = base 0x0, limit 0xfffff, type 0x1b >> = DPL 0, pres 1, long 1, def32 0, gran 1 >> processor eflags = interrupt enabled, resume, IOPL = 0 >> current process = 0 (zio_write_issue_hig) >> trap number = 12 >> panic: page fault >> cpuid = 22 >> KDB: stack backtrace: >> db_trace_self_wrapper() at db_trace_self_wrapper+0x2a/**frame >> 0xffffffcf75bd5b30 >> kdb_backtrace() at kdb_backtrace+0x37/frame 0xffffffcf75bd5bf0 >> panic() at panic+0x1ce/frame 0xffffffcf75bd5cf0 >> trap_fatal() at trap_fatal+0x290/frame 0xffffffcf75bd5d50 >> trap_pfault() at trap_pfault+0x211/frame 0xffffffcf75bd5de0 >> trap() at trap+0x344/frame 0xffffffcf75bd5fe0 >> calltrap() at calltrap+0x8/frame 0xffffffcf75bd5fe0 >> --- trap 0xc, rip = 0xffffffff81983055, rsp = 0xffffffcf75bd60a0, rbp = >> 0xffffffcf75bd68f0 --- >> lzjb_compress() at lzjb_compress+0x185/frame 0xffffffcf75bd68f0 >> zio_compress_data() at zio_compress_data+0x92/frame 0xffffffcf75bd6920 >> zio_write_bp_init() at zio_write_bp_init+0x24b/frame 0xffffffcf75bd6970 >> zio_execute() at zio_execute+0xc3/frame 0xffffffcf75bd69b0 >> taskqueue_run_locked() at taskqueue_run_locked+0x74/**frame >> 0xffffffcf75bd6a00 >> taskqueue_thread_loop() at taskqueue_thread_loop+0x46/**frame >> 0xffffffcf75bd6a20 >> fork_exit() at fork_exit+0x11f/frame 0xffffffcf75bd6a70 >> fork_trampoline() at fork_trampoline+0xe/frame 0xffffffcf75bd6a70 >> --- trap 0, rip = 0, rsp = 0xffffffcf75bd6b30, rbp = 0 --- >> >> lzjb_compress+0x185 corresponds to line 85 in >> 80 cpy = src - offset; >> 81 if (cpy >= (uchar_t *)s_start && cpy != src && >> 82 src[0] == cpy[0] && src[1] == cpy[1] && src[2] == cpy[2]) { >> 83 *copymap |= copymask; >> 84 for (mlen = MATCH_MIN; mlen < MATCH_MAX; mlen++) >> 85 if (src[mlen] != cpy[mlen]) >> 86 break; >> 87 *dst++ = ((mlen - MATCH_MIN) << (NBBY - MATCH_BITS)) | >> 88 (offset >> NBBY); >> 89 *dst++ = (uchar_t)offset; >> >> I think it's the first time I've seen this panic. It happened while doing >> a >> send/receive. I have two pools with lzjb compression; I don't know which >> of >> these pools caused the problem, but one of them was the source of the >> send/receive. >> >> I only have a textdump but I'm happy to try to provide more information >> that could help anyone look into this. >> Thanks >> Olivier >> > > Oh, I can add to this one. I have a full core dump of the same problem > caused by copying large set of files from lzjb compressed pool to lz4 > compressed pool. vfs.zfs.recover was set. > > #1 0xffffffff8039d954 in kern_reboot (howto=260) > at /usr/src/sys/kern/kern_**shutdown.c:449 > #2 0xffffffff8039ddce in panic (fmt=) > at /usr/src/sys/kern/kern_**shutdown.c:637 > #3 0xffffffff80620a6a in trap_fatal (frame=, > eva=) at /usr/src/sys/amd64/amd64/trap.**c:879 > #4 0xffffffff80620d25 in trap_pfault (frame=0x0, usermode=0) > at /usr/src/sys/amd64/amd64/trap.**c:700 > #5 0xffffffff806204f6 in trap (frame=0xffffff821ca43600) > at /usr/src/sys/amd64/amd64/trap.**c:463 > #6 0xffffffff8060a032 in calltrap () > at /usr/src/sys/amd64/amd64/**exception.S:232 > #7 0xffffffff805a9367 in vm_page_alloc (object=0xffffffff80a34030, > pindex=16633, req=97) at /usr/src/sys/vm/vm_page.c:1445 > #8 0xffffffff8059c42e in kmem_back (map=0xfffffe00010000e8, > addr=18446743524021862400, size=16384, flags=) > at /usr/src/sys/vm/vm_kern.c:362 > #9 0xffffffff8059c2ac in kmem_malloc (map=0xfffffe00010000e8, size=16384, > flags=257) at /usr/src/sys/vm/vm_kern.c:313 > #10 0xffffffff80595104 in uma_large_malloc (size=, > wait=257) at /usr/src/sys/vm/uma_core.c:994 > #11 0xffffffff80386b80 in malloc (size=16384, mtp=0xffffffff80ea7c40, > flags=0) > at /usr/src/sys/kern/kern_malloc.**c:492 > #12 0xffffffff80c9e13c in lz4_compress (s_start=0xffffff80d0b19000, > d_start=0xffffff8159445000, s_len=131072, d_len=114688, n=-2) > at /usr/src/sys/modules/zfs/../..**/cddl/contrib/opensolaris/uts/** > common/fs/zfs/lz4.c:843 > #13 0xffffffff80cdde25 in zio_compress_data (c=, > src=, dst=0xffffff8159445000, s_len=131072) > at /usr/src/sys/modules/zfs/../..**/cddl/contrib/opensolaris/uts/** > common/fs/zfs/zio_compress.c:**109 > #14 0xffffffff80cda012 in zio_write_bp_init (zio=0xfffffe0143a12000) > at /usr/src/sys/modules/zfs/../..**/cddl/contrib/opensolaris/uts/** > common/fs/zfs/zio.c:1107 > #15 0xffffffff80cd8ec6 in zio_execute (zio=0xfffffe0143a12000) > at /usr/src/sys/modules/zfs/../..**/cddl/contrib/opensolaris/uts/** > common/fs/zfs/zio.c:1305 > #16 0xffffffff803e25e6 in taskqueue_run_locked (queue=0xfffffe00060ca300) > at /usr/src/sys/kern/subr_**taskqueue.c:312 > #17 0xffffffff803e2e38 in taskqueue_thread_loop (arg=) > at /usr/src/sys/kern/subr_**taskqueue.c:501 > #18 0xffffffff8036f40a in fork_exit ( > callout=0xffffffff803e2da0 , > arg=0xfffffe00060cc3d0, frame=0xffffff821ca43a80) > at /usr/src/sys/kern/kern_fork.c:**988 > #19 0xffffffff8060a56e in fork_trampoline () > at /usr/src/sys/amd64/amd64/**exception.S:606 > > I have a full crash dump in case someone wants to look at it. > > -- > Sphinx of black quartz, judge my vow. > From owner-zfs-devel@FreeBSD.ORG Fri Sep 20 21:32:11 2013 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id C2AE4E5; Fri, 20 Sep 2013 21:32:11 +0000 (UTC) (envelope-from olivier777a7@gmail.com) Received: from mail-la0-x22d.google.com (mail-la0-x22d.google.com [IPv6:2a00:1450:4010:c03::22d]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id CF8512367; Fri, 20 Sep 2013 21:32:10 +0000 (UTC) Received: by mail-la0-f45.google.com with SMTP id eh20so789660lab.4 for ; Fri, 20 Sep 2013 14:32:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=MaQm1gYA+dE/PL4YzHmaT8RlLpJrguEVQw4LCaopWOk=; b=LsKZbLxGq9R/b0YbSvug5i2HTd1TR8vsvRChHOhEBxeRgjbcBU141GeHzx4insjLmg shVkQqaSmed1LjmNo8IhrtGdE+LPdegcfwa4SSpfplkcWIONknuoDEptrqYLwK4Ob+7l umDfpHd4Peiuzyakrk2ngn2HDnKi01Qg1ffg5va8fVnVXwdiZQEp5JmYFsBoepZSWmW7 +GBSqtcwkFYLGw50TYyXyuexZvLgCULp7GT5/BGhT1ZCK5yr+UNVA6ZyKq6kIYhKVozR vE01RIVNvlgsToYYRdZy3b2YEJCcKtMnOwzvm/vyo5m3rHLeGvrwnV0Fje5zxvtIIghf ILsg== MIME-Version: 1.0 X-Received: by 10.152.36.98 with SMTP id p2mr7723721laj.14.1379712728547; Fri, 20 Sep 2013 14:32:08 -0700 (PDT) Received: by 10.114.176.69 with HTTP; Fri, 20 Sep 2013 14:32:08 -0700 (PDT) In-Reply-To: References: <51E944B0.5080409@gmail.com> Date: Fri, 20 Sep 2013 14:32:08 -0700 Message-ID: Subject: Re: 9.2PRERELEASE ZFS panic in lzjb_compress From: olivier To: Volodymyr Kostyrko Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: "freebsd-stable@freebsd.org" , zfs-devel@freebsd.org X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Sep 2013 21:32:11 -0000 One last piece of information I just got: the problem is not specific to LZJB compression. I switched to LZ4 and get the same sort of panic: Fatal trap 12: page fault while in kernel mode cpuid = 8; apic id = 28 fault virtual address = 0xffffff8581c48000 fault code = supervisor read data, page not present instruction pointer = 0x20:0xffffffff8195f6d1 stack pointer = 0x28:0xffffffcf950ee850 frame pointer = 0x28:0xffffffcf950ee8f0 code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags = interrupt enabled, resume, IOPL = 0 current process = 0 (zio_write_issue_hig) trap number = 12 panic: page fault cpuid = 8 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2a/frame 0xffffffcf950ee2e0 kdb_backtrace() at kdb_backtrace+0x37/frame 0xffffffcf950ee3a0 panic() at panic+0x1ce/frame 0xffffffcf950ee4a0 trap_fatal() at trap_fatal+0x290/frame 0xffffffcf950ee500 trap_pfault() at trap_pfault+0x211/frame 0xffffffcf950ee590 trap() at trap+0x344/frame 0xffffffcf950ee790 calltrap() at calltrap+0x8/frame 0xffffffcf950ee790 --- trap 0xc, rip = 0xffffffff8195f6d1, rsp = 0xffffffcf950ee850, rbp = 0xffffffcf950ee8f0 --- lz4_compress() at lz4_compress+0x81/frame 0xffffffcf950ee8f0 zio_compress_data() at zio_compress_data+0x92/frame 0xffffffcf950ee920 zio_write_bp_init() at zio_write_bp_init+0x24b/frame 0xffffffcf950ee970 zio_execute() at zio_execute+0xc3/frame 0xffffffcf950ee9b0 taskqueue_run_locked() at taskqueue_run_locked+0x74/frame 0xffffffcf950eea00 taskqueue_thread_loop() at taskqueue_thread_loop+0x46/frame 0xffffffcf950eea20 fork_exit() at fork_exit+0x11f/frame 0xffffffcf950eea70 fork_trampoline() at fork_trampoline+0xe/frame 0xffffffcf950eea70 --- trap 0, rip = 0, rsp = 0xffffffcf950eeb30, rbp = 0 --- (I am now trying without any compression.) On Fri, Sep 20, 2013 at 11:25 AM, olivier wrote: > Got another, very similar panic again on recent 9-STABLE (r255602); I > assume the latest 9.2 release candidate is affected too. Anybody have any > idea of what could be causing this, and of a workaround other than turning > compression off? > Unlike the last panic I reported, this one did not occur during a zfs > send/receive operation. There were just a number of processes potentially > writing to disk at the same time. > All hardware is healthy as far as I can tell (memory is ECC and no errors > in logs; zpool status and smartctl show no problems). > > Fatal trap 12: page fault while in kernel mode > > > cpuid = 4; apic id = 24 > cpuid = 51; apic id = 83 > fault virtual address = 0xffffff8700a9cc65 > fault virtual address = 0xffffff8700ab0ea9 > fault code = supervisor read data, page not present > > instruction pointer = 0x20:0xffffffff8195ff47 > fault code = supervisor read data, page not present > stack pointer = 0x28:0xffffffcf951390a0 > Fatal trap 12: page fault while in kernel mode > frame pointer = 0x28:0xffffffcf951398f0 > Fatal trap 12: page fault while in kernel mode > code segment = base 0x0, limit 0xfffff, type 0x1b > = DPL 0, pres 1, long 1, def32 0, gran 1 > instruction pointer = 0x20:0xffffffff8195ffa4 > stack pointer = 0x28:0xffffffcf951250a0 > processor eflags = frame pointer = 0x28:0xffffffcf951258f0 > interrupt enabled, code segment = base 0x0, limit 0xfffff, type 0x1b > > resume, IOPL = 0 > cpuid = 28; apic id = 4c > Fatal trap 12: page fault while in kernel mode > = DPL 0, pres 1, long 1, def32 0, gran 1 > current process = 0 (zio_write_issue_hig) > processor eflags = fault virtual address = 0xffffff8700aa22ac > interrupt enabled, fault code = supervisor read data, page not present > resume, IOPL = 0 > trap number = 12 > instruction pointer = 0x20:0xffffffff8195ffa4 > current process = 0 (zio_write_issue_hig) > panic: page fault > cpuid = 4 > KDB: stack backtrace: > db_trace_self_wrapper() at db_trace_self_wrapper+0x2a/frame > 0xffffffcf95138b30 > kdb_backtrace() at kdb_backtrace+0x37/frame 0xffffffcf95138bf0 > panic() at panic+0x1ce/frame 0xffffffcf95138cf0 > trap_fatal() at trap_fatal+0x290/frame 0xffffffcf95138d50 > trap_pfault() at trap_pfault+0x211/frame 0xffffffcf95138de0 > trap() at trap+0x344/frame 0xffffffcf95138fe0 > calltrap() at calltrap+0x8/frame 0xffffffcf95138fe0 > --- trap 0xc, rip = 0xffffffff8195ff47, rsp = 0xffffffcf951390a0, rbp = > 0xffffffcf951398f0 --- > lzjb_compress() at lzjb_compress+0xa7/frame 0xffffffcf951398f0 > zio_compress_data() at zio_compress_data+0x92/frame 0xffffffcf95139920 > zio_write_bp_init() at zio_write_bp_init+0x24b/frame 0xffffffcf95139970 > zio_execute() at zio_execute+0xc3/frame 0xffffffcf951399b0 > taskqueue_run_locked() at taskqueue_run_locked+0x74/frame > 0xffffffcf95139a00 > taskqueue_thread_loop() at taskqueue_thread_loop+0x46/frame > 0xffffffcf95139a20 > fork_exit() at fork_exit+0x11f/frame 0xffffffcf95139a70 > fork_trampoline() at fork_trampoline+0xe/frame 0xffffffcf95139a70 > --- trap 0, rip = 0, rsp = 0xffffffcf95139b30, rbp = 0 --- > > > 0x51f47 is in lzjb_compress > (/usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/lzjb.c:74). > 69 } > 70 if (src > (uchar_t *)s_start + s_len - MATCH_MAX) { > 71 *dst++ = *src++; > 72 continue; > 73 } > 74 hash = (src[0] << 16) + (src[1] << 8) + src[2]; > 75 hash += hash >> 9; > 76 hash += hash >> 5; > 77 hp = &lempel[hash & (LEMPEL_SIZE - 1)]; > 78 offset = (intptr_t)(src - *hp) & OFFSET_MASK; > > dmesg output is at http://pastebin.com/U34fwJ5f > kernel config is at http://pastebin.com/c9HKfcsz > I can provide more information if useful. > Thanks > > > On Fri, Jul 19, 2013 at 6:52 AM, Volodymyr Kostyrko wrote: > >> 19.07.2013 07:04, olivier wrote: >> >>> Hi, >>> Running 9.2-PRERELEASE #19 r253313 I got the following panic >>> >>> Fatal trap 12: page fault while in kernel mode >>> cpuid = 22; apic id = 46 >>> fault virtual address = 0xffffff827ebca30c >>> fault code = supervisor read data, page not present >>> instruction pointer = 0x20:0xffffffff81983055 >>> stack pointer = 0x28:0xffffffcf75bd60a0 >>> frame pointer = 0x28:0xffffffcf75bd68f0 >>> code segment = base 0x0, limit 0xfffff, type 0x1b >>> = DPL 0, pres 1, long 1, def32 0, gran 1 >>> processor eflags = interrupt enabled, resume, IOPL = 0 >>> current process = 0 (zio_write_issue_hig) >>> trap number = 12 >>> panic: page fault >>> cpuid = 22 >>> KDB: stack backtrace: >>> db_trace_self_wrapper() at db_trace_self_wrapper+0x2a/**frame >>> 0xffffffcf75bd5b30 >>> kdb_backtrace() at kdb_backtrace+0x37/frame 0xffffffcf75bd5bf0 >>> panic() at panic+0x1ce/frame 0xffffffcf75bd5cf0 >>> trap_fatal() at trap_fatal+0x290/frame 0xffffffcf75bd5d50 >>> trap_pfault() at trap_pfault+0x211/frame 0xffffffcf75bd5de0 >>> trap() at trap+0x344/frame 0xffffffcf75bd5fe0 >>> calltrap() at calltrap+0x8/frame 0xffffffcf75bd5fe0 >>> --- trap 0xc, rip = 0xffffffff81983055, rsp = 0xffffffcf75bd60a0, rbp = >>> 0xffffffcf75bd68f0 --- >>> lzjb_compress() at lzjb_compress+0x185/frame 0xffffffcf75bd68f0 >>> zio_compress_data() at zio_compress_data+0x92/frame 0xffffffcf75bd6920 >>> zio_write_bp_init() at zio_write_bp_init+0x24b/frame 0xffffffcf75bd6970 >>> zio_execute() at zio_execute+0xc3/frame 0xffffffcf75bd69b0 >>> taskqueue_run_locked() at taskqueue_run_locked+0x74/**frame >>> 0xffffffcf75bd6a00 >>> taskqueue_thread_loop() at taskqueue_thread_loop+0x46/**frame >>> 0xffffffcf75bd6a20 >>> fork_exit() at fork_exit+0x11f/frame 0xffffffcf75bd6a70 >>> fork_trampoline() at fork_trampoline+0xe/frame 0xffffffcf75bd6a70 >>> --- trap 0, rip = 0, rsp = 0xffffffcf75bd6b30, rbp = 0 --- >>> >>> lzjb_compress+0x185 corresponds to line 85 in >>> 80 cpy = src - offset; >>> 81 if (cpy >= (uchar_t *)s_start && cpy != src && >>> 82 src[0] == cpy[0] && src[1] == cpy[1] && src[2] == cpy[2]) { >>> 83 *copymap |= copymask; >>> 84 for (mlen = MATCH_MIN; mlen < MATCH_MAX; mlen++) >>> 85 if (src[mlen] != cpy[mlen]) >>> 86 break; >>> 87 *dst++ = ((mlen - MATCH_MIN) << (NBBY - MATCH_BITS)) | >>> 88 (offset >> NBBY); >>> 89 *dst++ = (uchar_t)offset; >>> >>> I think it's the first time I've seen this panic. It happened while >>> doing a >>> send/receive. I have two pools with lzjb compression; I don't know which >>> of >>> these pools caused the problem, but one of them was the source of the >>> send/receive. >>> >>> I only have a textdump but I'm happy to try to provide more information >>> that could help anyone look into this. >>> Thanks >>> Olivier >>> >> >> Oh, I can add to this one. I have a full core dump of the same problem >> caused by copying large set of files from lzjb compressed pool to lz4 >> compressed pool. vfs.zfs.recover was set. >> >> #1 0xffffffff8039d954 in kern_reboot (howto=260) >> at /usr/src/sys/kern/kern_**shutdown.c:449 >> #2 0xffffffff8039ddce in panic (fmt=) >> at /usr/src/sys/kern/kern_**shutdown.c:637 >> #3 0xffffffff80620a6a in trap_fatal (frame=, >> eva=) at /usr/src/sys/amd64/amd64/trap.**c:879 >> #4 0xffffffff80620d25 in trap_pfault (frame=0x0, usermode=0) >> at /usr/src/sys/amd64/amd64/trap.**c:700 >> #5 0xffffffff806204f6 in trap (frame=0xffffff821ca43600) >> at /usr/src/sys/amd64/amd64/trap.**c:463 >> #6 0xffffffff8060a032 in calltrap () >> at /usr/src/sys/amd64/amd64/**exception.S:232 >> #7 0xffffffff805a9367 in vm_page_alloc (object=0xffffffff80a34030, >> pindex=16633, req=97) at /usr/src/sys/vm/vm_page.c:1445 >> #8 0xffffffff8059c42e in kmem_back (map=0xfffffe00010000e8, >> addr=18446743524021862400, size=16384, flags=) >> at /usr/src/sys/vm/vm_kern.c:362 >> #9 0xffffffff8059c2ac in kmem_malloc (map=0xfffffe00010000e8, size=16384, >> flags=257) at /usr/src/sys/vm/vm_kern.c:313 >> #10 0xffffffff80595104 in uma_large_malloc (size=, >> wait=257) at /usr/src/sys/vm/uma_core.c:994 >> #11 0xffffffff80386b80 in malloc (size=16384, mtp=0xffffffff80ea7c40, >> flags=0) >> at /usr/src/sys/kern/kern_malloc.**c:492 >> #12 0xffffffff80c9e13c in lz4_compress (s_start=0xffffff80d0b19000, >> d_start=0xffffff8159445000, s_len=131072, d_len=114688, n=-2) >> at /usr/src/sys/modules/zfs/../..**/cddl/contrib/opensolaris/uts/** >> common/fs/zfs/lz4.c:843 >> #13 0xffffffff80cdde25 in zio_compress_data (c=, >> src=, dst=0xffffff8159445000, s_len=131072) >> at /usr/src/sys/modules/zfs/../..**/cddl/contrib/opensolaris/uts/** >> common/fs/zfs/zio_compress.c:**109 >> #14 0xffffffff80cda012 in zio_write_bp_init (zio=0xfffffe0143a12000) >> at /usr/src/sys/modules/zfs/../..**/cddl/contrib/opensolaris/uts/** >> common/fs/zfs/zio.c:1107 >> #15 0xffffffff80cd8ec6 in zio_execute (zio=0xfffffe0143a12000) >> at /usr/src/sys/modules/zfs/../..**/cddl/contrib/opensolaris/uts/** >> common/fs/zfs/zio.c:1305 >> #16 0xffffffff803e25e6 in taskqueue_run_locked (queue=0xfffffe00060ca300) >> at /usr/src/sys/kern/subr_**taskqueue.c:312 >> #17 0xffffffff803e2e38 in taskqueue_thread_loop (arg=> out>) >> at /usr/src/sys/kern/subr_**taskqueue.c:501 >> #18 0xffffffff8036f40a in fork_exit ( >> callout=0xffffffff803e2da0 , >> arg=0xfffffe00060cc3d0, frame=0xffffff821ca43a80) >> at /usr/src/sys/kern/kern_fork.c:**988 >> #19 0xffffffff8060a56e in fork_trampoline () >> at /usr/src/sys/amd64/amd64/**exception.S:606 >> >> I have a full crash dump in case someone wants to look at it. >> >> -- >> Sphinx of black quartz, judge my vow. >> > > From owner-zfs-devel@FreeBSD.ORG Wed Sep 25 14:40:48 2013 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 681C63EB for ; Wed, 25 Sep 2013 14:40:48 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id B88762A8A for ; Wed, 25 Sep 2013 14:40:47 +0000 (UTC) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id RAA22574 for ; Wed, 25 Sep 2013 17:40:39 +0300 (EEST) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1VOqGd-0002tW-7V for zfs-devel@FreeBSD.org; Wed, 25 Sep 2013 17:40:39 +0300 Message-ID: <5242F5AE.6090407@FreeBSD.org> Date: Wed, 25 Sep 2013 17:39:42 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.0 MIME-Version: 1.0 To: zfs-devel@FreeBSD.org Subject: Re: 4045 zfs write throttle & i/o scheduler performance work References: <5235E97F.6050801@FreeBSD.org> In-Reply-To: <5235E97F.6050801@FreeBSD.org> X-Enigmail-Version: 1.5.2 Content-Type: text/plain; charset=x-viet-vps Content-Transfer-Encoding: 7bit X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Sep 2013 14:40:48 -0000 on 15/09/2013 20:08 Andriy Gapon said the following: > > I wonder if anyone already has a patch for integrating "4045 zfs write throttle > & i/o scheduler performance work" from Illumos to FreeBSD. Or if anyone is > working on this. > > We would like to test this change in our environment as soon as possible. So I > could start working on it if needed. > > BTW, it seems that "3581 spa_zio_taskq[ZIO_TYPE_FREE][ZIO_TASKQ_ISSUE]->tq_lock > is piping hot" was not integrated into FreeBSD despite being committed to > Illumos back in February. I am not sure if we need the main change from this > commit, but, as it is frequent with Illumos commits, it contains some changes to > definitions of some macros, enums and structs. So I think it is worth having > that change merged, at least partially. That would help with merge the future > changing including "4045 zfs write throttle & i/o scheduler performance work". > OK, I've done some work on merging these changes. Please review: https://github.com/avg-I/freebsd/compare/illumos;merge-zfs-write-throttle https://github.com/avg-I/freebsd/commits/illumos/merge-zfs-write-throttle As I am not sure that I've got everything right, I will appreciate it. Especially please see my concerns for this commit: https://github.com/avg-I/freebsd/commit/5c7d4daa51c78477c8f0c9af6de7c99a81ff6418 I am sure that there is something to be concerned about :) -- Andriy Gapon From owner-zfs-devel@FreeBSD.ORG Thu Sep 26 13:28:57 2013 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 5027DA95 for ; Thu, 26 Sep 2013 13:28:57 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: from mail-la0-x22d.google.com (mail-la0-x22d.google.com [IPv6:2a00:1450:4010:c03::22d]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id B351429CA for ; Thu, 26 Sep 2013 13:28:56 +0000 (UTC) Received: by mail-la0-f45.google.com with SMTP id eh20so935701lab.4 for ; Thu, 26 Sep 2013 06:28:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=delphix.com; s=google; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=TVOqweKbBYiWG+YWBDpryKn8ryGNFb6AedcDyeMOXrA=; b=LGFFMN901089GZVsV+jTzd6smXBXVI6vRzBm0fyA2TAVGkGcLWXIpg1v5Ei2dFsBDW QeFIs78ieUJCDm51OHRy5AXqum4e2xeOsAQpBPjyQ13MI1vRSIEe8zzDDZGcZ1VGo9hY oLA8ZF8wOsrCpKWApKBxt3X5e5IbVxQiXYtck= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=TVOqweKbBYiWG+YWBDpryKn8ryGNFb6AedcDyeMOXrA=; b=mPSTvD/Q73gFONhOLHskVL2jdChZLqOlpo2wvJ5n7gY7fNHQvUEPNI5LgZS25ZcIt2 rTY5e9QFMgSDrhFXrsEG0K70lFab8ZOJiWuk2Q8X1UzcEJlfA0iGasHxncq6X/ERG6L6 HxpfRli9Hnx0lOzhgno/PJOiZieT4TP72uRQJekCMfPtIkErfPlLITIaFHQVzDyeYRqY iGMJBEUYsRlJ53G+TvyAn/8StWtbOWfxArbw7c7zgvtSLH9oEC3F3OQ5r/8wuY32hdfw n34Xu/OG1WLR8Bh3x4tM81xkEdRKiEkUyA+aa/sP4f9qkIsJ/cPM++elJNfR87qspeIG sI6Q== X-Gm-Message-State: ALoCoQnY1692C5XgMKxKSJSl2WEyLoueENO0hsTwmy7NJ2MIJTbkniyxmwhlwfn/XaXLbdrf8EeG MIME-Version: 1.0 X-Received: by 10.112.77.134 with SMTP id s6mr1491921lbw.38.1380202133512; Thu, 26 Sep 2013 06:28:53 -0700 (PDT) Received: by 10.114.186.8 with HTTP; Thu, 26 Sep 2013 06:28:53 -0700 (PDT) In-Reply-To: <51F03331.6060403@delphij.net> References: <51F03331.6060403@delphij.net> Date: Thu, 26 Sep 2013 15:28:53 +0200 Message-ID: Subject: Re: Is 'zpool clear' supposed to work in case when pool I/O is suspended? From: Matthew Ahrens To: Xin LI Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: zfs-devel@freebsd.org, George Wilson X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Sep 2013 13:28:57 -0000 Sorry this email slipped off my radar. Hopefully this info is still relevant to you. This was mistakenly changed with the libzfs_core push. It was fixed in the following illumos commit: https://www.illumos.org/issues/4080 commit 22e30981d82a0b6dc89253596ededafae8655e00 Author: George Wilson Date: Thu Aug 29 10:56:49 2013 -0800 3954 metaslabs continue to load even after hitting zfs_mg_alloc_failure limit 4080 zpool clear fails to clear pool 4081 need zfs_mg_noalloc_threshold Reviewed by: Adam Leventhal Reviewed by: Matthew Ahrens Approved by: Richard Lowe --matt On Wed, Jul 24, 2013 at 10:04 PM, Xin Li wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA512 > > Hi, Matthew, > > Looking at zfs_ioc_clear(), it seems like we wanted to use it as a way > of waking up the pool from suspended I/O: > > %%% > /* > * Resume any suspended I/Os. > */ > if (zio_resume(spa) != 0) > error = SET_ERROR(EIO); > > spa_close(spa, FTAG); > %%% > > However, it's marked as POOL_CHECK_SUSPENDED: > > %%% > zfs_ioctl_register_pool(ZFS_IOC_CLEAR, zfs_ioc_clear, > zfs_secpolicy_config, B_TRUE, POOL_CHECK_SUSPENDED); > %%% > > And the change was introduced in Illumos revision 4445fffb > (libzfs_core). Is this part intentional? Before this it was > POOL_CHECK_NONE. > > Cheers, > - -- > Xin LI https://www.delphij.net/ > FreeBSD - The Power to Serve! Live free or die > -----BEGIN PGP SIGNATURE----- > > iQEcBAEBCgAGBQJR8DMxAAoJEG80Jeu8UPuzpLcH/iZD7lkMyvEaKrzs37UgqPHH > +gxvqozX9U620Bog8IQ+vKsV+8G6zWpFWfLb3OhNewNJwgojynaRShToPwr70sBR > 72SuJ8LaUq4FafgOPQhJRyLHYnvF4986S93JIgOuHprzRFhWsLzBP8OSaKrfbAFp > 89YE8EIMoc91L+c6gvGcfcDAdb25J4xeRlv4ZLeUf6pMmQ6IRnoZ15XrKhkT9J8b > ZRNQO1g6Xy3Ub8XjxuANAVLH+lUq8APKBhoQO82vsfVgnm6U4U4o1MKDI2ooV/94 > hf+bwPZ/p63z9b5cf0wrI1B6lZ06JBjp8WTf7DbufhNik6Kg+orlYvaMyVt9kMk= > =bBCp > -----END PGP SIGNATURE----- > From owner-zfs-devel@FreeBSD.ORG Fri Sep 27 13:06:42 2013 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id AA9F6CA4 for ; Fri, 27 Sep 2013 13:06:42 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: from mail-pa0-x233.google.com (mail-pa0-x233.google.com [IPv6:2607:f8b0:400e:c03::233]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 7D29E2B93 for ; Fri, 27 Sep 2013 13:06:42 +0000 (UTC) Received: by mail-pa0-f51.google.com with SMTP id kp14so2762035pab.24 for ; Fri, 27 Sep 2013 06:06:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=delphix.com; s=google; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=hZ0eHa+uvZpDoL9uMeLpeAnz+vMEzH1K77pg6dqVarA=; b=Yxm5+EZ3rpIFvWeNVWos9FZQ3CdAgh/b/zf47Vjd63a1ngNk/+Jjytz4z9yXbclBiE YzSSCErTSCYWscmeUTDn1Jd/r80sLuB0cRLsj3PtFPYTltjCoWgukL9gHqdpSCjfQAdW 4fzBw6y8Ctpky6jwBkKdPctzU+c4dwKitP3Dg= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=hZ0eHa+uvZpDoL9uMeLpeAnz+vMEzH1K77pg6dqVarA=; b=GsWxP/G2qQ3LeuYbgyHzF2ZxbXQlbNd3bZryrXK8xhraNcEiZlgB7C/hOfgzbW8Or5 72MrPu411DPEwyziUjogJFDMVQhii1GbNKEHVcI9FNb1Ky56tZWsC4tLNwmdApNXU1u1 QET9Mw1kL2ivvCaUnIQZlxBYzMGzQWrWckXU3lbkKYjE1N/FDfvjcOOU5LujIlb9zfyy j6soSoFpvhNKjMKvYUAO6Dh9ACeWFcLEvOzo/tnN4a8vIe/si+RqOxS0I8jhY8GfGPiL HT+90FTHYsAQO4n1oc8DuWctqiUsLdndwDis4tWZEWIPtzOnys5s7B6ey0KuJAhLwPYp QIYg== X-Gm-Message-State: ALoCoQkCJyzHw3EqQkfcwhjhHATARtL5pYhV++qyuwYKLNvgEwz7TV4mwJSQbfbHyrOtAccWi4bx MIME-Version: 1.0 X-Received: by 10.66.249.231 with SMTP id yx7mr11598134pac.116.1380287202012; Fri, 27 Sep 2013 06:06:42 -0700 (PDT) Received: by 10.70.41.162 with HTTP; Fri, 27 Sep 2013 06:06:41 -0700 (PDT) In-Reply-To: <20130918135212.GA1370@garage.freebsd.pl> References: <523989C2.9070706@FreeBSD.org> <20130918135212.GA1370@garage.freebsd.pl> Date: Fri, 27 Sep 2013 15:06:41 +0200 Message-ID: Subject: Re: r205231 From: Matthew Ahrens To: Pawel Jakub Dawidek Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: zfs-devel@freebsd.org X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 27 Sep 2013 13:06:42 -0000 On Wed, Sep 18, 2013 at 3:52 PM, Pawel Jakub Dawidek wrote: > On Wed, Sep 18, 2013 at 02:08:50PM +0300, Andriy Gapon wrote: > > > > Guys, > > > > I would like to ask your opinion of r205231, specifically the part that > splits > > the state lists. > > The change is quite large. I can admit that I do not fully understand > it. And > > it introduces many differences to the upstream code which makes merges > quite a > > bit harder. > > AFAIR Kip observed this contention for workload with multiple sequential > streams, like video streaming or something. > > If it really reduces contention and we can prove it then I think it > would be nice to upstream the change. My guess would be that it does > help, the hard part might be to find a case where it is clearly visible. > > I'm also not fun of hardcoding numer of locks to 16. It should scale > with the number of cores at least. > Agreed, I'd be happy to review this for inclusion in illumos. I responded to Andriy's post on zfs@lists.illumos.org: I have seen contention on arcs_mtx before. To validate that the changes help, it would be good to see lockstat (or equivalent) data before + after, and a description of the workload. --matt From owner-zfs-devel@FreeBSD.ORG Tue Nov 5 08:56:01 2013 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 31FD71A3 for ; Tue, 5 Nov 2013 08:56:01 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 8113726D0 for ; Tue, 5 Nov 2013 08:55:57 +0000 (UTC) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id KAA00444 for ; Tue, 05 Nov 2013 10:55:56 +0200 (EET) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1VdcQV-000AOQ-Mv for zfs-devel@FreeBSD.org; Tue, 05 Nov 2013 10:55:55 +0200 Message-ID: <5278B264.8010809@FreeBSD.org> Date: Tue, 05 Nov 2013 10:55:00 +0200 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.1.0 MIME-Version: 1.0 To: zfs-devel@FreeBSD.org Subject: Re: 4045 zfs write throttle & i/o scheduler performance work References: <5235E97F.6050801@FreeBSD.org> <5242F5AE.6090407@FreeBSD.org> In-Reply-To: <5242F5AE.6090407@FreeBSD.org> X-Enigmail-Version: 1.6 Content-Type: text/plain; charset=x-viet-vps Content-Transfer-Encoding: 7bit X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Nov 2013 08:56:01 -0000 on 25/09/2013 17:39 Andriy Gapon said the following: > on 15/09/2013 20:08 Andriy Gapon said the following: >> >> I wonder if anyone already has a patch for integrating "4045 zfs write throttle >> & i/o scheduler performance work" from Illumos to FreeBSD. Or if anyone is >> working on this. >> >> We would like to test this change in our environment as soon as possible. So I >> could start working on it if needed. >> >> BTW, it seems that "3581 spa_zio_taskq[ZIO_TYPE_FREE][ZIO_TASKQ_ISSUE]->tq_lock >> is piping hot" was not integrated into FreeBSD despite being committed to >> Illumos back in February. I am not sure if we need the main change from this >> commit, but, as it is frequent with Illumos commits, it contains some changes to >> definitions of some macros, enums and structs. So I think it is worth having >> that change merged, at least partially. That would help with merge the future >> changing including "4045 zfs write throttle & i/o scheduler performance work". >> > > OK, I've done some work on merging these changes. > Please review: > https://github.com/avg-I/freebsd/compare/illumos;merge-zfs-write-throttle > https://github.com/avg-I/freebsd/commits/illumos/merge-zfs-write-throttle > As I am not sure that I've got everything right, I will appreciate it. > > Especially please see my concerns for this commit: > https://github.com/avg-I/freebsd/commit/5c7d4daa51c78477c8f0c9af6de7c99a81ff6418 > I am sure that there is something to be concerned about :) I am going to start committing these (and some more) changes into the tree. Please try to find some time to review them before that. Also, I could use an advice on how to put the illumos changes into vendor and vendor-sys? Would procedure do you typically use? Perhaps you have some script(s) to automate the job? Additionally, if I switch commit references from illumos mercurial repo to illumos github git repo, would that be okay with everyone? My reason is that the latter is a little bit easier to access and browse. illumos opengrok also uses the git repo for project history. Thank you! -- Andriy Gapon From owner-zfs-devel@FreeBSD.ORG Tue Nov 5 09:07:55 2013 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 688B0563; Tue, 5 Nov 2013 09:07:55 +0000 (UTC) (envelope-from delphij@delphij.net) Received: from anubis.delphij.net (anubis.delphij.net [IPv6:2001:470:1:117::25]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 49E952778; Tue, 5 Nov 2013 09:07:55 +0000 (UTC) Received: from delphij-macbook.local (c-67-188-85-47.hsd1.ca.comcast.net [67.188.85.47]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by anubis.delphij.net (Postfix) with ESMTPSA id CE42D1FFBA; Tue, 5 Nov 2013 01:07:54 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=delphij.net; s=anubis; t=1383642475; bh=DK64M02d1gO6LCMvR0riBjH8EQ+L/F74xbJgnKmolfI=; h=Date:From:Reply-To:To:Subject:References:In-Reply-To; b=e+ubywNs+6cYgIyVO1Nv49i3x6a5dFTB7Mi47FtFuCQP6TvOTRixw8fv21H6fS3zW NCnfwiqT6i2ra7ejqOWLtApQ8MnhPrs5FocygO0Luldcy3pu/vxMbGHXIIDtt6Hawh qjIuSTJWNc0i2hep9RDw4OqxqrWZB5BlR8AqJRyU= Message-ID: <5278B566.2040809@delphij.net> Date: Tue, 05 Nov 2013 01:07:50 -0800 From: Xin Li Organization: The FreeBSD Project MIME-Version: 1.0 To: Andriy Gapon , zfs-devel@FreeBSD.org Subject: Re: 4045 zfs write throttle & i/o scheduler performance work References: <5235E97F.6050801@FreeBSD.org> <5242F5AE.6090407@FreeBSD.org> <5278B264.8010809@FreeBSD.org> In-Reply-To: <5278B264.8010809@FreeBSD.org> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list Reply-To: d@delphij.net List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Nov 2013 09:07:55 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 On 11/5/13, 12:55 AM, Andriy Gapon wrote: [...] > I am going to start committing these (and some more) changes into > the tree. Please try to find some time to review them before that. > > Also, I could use an advice on how to put the illumos changes into > vendor and vendor-sys? Would procedure do you typically use? > Perhaps you have some script(s) to automate the job? I think mm@ may have some scripts. I do it manually, it's mostly hg up -r and rsync to corresponding vendor/vendor-sys trees, then commit them together if the changeset touches both parts. When merging, we use src/cddl/contrib/opensolaris and src/sys/cddl/contrib/opensolaris as merge points. > Additionally, if I switch commit references from illumos mercurial > repo to illumos github git repo, would that be okay with everyone? I have proposed this but nobody responded, personally, I am for this change. The benefit of switching to github would be it's more up-to-date (sometimes the Illumos hg could lag behind github a little bit), and the downside of doing so is the version number is not quite linear. It would be helpful to import one or a few sentinel change that references to both hg and github revisions. Cheers, -----BEGIN PGP SIGNATURE----- iQIcBAEBCgAGBQJSeLVmAAoJEJW2GBstM+ns21AP/RDfmR85zu6nw4twh8OLOLDW /SYo3Tu33flnYv3+B2qz9ceBRLfYyjFWtq0+/Rr1r5VNCBtxN9tQQmxaHGNCp4yb Jh0JmL3bnBwG6EAIj9A+EXzPZ0hdji8D5HWbSDRgtG/qDp/QNxTEL5br85aAUIK/ 4GAifJwY85wujlHXozoMswmBqvCWxawESUJM5EyIaHq8NS3SyfkBBLpVoS8zTwBP hpgbR91zUKYUDSGVQ2UsxYD0K66NESAn6kzg5EuIcjgB4jRoBDrIzY5Dihm7UvWH 22tzxCBM6+n0rJbMPHj4bUfrsaOQjVYzn+p/v51lXacH2KUodyIMymnVTmREv+RL gybt4OqSEhGj1iLqRlw1/7poB0dpJAH3QfFyYsIlXMcysb6QFABmqmnQm0AULHSw NdQxTvjk78KxJDbliGIXbIIjJCi6BEsSLJe4VkxWMsa7FLw0tiv5Ebve2VMGs2T+ BbXvGE4PQT+vXQZygc2vSvM/sA+cN/XOXHiaZUCzXNpmiuKOJqb98B8IXhE9LnyZ UpCvmGAVrbETzhTgynt/c0Wh7x7wnK/TynTsoPctLEdhf2lZ+Gi4Iq0FVQW4YaaH foBdYyW1VmaoXhtOh9QVM1rrRINA8mmeK4vJiSAFBqJdBvuxunvuVwhMMDfcnEll fFKPeyw7V9hLLXNELzaD =gkTK -----END PGP SIGNATURE----- From owner-zfs-devel@FreeBSD.ORG Tue Nov 5 10:07:59 2013 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 57169993 for ; Tue, 5 Nov 2013 10:07:59 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id A75FA2ACD for ; Tue, 5 Nov 2013 10:07:58 +0000 (UTC) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id MAA01851; Tue, 05 Nov 2013 12:07:51 +0200 (EET) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1VddY7-000AU5-0L; Tue, 05 Nov 2013 12:07:51 +0200 Message-ID: <5278C33E.8020800@FreeBSD.org> Date: Tue, 05 Nov 2013 12:06:54 +0200 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.1.0 MIME-Version: 1.0 To: d@delphij.net, zfs-devel@FreeBSD.org Subject: Re: 4045 zfs write throttle & i/o scheduler performance work References: <5235E97F.6050801@FreeBSD.org> <5242F5AE.6090407@FreeBSD.org> <5278B264.8010809@FreeBSD.org> <5278B566.2040809@delphij.net> In-Reply-To: <5278B566.2040809@delphij.net> X-Enigmail-Version: 1.6 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Nov 2013 10:07:59 -0000 on 05/11/2013 11:07 Xin Li said the following: > On 11/5/13, 12:55 AM, Andriy Gapon wrote: >> Additionally, if I switch commit references from illumos mercurial repo >> to illumos github git repo, would that be okay with everyone? > > I have proposed this but nobody responded, personally, I am for this > change. > > The benefit of switching to github would be it's more up-to-date (sometimes > the Illumos hg could lag behind github a little bit), and Indeed, it seems that right now hg is about two months behind (using a fresh clone of ssh://anonhg@hg.illumos.org/illumos-gate). > the downside of doing so is the version number is not quite linear. It > would be helpful to import one or a few sentinel change that references to > both hg and github revisions. Given the above not sure if that would be even possible. And thank you for the other hints and suggestions that you provided! -- Andriy Gapon From owner-zfs-devel@FreeBSD.ORG Tue Nov 5 14:12:25 2013 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 3EB861EB for ; Tue, 5 Nov 2013 14:12:25 +0000 (UTC) (envelope-from will@firepipe.net) Received: from mail-vc0-f180.google.com (mail-vc0-f180.google.com [209.85.220.180]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 008772A9D for ; Tue, 5 Nov 2013 14:12:24 +0000 (UTC) Received: by mail-vc0-f180.google.com with SMTP id lc6so5723542vcb.11 for ; Tue, 05 Nov 2013 06:12:18 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=od1z7aSGx21e3A7yESFbbZj9lYizrR9tVMyILOU1afM=; b=LHcY8aJI9Lpuoims9U6runQID/Y6Q9e1srLkmiy/MzTbgPPWeCDKMjq+aPuXoR8aAX doew7FbrNv+9+eW0/mEXDQfXAa1GpsLv99xgolspiq2S+VsMMKvCu7kfAeyashQanrQ0 9uJC+Z7BMOVpWChfkuEp+YcBjIUHvzJW/P3LnHRfK2LjS/tR3mORmKWzM88BaNQfmSI4 vOd3eGc4yN+uMPcZRz2Ab1jB8xoOaZ22Lg64giopbOJTvC5Dv2DgVxx0+qH53sn2q1yk Qy2UomYebqfNCE5wrlyr99153oEkvC7oRJTsEbvf8H+quGwDxaqKc0/HDhN+HYbiv0/a HPWw== X-Gm-Message-State: ALoCoQlfE9f/RIrZJe2QpO48nxJrMuwQyQBw0qfEqpFMm1Psa12TP2CbMvMC3zinRDWqOEfDpLNO MIME-Version: 1.0 X-Received: by 10.52.187.138 with SMTP id fs10mr12871445vdc.10.1383660738146; Tue, 05 Nov 2013 06:12:18 -0800 (PST) Received: by 10.58.161.196 with HTTP; Tue, 5 Nov 2013 06:12:18 -0800 (PST) In-Reply-To: <5278B264.8010809@FreeBSD.org> References: <5235E97F.6050801@FreeBSD.org> <5242F5AE.6090407@FreeBSD.org> <5278B264.8010809@FreeBSD.org> Date: Tue, 5 Nov 2013 07:12:18 -0700 Message-ID: Subject: Re: 4045 zfs write throttle & i/o scheduler performance work From: Will Andrews To: Andriy Gapon Content-Type: text/plain; charset=UTF-8 Cc: "zfs-devel@freebsd.org" X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Nov 2013 14:12:25 -0000 Hi, On Tue, Nov 5, 2013 at 1:55 AM, Andriy Gapon wrote: > Also, I could use an advice on how to put the illumos changes into vendor and > vendor-sys? Would procedure do you typically use? Perhaps you have some > script(s) to automate the job? My illumos-gate fork contains a script that I wrote which helps copy files between illumos-gate and a FreeBSD(-like) tree: https://github.com/wca/illumos-gate/blob/helper-scripts/freebsd_transfer.rb I last used it to update my fork about two months ago. Note that it ignores new/deleted files, on assumption that such files are not shared between illumos and FreeBSD (for example, vdev_geom.c). I normally use it in the opposite direction, but it should work for an illumos-gate import too. It would be nice if the relationship between illumos-gate and FreeBSD files was less complicated. > Additionally, if I switch commit references from illumos mercurial repo to > illumos github git repo, would that be okay with everyone? > My reason is that the latter is a little bit easier to access and browse. > illumos opengrok also uses the git repo for project history. It makes sense to me to switch to referencing the git repository, since illumos began using it as their reference many moons ago. --Will. From owner-zfs-devel@FreeBSD.ORG Wed Nov 6 20:47:38 2013 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 33A50D51 for ; Wed, 6 Nov 2013 20:47:38 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 7F1752CD6 for ; Wed, 6 Nov 2013 20:47:37 +0000 (UTC) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id WAA29754; Wed, 06 Nov 2013 22:47:34 +0200 (EET) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1VeA0k-000FQi-2k; Wed, 06 Nov 2013 22:47:34 +0200 Message-ID: <527AAAC2.9010708@FreeBSD.org> Date: Wed, 06 Nov 2013 22:46:58 +0200 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.1.0 MIME-Version: 1.0 To: Will Andrews , "zfs-devel@freebsd.org" Subject: Re: 4045 zfs write throttle & i/o scheduler performance work References: <5235E97F.6050801@FreeBSD.org> <5242F5AE.6090407@FreeBSD.org> <5278B264.8010809@FreeBSD.org> In-Reply-To: X-Enigmail-Version: 1.6 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Nov 2013 20:47:38 -0000 on 05/11/2013 16:12 Will Andrews said the following: > My illumos-gate fork contains a script that I wrote which helps copy > files between illumos-gate and a FreeBSD(-like) tree: > > https://github.com/wca/illumos-gate/blob/helper-scripts/freebsd_transfer.rb > > I last used it to update my fork about two months ago. Note that it > ignores new/deleted files, on assumption that such files are not > shared between illumos and FreeBSD (for example, vdev_geom.c). I > normally use it in the opposite direction, but it should work for an > illumos-gate import too. Will, thank you for the script! > It would be nice if the relationship between illumos-gate and FreeBSD > files was less complicated. Indeed. BTW, it seems that we don't pull each and every illumos commit into vendor / vendor-sys. What is the criterion for deciding whether we have to import the commit or not? Thank you! -- Andriy Gapon From owner-zfs-devel@FreeBSD.ORG Wed Nov 6 21:32:46 2013 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id C6FF0DA2; Wed, 6 Nov 2013 21:32:46 +0000 (UTC) (envelope-from delphij@delphij.net) Received: from anubis.delphij.net (anubis.delphij.net [64.62.153.212]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id AE005201F; Wed, 6 Nov 2013 21:32:45 +0000 (UTC) Received: from zeta.ixsystems.com (unknown [69.198.165.132]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by anubis.delphij.net (Postfix) with ESMTPSA id 7D1E618862; Wed, 6 Nov 2013 13:32:45 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=delphij.net; s=anubis; t=1383773565; bh=hV4Z+muNPESkjkQNu/y/dy2HKroswDHpJq0vJtzd2dI=; h=Date:From:Reply-To:To:Subject:References:In-Reply-To; b=a3bmhlik/f08q3g2w6x6iovKSB4Sg7+V+j9dP0J9LZJ1MtBEyD/VAa3eL9bA8mXea pO4shwN08tXiqodjAgcy2jeok1uqjLjBV+cB5L6fW6Q5n8QxP9VTEB9clglY7CR5vG PyIlGUxRMmBLVpyQnzLWZ8IDq5Mt5CGMEYTzopzE= Message-ID: <527AB57D.7090306@delphij.net> Date: Wed, 06 Nov 2013 13:32:45 -0800 From: Xin Li Organization: The FreeBSD Project MIME-Version: 1.0 To: Andriy Gapon , Will Andrews , "zfs-devel@freebsd.org" Subject: Re: 4045 zfs write throttle & i/o scheduler performance work References: <5235E97F.6050801@FreeBSD.org> <5242F5AE.6090407@FreeBSD.org> <5278B264.8010809@FreeBSD.org> <527AAAC2.9010708@FreeBSD.org> In-Reply-To: <527AAAC2.9010708@FreeBSD.org> X-Enigmail-Version: 1.6 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list Reply-To: d@delphij.net List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Nov 2013 21:32:46 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 On 11/06/13 12:46, Andriy Gapon wrote: > on 05/11/2013 16:12 Will Andrews said the following: >> My illumos-gate fork contains a script that I wrote which helps >> copy files between illumos-gate and a FreeBSD(-like) tree: >> >> https://github.com/wca/illumos-gate/blob/helper-scripts/freebsd_transfer.rb >> >> >> I last used it to update my fork about two months ago. Note that it >> ignores new/deleted files, on assumption that such files are not >> shared between illumos and FreeBSD (for example, vdev_geom.c). >> I normally use it in the opposite direction, but it should work >> for an illumos-gate import too. > > Will, > > thank you for the script! > >> It would be nice if the relationship between illumos-gate and >> FreeBSD files was less complicated. > > Indeed. > > BTW, it seems that we don't pull each and every illumos commit into > vendor / vendor-sys. What is the criterion for deciding whether we > have to import the commit or not? We pull changes when it touches trees we have (i.e. ZFS or DTrace). Other changes are not imported because it would not be useful (or even won't generate a commit). Cheers, - -- Xin LI https://www.delphij.net/ FreeBSD - The Power to Serve! Live free or die -----BEGIN PGP SIGNATURE----- iQIcBAEBCgAGBQJSerV8AAoJEJW2GBstM+nsa+gP/RHzZxya0k2qGnE/QNh4ifIm 8E3nwd2InAYXrf34foKzJ35FJ+PxU6IMKvqNJnypPRylFIRE7ynpqX107c/+mlvF Bpw6H9+ld8hj6Y1N04hnjpgG2axtCO5e6JZCvuYHUKCzRfTITNgvK/Z+Yc2l5ijJ m+Ey7A81mNaaxCd7ksNTMXIImoYioydRCS7zXfET1MVsoPcNcG2CCDiw+MSaH6lx zNm1YbjR0tnGbcXZ5tVtJriURnhtzdVAgM5Ek8LEXwHAJx1n/OaS6XpJIEHVk3ag BU1FyxUdWuQV0aL05ObSc/9Sq6+FhHOE5T7pWTvVs3qywKvTACpyEev8y2WrS3VX 91FZPBrjk87v6Buwi0Aah+A5nBrK83LcwAFRYyZkOQQxSKWzo7kGFjl4/74E6WBT ijOl3LEO+38zKWr9AUfBa7ECbrxrNm6tjT8oGlhjtgxHoDLYeknethUpz04M3KrJ agFKnSD2n8B11YO3z2FYj5lOLnqBAyJxycI+1jDcQ3WAXOLnYdDbCr302unn/5ee TsToocj4HWpwgeLJrIwZ27qT1KIqKqkZnC7xWk9OCFjdw4ZMVmTVsh5XtZkIFQDR Q9Xu3kcJHut50+mBIXEMl4xhTp2E3wnWZ4JdpGRrRNYJP+LAOGIwL37vH+nbddGq BSwnG3Tgug4K+qRyQbbj =URZQ -----END PGP SIGNATURE----- From owner-zfs-devel@FreeBSD.ORG Wed Nov 20 12:18:27 2013 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 67C2D347 for ; Wed, 20 Nov 2013 12:18:27 +0000 (UTC) Received: from nm41.bullet.mail.ne1.yahoo.com (nm41.bullet.mail.ne1.yahoo.com [98.138.120.48]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 1F75625E3 for ; Wed, 20 Nov 2013 12:18:25 +0000 (UTC) Received: from [127.0.0.1] by nm41.bullet.mail.ne1.yahoo.com with NNFMP; 20 Nov 2013 12:18:19 -0000 Received: from [98.138.226.180] by nm41.bullet.mail.ne1.yahoo.com with NNFMP; 20 Nov 2013 12:15:22 -0000 Received: from [98.137.12.55] by tm15.bullet.mail.ne1.yahoo.com with NNFMP; 20 Nov 2013 12:15:22 -0000 Received: from [216.39.60.200] by tm15.bullet.mail.gq1.yahoo.com with NNFMP; 20 Nov 2013 12:15:21 -0000 Received: from [127.0.0.1] by omp1087.mail.gq1.yahoo.com with NNFMP; 20 Nov 2013 12:15:21 -0000 X-Yahoo-Newman-Property: ymail-4 X-Yahoo-Newman-Id: 905553.48933.bm@omp1087.mail.gq1.yahoo.com Received: (qmail 84816 invoked by uid 60001); 20 Nov 2013 12:15:21 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1384949721; bh=UFe8D+CQbfZajN7hwqe/zWYu3d/h+Aq+Gc+XGbTlmMQ=; h=X-YMail-OSG:Received:X-Rocket-MIMEInfo:X-Mailer:Message-ID:Date:From:Reply-To:Subject:To:MIME-Version:Content-Type; b=BCu8Suc8VMzzOeEdeJFLh+fV17kWO4X+qe1oqKiv7QVapkPUyupMIaNpY8W5wmCzi7cCLWJv4F67gZgufDP18sDWupOihNue4QEcjOqh9zqCHOwdC4A3qSAjwA8s/rUJU1AScS/Wpe78e6ehMlZHdhV+Gp2umwF3GVVszWUEkZI= DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=X-YMail-OSG:Received:X-Rocket-MIMEInfo:X-Mailer:Message-ID:Date:From:Reply-To:Subject:To:MIME-Version:Content-Type; b=5NkuLfW4wXLy9AlS/bc6g/jL0hEzh8Z4CAS/hwFfW/ZMeM6HtTimhdWzjrVLr1X4qvNQgXl9OcfUIe7G1ozuW3mH6jQJ0XeOcgZegVGapMa3X9rIbYlkYg2H4kb2gPqjneQcvBuuVeEx/fvGOQj2l5xO6FGT5EWJS9tpMFDaU+c=; X-YMail-OSG: 0pDJhFsVM1lBYOGauHpBygiM0oZLItmuPVKVtAc2re0lgvV c6WIKzKNCxLDG4mXFIZtXDrx8zAE1AoDt8CoWW3_LtLzDpUHQ2ACkirksYKA Te9MI0e4C4iHO8f6qzkCCmD1ncQGMB8CZQsN3HbQp7Mp8kOsdrJvRPBInloy J2qqSPkhlM6ldnttFLy_fp1N91a.NUp8eRZAQLCewdHL81OgP_M7Aj1pu5bC KXQsz.uwcZgjFmjE3GgaFUVDie5sRo.je9O5J4CNNkqeoM48sBQU5WiriPvP V0xsSIeTrCiENvx9iD78yftFV6BneNZnNIeu4aKOCZRV1NsQg_Qua7CbiSH8 iv32a7TIT8zLBaFyREBsbXuQ7cmKzmmAKYn0bTCK4dLM2jiCEshRVpVFn0zg _f2SJyQS_CS6S1hEMLxml1hbCuOfI5pBcDNOcqlMnAgAecg20IHFgYUaCKsu VZWO9rq.iJNB31dEt0yfVUIU3OIpI270INPT.Hixqbh.vMVqzZFNDen8gUWl fafIVV7gMJgxm_4Agies5LVXCYLE0dMs- Received: from [27.154.58.234] by web164006.mail.gq1.yahoo.com via HTTP; Wed, 20 Nov 2013 04:15:21 PST X-Rocket-MIMEInfo: 002.001, SGksCkkgaGl0IGEgWkZTIFNBIHByb2JsZW0gb24gRnJlZUJTRCA5LjIsIGJ1dCBJIGJlbGlldmUgdGhlIGlzc3VlIGV4aXN0cyBvbiBvdGhlciBwbGF0Zm9ybSB0b28uIEhlcmUgaXMgdGhlIGRlc2NyaXB0aW9uIG9mIHRoZSBidWcuCgoKUFJPQkxFTToKcnVuIHRoZSBhdHRhY2hlZCBzY3JpcHQgb24gYSBaRlMsIGFmdGVyIGEgZmV3IHNlY29uZHMsIHJ1biB6ZGIgLXZ2diBvbiB0aGUgWkZTLCB6ZGIgd2lsbCBjcmFzaCBhdCB0aGUgZm9sbG93aW5nIGFzc2VydGlvbjoKCgpBc3NlcnRpb24gZmFpbGVkOiAoSVMBMAEBAQE- X-Mailer: YahooMailWebService/0.8.166.601 Message-ID: <1384949721.84395.YahooMailNeo@web164006.mail.gq1.yahoo.com> Date: Wed, 20 Nov 2013 04:15:21 -0800 (PST) From: James Pan Subject: a ZFS SA bug and my patch To: "developer@open-zfs.org" , "zfs-devel@freebsd.org" MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="-2043806698-1716874501-1384949721=:84395" X-Content-Filtered-By: Mailman/MimeDel 2.1.16 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.16 Precedence: list Reply-To: James Pan List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 20 Nov 2013 12:18:27 -0000 ---2043806698-1716874501-1384949721=:84395 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable Hi,=0AI hit a ZFS SA problem on FreeBSD 9.2, but I believe the issue exists= on other platform too. Here is the description of the bug.=0A=0A=0APROBLEM= :=0Arun the attached script on a ZFS, after a few seconds, run zdb -vvv on = the ZFS, zdb will crash at the following assertion:=0A=0A=0AAssertion faile= d: (IS_SA_BONUSTYPE(bonustype) && SA_HDR_SIZE_MATCH_LAYOUT(hdr, tb) || !IS_= SA_BONUSTYPE(bonustype) || (IS_SA_BONUSTYPE(bonustype) && hdr->sa_layout_in= fo =3D=3D 0)), file /usr/src/cddl/lib/libzpool/../../../sys/cddl/contrib/op= ensolaris/uts/common/fs/zfs/sa.c, line 1509.=0AAbort (core dumped)=0A=0Athe= reason is the SA's header size does not match its layout.=0A=0A=0AROOT CAU= SE:=0AThe issue will be hit when a file has more than 2 variable-length SA = and the total SA size is larger than the bonus buffer's length - =A0sizeof = (blkptr_t), but less the bonus buffer's length.=0A=0Ain=A0sa_find_sizes(), = done is set to TRUE=A0if the SA size + header > the bonus buffer's length -= sizeof (blkptr_t), then hdrsize +=3D sizeof (uint16_t) will be skipped for= the second variable-length SA. If finally all SA can fit in the bonus buff= er and no spill block is needed, we will get a wrong hdrsize.=0A=0AMY FIX:= =0AI've also attached my simple fix for this issue, anyone who might have i= nterest could you please take a look? Thanks a lot!=0A ---2043806698-1716874501-1384949721=:84395 Content-Type: application/octet-stream; name="sa.c.diff" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="sa.c.diff" LS0tIHNhLmMub3JpZwkyMDEzLTA5LTI3IDA5OjEwOjI5LjAwMDAwMDAwMCAr MDgwMAorKysgc2EuYwkyMDEzLTExLTE2IDE5OjM5OjE2LjAwMDAwMDAwMCAr MDgwMApAQCAtNTUyLDkgKzU1Miw5IEBAIHNhX2ZpbmRfc2l6ZXMoc2Ffb3Nf dCAqc2EsIHNhX2J1bGtfYXR0cl8KIHsKIAlpbnQgdmFyX3NpemUgPSAwOwog CWludCBpOwotCWludCBqID0gLTE7CiAJaW50IGZ1bGxfc3BhY2U7CiAJaW50 IGhkcnNpemU7CisJaW50IGV4dHJhX2hkcnNpemU7CiAJYm9vbGVhbl90IGRv bmUgPSBCX0ZBTFNFOwogCiAJaWYgKGJ1ZnR5cGUgPT0gU0FfQk9OVVMgJiYg c2EtPnNhX2ZvcmNlX3NwaWxsKSB7CkBAIC01NzAsNiArNTcwLDcgQEAgc2Ff ZmluZF9zaXplcyhzYV9vc190ICpzYSwgc2FfYnVsa19hdHRyXwogCWlmIChi dWZ0eXBlID09IFNBX0JPTlVTKQogCQkqd2lsbF9zcGlsbCA9IEJfRkFMU0U7 CiAKKwlleHRyYV9oZHJzaXplID0gMDsKIAloZHJzaXplID0gKFNBX0JPTlVT VFlQRV9GUk9NX0RCKGRiKSA9PSBETVVfT1RfWk5PREUpID8gMCA6CiAJICAg IHNpemVvZiAoc2FfaGRyX3BoeXNfdCk7CiAKQEAgLTU4MSw4ICs1ODIsMTAg QEAgc2FfZmluZF9zaXplcyhzYV9vc190ICpzYSwgc2FfYnVsa19hdHRyXwog CiAJCSp0b3RhbCA9IFAyUk9VTkRVUCgqdG90YWwsIDgpOwogCQkqdG90YWwg Kz0gYXR0cl9kZXNjW2ldLnNhX2xlbmd0aDsKLQkJaWYgKGRvbmUpCi0JCQln b3RvIG5leHQ7CisKKwkJaWYgKGJ1ZnR5cGUgPT0gU0FfQk9OVVMgJiYgKndp bGxfc3BpbGwpIHsKKwkJCWNvbnRpbnVlOworCQl9CiAKIAkJaXNfdmFyX3N6 ID0gKFNBX1JFR0lTVEVSRURfTEVOKHNhLCBhdHRyX2Rlc2NbaV0uc2FfYXR0 cikgPT0gMCk7CiAJCWlmIChpc192YXJfc3opIHsKQEAgLTU5MCwxOSArNTkz LDI2IEBAIHNhX2ZpbmRfc2l6ZXMoc2Ffb3NfdCAqc2EsIHNhX2J1bGtfYXR0 cl8KIAkJfQogCiAJCWlmIChpc192YXJfc3ogJiYgdmFyX3NpemUgPiAxKSB7 Ci0JCQlpZiAoUDJST1VORFVQKGhkcnNpemUgKyBzaXplb2YgKHVpbnQxNl90 KSwgOCkgKworCQkJLyogRG9uJ3Qgd29ycnkgdGhlIHNwaWxsIGJsb2NrIHdp bGwgYmUgb3ZlcmZsb3dlZCwKKwkJCSAqIHdlIHdpbGwgcmVzaXplIHRoZSBz cGlsbCBibG9jayBsYXRlciBpbiBzYV9idWlsZF9sYXlvdXRzKCkKKwkJCSAq LworCQkJaWYgKGJ1ZnR5cGUgPT0gU0FfU1BJTEwgfHwgCisJCQkgICAgUDJS T1VORFVQKGhkcnNpemUgKyBzaXplb2YgKHVpbnQxNl90KSwgOCkgKwogCQkJ ICAgICp0b3RhbCA8IGZ1bGxfc3BhY2UpIHsKIAkJCQkvKgogCQkJCSAqIEFj Y291bnQgZm9yIGhlYWRlciBzcGFjZSB1c2VkIGJ5IGFycmF5IG9mCiAJCQkJ ICogb3B0aW9uYWwgc2l6ZXMgb2YgdmFyaWFibGUtbGVuZ3RoIGF0dHJpYnV0 ZXMuCi0JCQkJICogUmVjb3JkIHRoZSBpbmRleCBpbiBjYXNlIHRoaXMgaW5j cmVhc2UgbmVlZHMKKwkJCQkgKiBSZWNvcmQgdGhlIGV4dHJhIGhlYWRlciBz aXplIGluIGNhc2UgdGhpcyBpbmNyZWFzZSBuZWVkcwogCQkJCSAqIHRvIGJl IHJldmVyc2VkIGR1ZSB0byBzcGlsbC1vdmVyLgogCQkJCSAqLwogCQkJCWhk cnNpemUgKz0gc2l6ZW9mICh1aW50MTZfdCk7Ci0JCQkJaiA9IGk7CisJCQkJ aWYgKGRvbmUpCisJCQkJCWV4dHJhX2hkcnNpemUgKz0gc2l6ZW9mICh1aW50 MTZfdCk7CiAJCQl9IGVsc2UgewotCQkJCWRvbmUgPSBCX1RSVUU7Ci0JCQkJ KmluZGV4ID0gaTsKKwkJCQlpZiAoIWRvbmUpIHsKKwkJCQkJZG9uZSA9IEJf VFJVRTsKKwkJCQkJKmluZGV4ID0gaTsKKwkJCQl9CiAJCQkJaWYgKGJ1ZnR5 cGUgPT0gU0FfQk9OVVMpCiAJCQkJCSp3aWxsX3NwaWxsID0gQl9UUlVFOwog CQkJCWNvbnRpbnVlOwpAQCAtNjIyLDE5ICs2MzIsMTMgQEAgc2FfZmluZF9z aXplcyhzYV9vc190ICpzYSwgc2FfYnVsa19hdHRyXwogCQkJZG9uZSA9IEJf VFJVRTsKIAkJfQogCi1uZXh0OgogCQlpZiAoKCp0b3RhbCArIFAyUk9VTkRV UChoZHJzaXplLCA4KSkgPiBmdWxsX3NwYWNlICYmCiAJCSAgICBidWZ0eXBl ID09IFNBX0JPTlVTKQogCQkJKndpbGxfc3BpbGwgPSBCX1RSVUU7CiAJfQog Ci0JLyoKLQkgKiBqIGhvbGRzIHRoZSBpbmRleCBvZiB0aGUgbGFzdCB2YXJp YWJsZS1zaXplZCBhdHRyaWJ1dGUgZm9yCi0JICogd2hpY2ggaGRyc2l6ZSB3 YXMgaW5jcmVhc2VkLiAgUmV2ZXJzZSB0aGUgaW5jcmVhc2UgaWYgdGhhdAot CSAqIGF0dHJpYnV0ZSB3aWxsIGJlIHJlbG9jYXRlZCB0byB0aGUgc3BpbGwg YmxvY2suCi0JICovCi0JaWYgKCp3aWxsX3NwaWxsICYmIGogPT0gKmluZGV4 KQotCQloZHJzaXplIC09IHNpemVvZiAodWludDE2X3QpOworCWlmIChidWZ0 eXBlID09IFNBX0JPTlVTICYmICp3aWxsX3NwaWxsKQorCQloZHJzaXplIC09 IGV4dHJhX2hkcnNpemU7CiAKIAloZHJzaXplID0gUDJST1VORFVQKGhkcnNp emUsIDgpOwogCXJldHVybiAoaGRyc2l6ZSk7Cg== ---2043806698-1716874501-1384949721=:84395-- From owner-zfs-devel@FreeBSD.ORG Wed Nov 20 20:43:58 2013 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id DF387378 for ; Wed, 20 Nov 2013 20:43:57 +0000 (UTC) Received: from prdiron-1.llnl.gov (prdiron-1.llnl.gov [128.15.143.171]) by mx1.freebsd.org (Postfix) with ESMTP id C83F0286B for ; Wed, 20 Nov 2013 20:43:57 +0000 (UTC) X-Attachments: Received: from eris.llnl.gov ([128.115.7.7]) by prdiron-1.llnl.gov with ESMTP; 20 Nov 2013 12:42:49 -0800 Received: from twist.chaos (twist.chaos [192.168.1.126]) by eris.llnl.gov (Postfix) with ESMTP id 4D46B7C4A5; Wed, 20 Nov 2013 12:42:49 -0800 (PST) Received: from twist.chaos (localhost [127.0.0.1]) by twist.chaos (8.14.4/8.14.4/Debian-2ubuntu2) with ESMTP id rAKKh2tZ046580; Wed, 20 Nov 2013 12:43:02 -0800 Received: (from bass6@localhost) by twist.chaos (8.14.4/8.14.4/Submit) id rAKKh2OF046578; Wed, 20 Nov 2013 12:43:02 -0800 X-Authentication-Warning: twist.chaos: bass6 set sender to bass6@llnl.gov using -f Date: Wed, 20 Nov 2013 12:43:02 -0800 From: Ned Bass To: James Pan Subject: Re: [OpenZFS Developer] a ZFS SA bug and my patch Message-ID: <20131120204302.GC9676@llnl.gov> References: <1384949721.84395.YahooMailNeo@web164006.mail.gq1.yahoo.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1384949721.84395.YahooMailNeo@web164006.mail.gq1.yahoo.com> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: "developer@open-zfs.org" , "zfs-devel@freebsd.org" X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.16 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 20 Nov 2013 20:43:58 -0000 On Wed, Nov 20, 2013 at 04:15:21AM -0800, James Pan wrote: > Hi, > I hit a ZFS SA problem on FreeBSD 9.2, but I believe the issue exists on other > platform too. Here is the description of the bug. > > > PROBLEM: > run the attached script on a ZFS, after a few seconds, run zdb -vvv on the ZFS, > zdb will crash at the following assertion: > > Assertion failed: (IS_SA_BONUSTYPE(bonustype) && SA_HDR_SIZE_MATCH_LAYOUT(hdr, > tb) || !IS_SA_BONUSTYPE(bonustype) || (IS_SA_BONUSTYPE(bonustype) && hdr-> > sa_layout_info == 0)), file /usr/src/cddl/lib/libzpool/../../../sys/cddl/ > contrib/opensolaris/uts/common/fs/zfs/sa.c, line 1509. > Abort (core dumped) > > the reason is the SA's header size does not match its layout. > > > ROOT CAUSE: > The issue will be hit when a file has more than 2 variable-length SA and the > total SA size is larger than the bonus buffer's length - sizeof (blkptr_t), > but less the bonus buffer's length. > > in sa_find_sizes(), done is set to TRUE if the SA size + header > the bonus > buffer's length - sizeof (blkptr_t), then hdrsize += sizeof (uint16_t) will be > skipped for the second variable-length SA. If finally all SA can fit in the > bonus buffer and no spill block is needed, we will get a wrong hdrsize. > > MY FIX: > I've also attached my simple fix for this issue, anyone who might have interest > could you please take a look? Thanks a lot! > Good catch. I'm responsible for the first attempt to correct hdrsize in the case of spill-over with variable-sized SAs. But I clearly didn't get it quite right. I haven't tested your patch, but it looks correct to me. + if (done) + extra_hdrsize += sizeof (uint16_t); } else { - done = B_TRUE; - *index = i; + if (!done) { + done = B_TRUE; + *index = i; + } It's not very obvious to the casual observer what we're "done" with when done == B_TRUE. It seems to mean we've found the index of the first SA that will spill over _if_ we need to use a spill buffer. While we're revising this code, this could be made much more clear with a comment and a self-documenting name (spill_index_found, perhaps?). + if (buftype == SA_BONUS && *will_spill) + hdrsize -= extra_hdrsize; *will_spill implies buftype == SA_BONUS. This doesn't hurt to double-check, but we could make it more explicit with an assertion: if (*will_spill) { ASSERT3U(buftype, ==, SA_BONUS); hdrsize -= extra_hdrsize; } Ned From owner-zfs-devel@FreeBSD.ORG Fri Nov 22 15:27:26 2013 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D41E12D5 for ; Fri, 22 Nov 2013 15:27:26 +0000 (UTC) Received: from nm41-vm6.bullet.mail.gq1.yahoo.com (nm41-vm6.bullet.mail.gq1.yahoo.com [67.195.87.93]) by mx1.freebsd.org (Postfix) with SMTP id A010A2B6A for ; Fri, 22 Nov 2013 15:27:26 +0000 (UTC) Received: from [98.137.12.62] by nm41.bullet.mail.gq1.yahoo.com with NNFMP; 22 Nov 2013 15:27:20 -0000 Received: from [98.137.12.239] by tm7.bullet.mail.gq1.yahoo.com with NNFMP; 22 Nov 2013 15:27:20 -0000 Received: from [127.0.0.1] by omp1047.mail.gq1.yahoo.com with NNFMP; 22 Nov 2013 15:27:20 -0000 X-Yahoo-Newman-Property: ymail-3 X-Yahoo-Newman-Id: 134375.97118.bm@omp1047.mail.gq1.yahoo.com Received: (qmail 61298 invoked by uid 60001); 22 Nov 2013 15:27:20 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1385134039; bh=GM+lM4goraiH+w77/k8OFSjmQeCq1T0WxI4EMqLnDyg=; h=X-YMail-OSG:Received:X-Rocket-MIMEInfo:X-Mailer:References:Message-ID:Date:From:Reply-To:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type; b=nt24prwgfQEzQ+DXpbKg8yOj+GA1FxPl0UHndj/+nlSYHew5i/Ms+KMmnFY+JIquzN3cEznYAVUWV0XICgK8+YTqNw3PTG88inp80Q4GPKi+ldBFbVLZd+CiaKq/sSh/Q2ClnZtMfiykmMGTI4EcL5FhT567p9yN1ua8o7tOTVI= DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=X-YMail-OSG:Received:X-Rocket-MIMEInfo:X-Mailer:References:Message-ID:Date:From:Reply-To:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type; b=ItNvYnckYLN2GbsztZNnFkk5Qj1vR+EtubjqnDo0ctQmHWJfxRFy73GgaGtT1JQtEjxdZeytPRwjReoeXdO0UgQOIYcVyPK261JOlqWwOhmSuqZ3neJvU7FUPU5cEdXQQck4NetiRhm9JVE5Zq6KdfQrQbfRV5DKJhgKqXj5Ya8=; X-YMail-OSG: 1.lDoAUVM1nA2SUGbV0rwXijxm0uGNl3xGEcS6DE3RR8OsQ UZQ8zttMv84kNgoo0O1lKhnTS2_ILAF4avAYgH8nhv6l46TuvblqV4fpQZH0 V5x42GA0b7Q.7CxiWJsKMuwEuOHlsNZR7Puhq.LP0VQyfAQwHJZho7QzF3Tg ZK4wjWniu0PeE.sOCUR23_7rCE52nkwIujLCXYTxvLN7BBMARLRxQVN3s6vt He_BPdjfLZTPPyDqAUkTxC1zfzjNYhqn_XORgd0VuEn85rV4lyz7lniZXulw ImlHOkREmcvLdy8b4AMnyI_wheGVbQavY8Lf.39lajkMQjrcmWrjKDFAIA2S brRey9DuUDmyYs_qks0EQAGub5UAwBH_bJNzpN4_hw_76VM2N.QTbDdqmOUD .Zkj7qq5E0YrcYRGi5Uu_9Dd9vbGiRgl30_TJX4fO0mIT.KiAlyCMyxVLjUc xLvbm7ywJ4htggbLnsO93fGC3ZUfg9lMfX1yNGCEMwWxi3f9wCnSOWKWr8js T4eAuKIzWmx7TbGF3oaujsaasAXsmHeWcyhZ5yojPiEY- Received: from [121.204.194.110] by web164005.mail.gq1.yahoo.com via HTTP; Fri, 22 Nov 2013 07:27:19 PST X-Rocket-MIMEInfo: 002.001, SGkgTmVkLApUaGFua3MgdmVyeSBtdWNoIGZvciB5b3VyIGNvbW1lbnRzLgpBY3R1YWxseSAnZG9uZScgaXMgbm90IG5lY2Vzc2FyeSBhbmQgY2FuIGJlIGNvbXBsZXRlbHkgcmVtb3ZlZC4KVGhlIHZhbHVlIG9mICp3aWxsX3NwaWxsIGlzIGluaXRpYWxpemVkIG9ubHkgd2hlbiBidWZ0eXBlIGlzIFNBX0JPTlVTLCBzbyB0byByZW1vdmUgYnVmdHlwZSA9PSBTQV9CT05VUyBmcm9tIHRoZSBjaGVjaywgd2UgbmVlZCB0byBzZXQgaXQgdG8gRmFsc2UgZm9yIFNBX1NQSUxMIGJ1ZnR5cGUgYXMgd2VsbC4KCkkndmUBMAEBAQE- X-Mailer: YahooMailWebService/0.8.166.601 References: <1384949721.84395.YahooMailNeo@web164006.mail.gq1.yahoo.com> <20131120204302.GC9676@llnl.gov> Message-ID: <1385134039.58782.YahooMailNeo@web164005.mail.gq1.yahoo.com> Date: Fri, 22 Nov 2013 07:27:19 -0800 (PST) From: James Pan Subject: Re: [OpenZFS Developer] a ZFS SA bug and my patch To: Ned Bass In-Reply-To: <20131120204302.GC9676@llnl.gov> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="-1802851953-218250990-1385134039=:58782" X-Content-Filtered-By: Mailman/MimeDel 2.1.16 Cc: "developer@open-zfs.org" , "zfs-devel@freebsd.org" X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.16 Precedence: list Reply-To: James Pan List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 22 Nov 2013 15:27:26 -0000 ---1802851953-218250990-1385134039=:58782 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable Hi Ned,=0AThanks very much for your comments.=0AActually 'done' is not nece= ssary and can be completely removed.=0AThe value of *will_spill is initiali= zed only when buftype is SA_BONUS, so to remove buftype =3D=3D SA_BONUS fro= m the check, we need to set it to False for SA_SPILL buftype as well.=0A=0A= I've revised the patch to reflect these changes, could you help review it a= gain and get it checked in to the main branch if it is OK?=0AThanks a lot.= =0A=0AJames Pan=0A=0A=0A=0AOn Thursday, November 21, 2013 4:43 AM, Ned Bass= wrote:=0A =0AOn Wed, Nov 20, 2013 at 04:15:21AM -0800, Ja= mes Pan wrote:=0A> Hi,=0A> I hit a ZFS SA problem on FreeBSD 9.2, but I bel= ieve the issue exists on other=0A> platform too. Here is the description of= the bug.=0A> =0A> =0A> PROBLEM:=0A> run the attached script on a ZFS, afte= r a few seconds, run zdb -vvv on the ZFS,=0A> zdb will crash at the followi= ng assertion:=0A> =0A> Assertion failed: (IS_SA_BONUSTYPE(bonustype) && SA_= HDR_SIZE_MATCH_LAYOUT(hdr,=0A> tb) || !IS_SA_BONUSTYPE(bonustype) || (IS_SA= _BONUSTYPE(bonustype) && hdr->=0A> sa_layout_info =3D=3D 0)), file /usr/src= /cddl/lib/libzpool/../../../sys/cddl/=0A> contrib/opensolaris/uts/common/fs= /zfs/sa.c, line 1509.=0A> Abort (core dumped)=0A> =0A> the reason is the SA= 's header size does not match its layout.=0A> =0A> =0A> ROOT CAUSE:=0A> The= issue will be hit when a file has more than 2 variable-length SA and the= =0A> total SA size is larger than the bonus buffer's length -=A0 sizeof (bl= kptr_t),=0A> but less the bonus buffer's length.=0A> =0A> in sa_find_sizes(= ), done is set to TRUE if the SA size + header > the bonus=0A> buffer's len= gth - sizeof (blkptr_t), then hdrsize +=3D sizeof (uint16_t) will be=0A> sk= ipped for the second variable-length SA. If finally all SA can fit in the= =0A> bonus buffer and no spill block is needed, we will get a wrong hdrsize= .=0A> =0A> MY FIX:=0A> I've also attached my simple fix for this issue, any= one who might have interest=0A> could you please take a look? Thanks a lot!= =0A> =0A=0AGood catch.=A0 I'm responsible for the first attempt to correct = hdrsize in=0Athe case of spill-over with variable-sized SAs.=A0 But I clear= ly didn't=0Aget it quite right.=0A=0AI haven't tested your patch, but it lo= oks correct to me.=0A=0A+=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 = =A0 =A0 =A0 if (done)=0A+=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 = =A0 =A0 =A0 =A0 =A0 =A0 =A0 extra_hdrsize +=3D sizeof (uint16_t);=0A=A0 = =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 } else {=0A-=A0 =A0 =A0 =A0 =A0= =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 done =3D B_TRUE;=0A-=A0 =A0 =A0 = =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 *index =3D i;=0A+=A0 =A0 = =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 if (!done) {=0A+=A0 = =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 do= ne =3D B_TRUE;=0A+=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 = =A0 =A0 =A0 =A0 =A0 *index =3D i;=0A+=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 = =A0 =A0 =A0 =A0 =A0 =A0 }=0A=0AIt's not very obvious to the casual observe= r what we're "done" with when=0Adone =3D=3D B_TRUE.=A0 It seems to mean we'= ve found the index of the first SA=0Athat will spill over _if_ we need to u= se a spill buffer.=A0 While we're=0Arevising this code, this could be made = much more clear with a comment=0Aand a self-documenting name (spill_index_f= ound, perhaps?).=0A=0A=0A+=A0 =A0 =A0 if (buftype =3D=3D SA_BONUS && *will= _spill)=0A+=A0 =A0 =A0 =A0 =A0 =A0 =A0 hdrsize -=3D extra_hdrsize;=0A=0A*w= ill_spill implies buftype =3D=3D SA_BONUS. This doesn't hurt to=0Adouble-ch= eck, but we could make it more explicit with an assertion:=0A=0A=A0=A0=A0 i= f (*will_spill) {=0A=A0=A0=A0 =A0=A0=A0 ASSERT3U(buftype, =3D=3D, SA_BONUS)= ;=0A=A0=A0=A0 =A0=A0=A0 hdrsize -=3D extra_hdrsize;=0A=A0=A0=A0 }=0A=0ANed= =0A=0A_______________________________________________=0Adeveloper mailing l= ist=0Adeveloper@open-zfs.org=0Ahttp://lists.open-zfs.org/mailman/listinfo/d= eveloper ---1802851953-218250990-1385134039=:58782 Content-Type: application/octet-stream; name="sa.c.diff2" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="sa.c.diff2" LS0tIHNhLmMub3JpZwkyMDEzLTA5LTI3IDA5OjEwOjI5LjAwMDAwMDAwMCAr MDgwMAorKysgc2EuYwkyMDEzLTExLTIxIDIxOjUwOjMwLjAwMDAwMDAwMCAr MDgwMApAQCAtNTUyLDEwICs1NTIsOSBAQAogewogCWludCB2YXJfc2l6ZSA9 IDA7CiAJaW50IGk7Ci0JaW50IGogPSAtMTsKIAlpbnQgZnVsbF9zcGFjZTsK IAlpbnQgaGRyc2l6ZTsKLQlib29sZWFuX3QgZG9uZSA9IEJfRkFMU0U7CisJ aW50IGV4dHJhX2hkcnNpemU7CiAKIAlpZiAoYnVmdHlwZSA9PSBTQV9CT05V UyAmJiBzYS0+c2FfZm9yY2Vfc3BpbGwpIHsKIAkJKnRvdGFsID0gMDsKQEAg LTU2NiwxMCArNTY1LDkgQEAKIAogCSppbmRleCA9IC0xOwogCSp0b3RhbCA9 IDA7CisJKndpbGxfc3BpbGwgPSBCX0ZBTFNFOwogCi0JaWYgKGJ1ZnR5cGUg PT0gU0FfQk9OVVMpCi0JCSp3aWxsX3NwaWxsID0gQl9GQUxTRTsKLQorCWV4 dHJhX2hkcnNpemUgPSAwOwogCWhkcnNpemUgPSAoU0FfQk9OVVNUWVBFX0ZS T01fREIoZGIpID09IERNVV9PVF9aTk9ERSkgPyAwIDoKIAkgICAgc2l6ZW9m IChzYV9oZHJfcGh5c190KTsKIApAQCAtNTgxLDggKzU3OSwxMCBAQAogCiAJ CSp0b3RhbCA9IFAyUk9VTkRVUCgqdG90YWwsIDgpOwogCQkqdG90YWwgKz0g YXR0cl9kZXNjW2ldLnNhX2xlbmd0aDsKLQkJaWYgKGRvbmUpCi0JCQlnb3Rv IG5leHQ7CisKKwkJaWYgKCp3aWxsX3NwaWxsKSB7CisJCQljb250aW51ZTsK KwkJfQogCiAJCWlzX3Zhcl9zeiA9IChTQV9SRUdJU1RFUkVEX0xFTihzYSwg YXR0cl9kZXNjW2ldLnNhX2F0dHIpID09IDApOwogCQlpZiAoaXNfdmFyX3N6 KSB7CkBAIC01OTAsMjEgKzU5MCwyOCBAQAogCQl9CiAKIAkJaWYgKGlzX3Zh cl9zeiAmJiB2YXJfc2l6ZSA+IDEpIHsKLQkJCWlmIChQMlJPVU5EVVAoaGRy c2l6ZSArIHNpemVvZiAodWludDE2X3QpLCA4KSArCisJCQkvKiAKKwkJCSAq IERvbid0IHdvcnJ5IHRoYXQgdGhlIHNwaWxsIGJsb2NrIHdpbGwgZ2V0IG92 ZXJmbG93ZWQsCisgICAgICAgICAgICAgICAgICAgICAgICAgKiB3ZSB3aWxs IHJlc2l6ZSBpdCBsYXRlciBpbiBzYV9idWlsZF9sYXlvdXRzKCkuCisJCQkg Ki8KKwkJCWlmIChidWZ0eXBlID09IFNBX1NQSUxMIHx8IAorCQkJICAgIFAy Uk9VTkRVUChoZHJzaXplICsgc2l6ZW9mICh1aW50MTZfdCksIDgpICsKIAkJ CSAgICAqdG90YWwgPCBmdWxsX3NwYWNlKSB7CiAJCQkJLyoKIAkJCQkgKiBB Y2NvdW50IGZvciBoZWFkZXIgc3BhY2UgdXNlZCBieSBhcnJheSBvZgogCQkJ CSAqIG9wdGlvbmFsIHNpemVzIG9mIHZhcmlhYmxlLWxlbmd0aCBhdHRyaWJ1 dGVzLgotCQkJCSAqIFJlY29yZCB0aGUgaW5kZXggaW4gY2FzZSB0aGlzIGlu Y3JlYXNlIG5lZWRzCisJCQkJICogUmVjb3JkIHRoZSBleHRyYSBoZWFkZXIg c2l6ZSBpbiBjYXNlIHRoaXMgaW5jcmVhc2UgbmVlZHMKIAkJCQkgKiB0byBi ZSByZXZlcnNlZCBkdWUgdG8gc3BpbGwtb3Zlci4KIAkJCQkgKi8KIAkJCQlo ZHJzaXplICs9IHNpemVvZiAodWludDE2X3QpOwotCQkJCWogPSBpOworCQkJ CWlmICgqaW5kZXggIT0gLTEpCisJCQkJCWV4dHJhX2hkcnNpemUgKz0gc2l6 ZW9mICh1aW50MTZfdCk7CiAJCQl9IGVsc2UgewotCQkJCWRvbmUgPSBCX1RS VUU7Ci0JCQkJKmluZGV4ID0gaTsKLQkJCQlpZiAoYnVmdHlwZSA9PSBTQV9C T05VUykKLQkJCQkJKndpbGxfc3BpbGwgPSBCX1RSVUU7CisJCQkJQVNTRVJU KGJ1ZnR5cGUgPT0gU0FfQk9OVVMpOworCQkJCWlmICgqaW5kZXggPT0gLTEp IHsKKwkJCQkJKmluZGV4ID0gaTsKKwkJCQl9CisJCQkJKndpbGxfc3BpbGwg PSBCX1RSVUU7CiAJCQkJY29udGludWU7CiAJCQl9CiAJCX0KQEAgLTYxOSwy MiArNjI2LDE1IEBACiAJCSAgICAoKnRvdGFsICsgUDJST1VORFVQKGhkcnNp emUsIDgpKSA+CiAJCSAgICAoZnVsbF9zcGFjZSAtIHNpemVvZiAoYmxrcHRy X3QpKSkgewogCQkJKmluZGV4ID0gaTsKLQkJCWRvbmUgPSBCX1RSVUU7CiAJ CX0KIAotbmV4dDoKIAkJaWYgKCgqdG90YWwgKyBQMlJPVU5EVVAoaGRyc2l6 ZSwgOCkpID4gZnVsbF9zcGFjZSAmJgogCQkgICAgYnVmdHlwZSA9PSBTQV9C T05VUykKIAkJCSp3aWxsX3NwaWxsID0gQl9UUlVFOwogCX0KIAotCS8qCi0J ICogaiBob2xkcyB0aGUgaW5kZXggb2YgdGhlIGxhc3QgdmFyaWFibGUtc2l6 ZWQgYXR0cmlidXRlIGZvcgotCSAqIHdoaWNoIGhkcnNpemUgd2FzIGluY3Jl YXNlZC4gIFJldmVyc2UgdGhlIGluY3JlYXNlIGlmIHRoYXQKLQkgKiBhdHRy aWJ1dGUgd2lsbCBiZSByZWxvY2F0ZWQgdG8gdGhlIHNwaWxsIGJsb2NrLgot CSAqLwotCWlmICgqd2lsbF9zcGlsbCAmJiBqID09ICppbmRleCkKLQkJaGRy c2l6ZSAtPSBzaXplb2YgKHVpbnQxNl90KTsKKwlpZiAoKndpbGxfc3BpbGwp CisJCWhkcnNpemUgLT0gZXh0cmFfaGRyc2l6ZTsKIAogCWhkcnNpemUgPSBQ MlJPVU5EVVAoaGRyc2l6ZSwgOCk7CiAJcmV0dXJuIChoZHJzaXplKTsK ---1802851953-218250990-1385134039=:58782-- From owner-zfs-devel@FreeBSD.ORG Fri Nov 22 18:07:59 2013 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 98B06985 for ; Fri, 22 Nov 2013 18:07:59 +0000 (UTC) Received: from prdiron-3.llnl.gov (prdiron-3.llnl.gov [128.15.143.173]) by mx1.freebsd.org (Postfix) with ESMTP id 7D135247B for ; Fri, 22 Nov 2013 18:07:59 +0000 (UTC) X-Attachments: Received: from eris.llnl.gov ([128.115.7.7]) by prdiron-3.llnl.gov with ESMTP; 22 Nov 2013 10:06:51 -0800 Received: from twist.chaos (twist.chaos [192.168.1.126]) by eris.llnl.gov (Postfix) with ESMTP id 26F687C4A5; Fri, 22 Nov 2013 10:06:51 -0800 (PST) Received: from twist.chaos (localhost [127.0.0.1]) by twist.chaos (8.14.4/8.14.4/Debian-2ubuntu2) with ESMTP id rAMI77wG041786; Fri, 22 Nov 2013 10:07:07 -0800 Received: (from bass6@localhost) by twist.chaos (8.14.4/8.14.4/Submit) id rAMI76rc041780; Fri, 22 Nov 2013 10:07:06 -0800 X-Authentication-Warning: twist.chaos: bass6 set sender to bass6@llnl.gov using -f Date: Fri, 22 Nov 2013 10:07:06 -0800 From: Ned Bass To: James Pan Subject: Re: [OpenZFS Developer] a ZFS SA bug and my patch Message-ID: <20131122180706.GB30149@llnl.gov> References: <1384949721.84395.YahooMailNeo@web164006.mail.gq1.yahoo.com> <20131120204302.GC9676@llnl.gov> <1385134039.58782.YahooMailNeo@web164005.mail.gq1.yahoo.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1385134039.58782.YahooMailNeo@web164005.mail.gq1.yahoo.com> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: "developer@open-zfs.org" , "zfs-devel@freebsd.org" X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.16 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 22 Nov 2013 18:07:59 -0000 Hi James, On Fri, Nov 22, 2013 at 07:27:19AM -0800, James Pan wrote: > Hi Ned, > Thanks very much for your comments. > Actually 'done' is not necessary and can be completely removed. > The value of *will_spill is initialized only when buftype is SA_BONUS, so to > remove buftype == SA_BONUS from the check, we need to set it to False for > SA_SPILL buftype as well. > > I've revised the patch to reflect these changes, could you help review it again > and get it checked in to the main branch if it is OK? > Thanks a lot. It looks good to me, except for a couple of formatting issues noted below. We'll get it tested in the Linux port and post results. I'm not an upstream ZFS maintainer so I can't check it in. This is a link to the official integration process, but for now you probably just need to stand by for responses from the senior Illumos folks. http://open-zfs.org/wiki/Illumos_integration_process + /* + * Don't worry that the spill block will get overflowed, + * we will resize it later in sa_build_layouts(). + */ Remove trailing whitespace and align *. + if (buftype == SA_SPILL || Remove trailing whitespace. + * Record the extra header size in case this increase needs Wrap line to 80 columns. Thanks, Ned From owner-zfs-devel@FreeBSD.ORG Wed Dec 4 20:38:08 2013 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A0F3C46C; Wed, 4 Dec 2013 20:38:08 +0000 (UTC) Received: from anubis.delphij.net (anubis.delphij.net [IPv6:2001:470:1:117::25]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 7FFB61F8E; Wed, 4 Dec 2013 20:38:08 +0000 (UTC) Received: from zeta.ixsystems.com (unknown [69.198.165.132]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by anubis.delphij.net (Postfix) with ESMTPSA id E8E0926AD5; Wed, 4 Dec 2013 12:38:07 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=delphij.net; s=anubis; t=1386189488; bh=GlF4HUUNNNW86NpbUXim5qNB53g8lW/PK1H9OLuQsxg=; h=Date:From:Reply-To:To:Subject; b=OHYEvyAi4f8y8BLYquZpj7kY/Va8ChTLTLdBZjQoYl4bBO0h9TLaRB8Q++ewkEd8G KQ3JrECoy9IiVNNdxNRZSI8h0XBKNxuX0kEVVepOE+pLQhacNuensgxqEXfT/zFw+p cZKzSBiyetipKslxHI4E2Tvun9RNDSyr/RykouGA= Message-ID: <529F92AC.6010100@delphij.net> Date: Wed, 04 Dec 2013 12:38:04 -0800 From: Xin Li Organization: The FreeBSD Project MIME-Version: 1.0 To: Andriy Gapon , "zfs-devel@freebsd.org" Subject: [PATCH] Expose spa_ashift_inflation X-Enigmail-Version: 1.6 Content-Type: multipart/mixed; boundary="------------070603050508000403010201" X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list Reply-To: d@delphij.net List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 04 Dec 2013 20:38:08 -0000 This is a multi-part message in MIME format. --------------070603050508000403010201 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 Hi, This is a trivial patch, tested against -CURRENT. Or, is this intentionally hidden? (I think we still want to expose it somewhere as we don't have mdb equivalent to provide access). If there is no objections by Friday I'll commit this change against -HEAD. Cheers, - -- Xin LI https://www.delphij.net/ FreeBSD - The Power to Serve! Live free or die -----BEGIN PGP SIGNATURE----- iQIcBAEBCgAGBQJSn5KsAAoJEJW2GBstM+nsWawP/3h/N6DcE7Akyr1aulLGFddb VW3MO2B8uSLaAIN+W/fnHND5LtdBzwUxcW0QGxapJXc1Dz+XCo0azQi20a1qFbQT fdterlbl/ttEvkSwR3xA+7HEIwUWOWgFKi9VRdwumItMzQORG7FRzD9rSlihh1Tt X1pFSuRSbi0fzc31YAwPxAvBkZm2booe0oVhLteB+3M9Dbef/L7kJUGErnRUDSUP fVCOqVORK0R7ECGxxPtH7DWoAeBxsxN5ei/snuB7BUkaB8Yr+Oliuz5H95Gd0/tj b57LhztnfYdFL8EfMIfHBvg48uV2PTn9uHs6HgVFA6Ex3v8VrT7+ZKfL5JimPUr3 HRsWTw/TV4Ohld9z0kLCaS5uhnlw3y8dgUbL0gDFcflcfhGL3wXLtjJ4wemVj6T6 yZ1nj22pT6lqVdp6cTuSH3x6zvGyt9EQZdTKQyo4z0ZC3X4YUJKKWJgmGm+Q4pxT tFRcDZJ8taM0qS0X5xc6bjukHVbCmkSUOSlT4uYwM8gcWshL5mvABJ4SbKfLfEdg d9upk1zkgKFrdHntdi+rZCzAWEMYyrWi5uueb7zfanKlxIYgqwl21XQDBtpNIen2 JO2c8u9GZ4t3+vyu5e0MsajOf3Tcd4lcc/WBuRZTqwTCQygynIsMMU2INszFv/fx LceIEh2XdQWE6juilmPi =Ousq -----END PGP SIGNATURE----- --------------070603050508000403010201 Content-Type: text/plain; charset=UTF-8; name="spa-asize-inflation.diff" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="spa-asize-inflation.diff" Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa_misc.c =================================================================== --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa_misc.c (revision 258941) +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa_misc.c (working copy) @@ -302,6 +302,9 @@ SYSCTL_INT(_vfs_zfs, OID_AUTO, deadman_enabled, CT * (VDEV_RAIDZ_MAXPARITY + 1) * SPA_DVAS_PER_BP * 2 == 24 */ int spa_asize_inflation = 24; +TUNABLE_INT("vfs.zfs.spa_asize_inflation", &spa_asize_inflation); +SYSCTL_INT(_vfs_zfs, OID_AUTO, spa_asize_inflation, CTLFLAG_RWTUN, + &spa_asize_inflation, 0, "Worst case inflation factor for single sector writes"); #ifndef illumos #ifdef _KERNEL --------------070603050508000403010201-- From owner-zfs-devel@FreeBSD.ORG Wed Dec 11 01:17:08 2013 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C22B8D93 for ; Wed, 11 Dec 2013 01:17:08 +0000 (UTC) Received: from prdiron-3.llnl.gov (prdiron-3.llnl.gov [128.15.143.173]) by mx1.freebsd.org (Postfix) with ESMTP id A49DD124C for ; Wed, 11 Dec 2013 01:17:08 +0000 (UTC) X-Attachments: Received: from eris.llnl.gov ([128.115.7.7]) by prdiron-3.llnl.gov with ESMTP; 10 Dec 2013 17:15:58 -0800 Received: from twist.chaos (twist.chaos [192.168.1.126]) by eris.llnl.gov (Postfix) with ESMTP id EFF5B7C4A5; Tue, 10 Dec 2013 17:15:57 -0800 (PST) Received: from twist.chaos (localhost [127.0.0.1]) by twist.chaos (8.14.4/8.14.4/Debian-2ubuntu2) with ESMTP id rBB1GWm3034162; Tue, 10 Dec 2013 17:16:32 -0800 Received: (from bass6@localhost) by twist.chaos (8.14.4/8.14.4/Submit) id rBB1GWle034160; Tue, 10 Dec 2013 17:16:32 -0800 X-Authentication-Warning: twist.chaos: bass6 set sender to bass6@llnl.gov using -f Date: Tue, 10 Dec 2013 17:16:32 -0800 From: Ned Bass To: James Pan Subject: Re: [OpenZFS Developer] a ZFS SA bug and my patch Message-ID: <20131211011632.GG9676@llnl.gov> References: <1384949721.84395.YahooMailNeo@web164006.mail.gq1.yahoo.com> <20131120204302.GC9676@llnl.gov> <1385134039.58782.YahooMailNeo@web164005.mail.gq1.yahoo.com> <20131122180706.GB30149@llnl.gov> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20131122180706.GB30149@llnl.gov> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: "developer@open-zfs.org" , "zfs-devel@freebsd.org" X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 Dec 2013 01:17:08 -0000 On Fri, Nov 22, 2013 at 10:07:06AM -0800, Ned Bass wrote: > Hi James, > > On Fri, Nov 22, 2013 at 07:27:19AM -0800, James Pan wrote: > > Hi Ned, > > Thanks very much for your comments. > > Actually 'done' is not necessary and can be completely removed. > > The value of *will_spill is initialized only when buftype is SA_BONUS, so to > > remove buftype == SA_BONUS from the check, we need to set it to False for > > SA_SPILL buftype as well. > > > > I've revised the patch to reflect these changes, could you help review it again > > and get it checked in to the main branch if it is OK? > > Thanks a lot. > > It looks good to me, except for a couple of formatting issues noted > below. We'll get it tested in the Linux port and post results. This patch checked out well in testing, and it's been merged in the Linux port: https://github.com/zfsonlinux/zfs/commit/472e7c6 I recommend that it be considered for inclusion upstream. Ned From owner-zfs-devel@FreeBSD.ORG Thu Dec 19 11:31:34 2013 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B429AB12; Thu, 19 Dec 2013 11:31:34 +0000 (UTC) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id D57071628; Thu, 19 Dec 2013 11:31:33 +0000 (UTC) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id NAA09033; Thu, 19 Dec 2013 13:31:32 +0200 (EET) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1VtbpD-00007h-OC; Thu, 19 Dec 2013 13:31:31 +0200 Message-ID: <52B2D8D6.8090306@FreeBSD.org> Date: Thu, 19 Dec 2013 13:30:30 +0200 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.2.0 MIME-Version: 1.0 To: freebsd-fs Subject: l2arc_feed_thread cpu utlization X-Enigmail-Version: 1.6 Content-Type: text/plain; charset=X-VIET-VPS Content-Transfer-Encoding: 7bit X-Mailman-Approved-At: Thu, 19 Dec 2013 12:26:38 +0000 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Dec 2013 11:31:34 -0000 This is just a heads up, no patch yet. l2arc_feed_thread periodically wakes up and scans certain amount of ARC buffers and writes eligible buffers to a cache device. Number of scanned buffers is limited by a threshold on the amount of data in the buffers seen. The threshold is applied on a per buffer list basis. In upstream there are 4 relevant lists: (data, metadata) X (MFU, MRU). In FreeBSD each of the lists was subdivided into 16 lists. This was done to reduce contention on the locks that protect the lists. But as a side effect l2arc_feed_thread can scan 16 times more data (~ buffers). So, if you have a rather large ARC and L2ARC and your buffers tend to be sufficiently small, then you could observe l2arc_feed_thread burning a noticeable amount of CPU. On some of our systems I observed it using up to 40% of a single core. Scaling back the threshold by factor of 16 makes CPU utilization go down by the same factor. I plan to commit this change to FreeBSD ZFS code. Any comments are welcome. -- Andriy Gapon From owner-zfs-devel@FreeBSD.ORG Fri Dec 20 13:41:06 2013 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0F977E21 for ; Fri, 20 Dec 2013 13:41:06 +0000 (UTC) Received: from mail.dawidek.net (garage.dawidek.net [91.121.88.72]) by mx1.freebsd.org (Postfix) with ESMTP id 2C3B415D5 for ; Fri, 20 Dec 2013 13:41:04 +0000 (UTC) Received: from localhost (58.wheelsystems.com [83.12.187.58]) by mail.dawidek.net (Postfix) with ESMTPSA id 2CD1569B for ; Fri, 20 Dec 2013 14:34:11 +0100 (CET) Date: Fri, 20 Dec 2013 14:41:37 +0100 From: Pawel Jakub Dawidek To: zfs-devel@FreeBSD.org Subject: [krichy@tvnetwork.hu: Re: kern/184677 / ZFS snapshot handling deadlocks] Message-ID: <20131220134136.GD1658@garage.freebsd.pl> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="MnLPg7ZWsaic7Fhd" Content-Disposition: inline X-OS: FreeBSD 11.0-CURRENT amd64 User-Agent: Mutt/1.5.22 (2013-10-16) X-Content-Filtered-By: Mailman/MimeDel 2.1.17 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Dec 2013 13:41:06 -0000 --MnLPg7ZWsaic7Fhd Content-Type: multipart/mixed; boundary="0ntfKIWw70PvrIHh" Content-Disposition: inline --0ntfKIWw70PvrIHh Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Guys, can someone please look into this report? It is very detailed and even have a patch. I can't get to it right now, but I also remember someone was going to rework the locking in vdev_geom.c. Did that happen? Maybe some regression was introduced? --=20 Pawel Jakub Dawidek http://www.wheelsystems.com FreeBSD committer http://www.FreeBSD.org Am I Evil? Yes, I Am! http://mobter.com --0ntfKIWw70PvrIHh Content-Type: message/rfc822 Content-Disposition: inline Return-Path: X-Original-To: pawel@dawidek.net Delivered-To: pawel@dawidek.net Received: from krichy.tvnetwork.hu (krichy.tvnetwork.hu [109.61.101.194]) by mail.dawidek.net (Postfix) with ESMTP id F374331D for ; Thu, 19 Dec 2013 16:39:47 +0100 (CET) Received: by krichy.tvnetwork.hu (Postfix, from userid 1000) id 7147F7917; Thu, 19 Dec 2013 16:46:11 +0100 (CET) Received: from localhost (localhost [127.0.0.1]) by krichy.tvnetwork.hu (Postfix) with ESMTP id 6E4447916; Thu, 19 Dec 2013 16:46:11 +0100 (CET) Date: Thu, 19 Dec 2013 16:46:11 +0100 (CET) From: krichy@tvnetwork.hu To: freebsd-fs@freebsd.org cc: pawel@dawidek.net Subject: Re: kern/184677 / ZFS snapshot handling deadlocks In-Reply-To: Message-ID: References: User-Agent: Alpine 2.10 (DEB 1266 2009-07-14) MIME-Version: 1.0 Content-Type: MULTIPART/MIXED; BOUNDARY="1030603365-686922855-1387467971=:4344" This message is in MIME format. The first part should be readable text, while the remaining parts are likely unreadable without MIME-aware tools. --1030603365-686922855-1387467971=:4344 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Dear devs, I am attaching a more clear patch/fix for my snapshot handling issues (0002), but I would be happy if some ZFS expert would comment it. I am trying to solve it at least for two weeks now, and an ACK or a NACK would be nice from someone. Also a commit is reverted since it also caused deadlocks. I've read its comments, which also eliminates deadlocks, but I did not find any reference how to produce that deadlock. In my view reverting that makes my issues disappear, but I dont know what new cases will it rise. I've rewritten traverse() to make more like upstream, added two extra VN_HOLD()s to snapdir_lookup() when traverse returned the same vnode what was passed to it (I dont know even that upon creating a snapshot vnode why is that extra two holds needed for the vnode.) And unfortunately, due to FreeBSD calls vop_inactive callbacks with vnodes locked, that could also cause deadlocks, so zfsctl_snapshot_inactive() and zfsctl_snapshot_vptocnp() has been rewritten to work that around. After this, one may also get a deadlock, when a simple access would call into zfsctl_snapshot_lookup(). The documentation says, that those vnodes should always be covered, but after some stress test, sometimes we hit that call, and that can cause again deadlocks. In our environment I've just uncommented that callback, which returns ENODIR on some calls, but at least not a deadlock. The attached script can be used to reproduce my cases (would one confirm that?), and after the patches applied, they disappear (confirm?). Thanks, Kojedzinszky Richard Euronet Magyarorszag Informatikai Zrt. On Tue, 17 Dec 2013, krichy@tvnetwork.hu wrote: > Date: Tue, 17 Dec 2013 14:50:16 +0100 (CET) > From: krichy@tvnetwork.hu > To: pjd@freebsd.org > Cc: freebsd-fs@freebsd.org > Subject: Re: kern/184677 (fwd) > > Dear devs, > > I will sum up my experience regarding the issue: > > The sympton is that a concurrent 'zfs send -R' and some activity on the > snapshot dir (or in the snapshot) may cause a deadlock. > > After investigating the problem, I found that zfs send umounts the snapshots, > and that causes the deadlock, so later I tested only with concurrent umount > and the "activity". More later I found that listing the snapshots in > .zfs/snapshot/ and unounting them can cause the found deadlock, so I used > them for the tests. But for my surprise, instead of a deadlock, a recursive > lock panic has arised. > > The vnode for the ".zfs/snapshot/" directory contains zfs's zfsctl_snapdir_t > structure (sdp). This contains a tree of mounted snapshots, and each entry > (sep) contains the vnode of entry on which the snapshot is mounted on top > (se_root). The strange is that the se_root member does not hold a reference > for the vnode, just a simple pointer to it. > > Upon entry lookup (zfsctl_snapdir_lookup()) the "snapshot" vnode is locked, > the zfsctl_snapdir_t's tree is locked, and searched for the mount if it > exists already. If it founds no entry, does the mount. In the case of an > entry was found, the se_root member contains the vnode which the snapshot is > mounted on. Thus, a reference is taken for it, and the traverse() call will > resolve to the real root vnode of the mounted snapshot, returning it as > locked. (Examining the traverse() code I've found that it did not follow > FreeBSD's lock order recommendation described in sys/kern/vfs_subr.c.) > > On the other way, when an umount is issued, the se_root vnode looses its last > reference (as only the mountpoint holds one for it), it goes through the > vinactive() path, to zfsctl_snapshot_inactive(). In FreeBSD this is called > with a locked vnode, so this is a deadlock race condition. While > zfsctl_snapdir_lookup() holds the mutex for the sdp tree, and traverse() > tries to acquire the se_root, zfsctl_snapshot_inactive() holds the lock on > se_root while tries to access the sdp lock. > > The zfsctl_snapshot_inactive() has an if statement checking the v_usecount, > which is incremented in zfsctl_snapdir_lookup(), but in that context it is > not covered by VI_LOCK. And it seems to me that FreeBSD's vinactive() path > assumes that the vnode remains inactive (as opposed to illumos, at least how > i read the code). So zfsctl_snapshot_inactive() must free resources while in > a locked state. I was a bit confused, and probably that is why the previously > posted patch is as is. > > Maybe if I had some clues on the directions of this problem, I could have > worked more for a nicer, shorter solution. > > Please someone comment on my post. > > Regards, > > > > Kojedzinszky Richard > Euronet Magyarorszag Informatikai Zrt. > > On Mon, 16 Dec 2013, krichy@tvnetwork.hu wrote: > >> Date: Mon, 16 Dec 2013 16:52:16 +0100 (CET) >> From: krichy@tvnetwork.hu >> To: pjd@freebsd.org >> Cc: freebsd-fs@freebsd.org >> Subject: Re: kern/184677 (fwd) >> >> Dear PJD, >> >> I am a happy FreeBSD user, I am sure you've read my previous posts >> regarding some issues in ZFS. Please give some advice for me, where to look >> for solutions, or how could I help to resolve those issues. >> >> Regards, >> Kojedzinszky Richard >> Euronet Magyarorszag Informatikai Zrt. >> >> ---------- Forwarded message ---------- >> Date: Mon, 16 Dec 2013 15:23:06 +0100 (CET) >> From: krichy@tvnetwork.hu >> To: freebsd-fs@freebsd.org >> Subject: Re: kern/184677 >> >> >> Seems that pjd did a change which eliminated the zfsdev_state_lock on Fri >> Aug 12 07:04:16 2011 +0000, which might introduced a new deadlock >> situation. Any comments on this? >> >> >> Kojedzinszky Richard >> Euronet Magyarorszag Informatikai Zrt. >> >> On Mon, 16 Dec 2013, krichy@tvnetwork.hu wrote: >> >>> Date: Mon, 16 Dec 2013 11:08:11 +0100 (CET) >>> From: krichy@tvnetwork.hu >>> To: freebsd-fs@freebsd.org >>> Subject: kern/184677 >>> >>> Dear devs, >>> >>> I've attached a patch, which makes the recursive lockmgr disappear, and >>> makes the reported bug to disappear. I dont know if I followed any >>> guidelines well, or not, but at least it works for me. Please some >>> ZFS/FreeBSD fs expert review it, and fix it where it needed. >>> >>> But unfortunately, my original problem is still not solved, maybe the same >>> as Ryan's: >>> http://lists.freebsd.org/pipermail/freebsd-fs/2013-December/018707.html >>> >>> Tracing the problem down is that zfsctl_snapdir_lookup() tries to acquire >>> spa_namespace_lock while when finishing a zfs send -R does a >>> zfsdev_close(), and that also holds the same mutex. And this causes a >>> deadlock scenario. I looked at illumos's code, and for some reason they >>> use another mutex on zfsdev_close(), which therefore may not deadlock with >>> zfsctl_snapdir_lookup(). But I am still investigating the problem. >>> >>> I would like to help making ZFS more stable on freebsd also with its whole >>> functionality. I would be very thankful if some expert would give some >>> advice, how to solve these bugs. PJD, Steven, Xin? >>> >>> Thanks in advance, >>> >>> >>> Kojedzinszky Richard >>> Euronet Magyarorszag Informatikai Zrt. >> _______________________________________________ >> freebsd-fs@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-fs >> To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" >> > --1030603365-686922855-1387467971=:4344 Content-Type: TEXT/x-diff; name=0001-Revert-Eliminate-the-zfsdev_state_lock-entirely-and-.patch Content-Transfer-Encoding: BASE64 Content-ID: Content-Description: Content-Disposition: attachment; filename=0001-Revert-Eliminate-the-zfsdev_state_lock-entirely-and-.patch RnJvbSAzOTI5OGRhODM4ZDAwNmFkMjI1ZTQxNTI5ZDdiN2YyNDBmY2NmZTcz IE1vbiBTZXAgMTcgMDA6MDA6MDAgMjAwMQ0KRnJvbTogUmljaGFyZCBLb2pl ZHppbnN6a3kgPGtyaWNoeUBjZmxpbnV4Lmh1Pg0KRGF0ZTogTW9uLCAxNiBE ZWMgMjAxMyAxNTozOToxMSArMDEwMA0KU3ViamVjdDogW1BBVENIIDEvMl0g UmV2ZXJ0ICJFbGltaW5hdGUgdGhlIHpmc2Rldl9zdGF0ZV9sb2NrIGVudGly ZWx5IGFuZA0KIHJlcGxhY2UgaXQgd2l0aCB0aGUiDQoNClRoaXMgcmV2ZXJ0 cyBjb21taXQgMWQ4OTcyYjNmMzUzZjk4NmViNWI4NWJjMTA4YjFjMGQ5NDZk MzIxOC4NCg0KQ29uZmxpY3RzOg0KCXN5cy9jZGRsL2NvbnRyaWIvb3BlbnNv bGFyaXMvdXRzL2NvbW1vbi9mcy96ZnMvenZvbC5jDQotLS0NCiAuLi4vb3Bl bnNvbGFyaXMvdXRzL2NvbW1vbi9mcy96ZnMvc3lzL3pmc19pb2N0bC5oICB8 ICAgMSArDQogLi4uL29wZW5zb2xhcmlzL3V0cy9jb21tb24vZnMvemZzL3Zk ZXZfZ2VvbS5jICAgICAgfCAgMTQgKystDQogLi4uL29wZW5zb2xhcmlzL3V0 cy9jb21tb24vZnMvemZzL3pmc19pb2N0bC5jICAgICAgfCAgMTYgKy0tDQog Li4uL2NvbnRyaWIvb3BlbnNvbGFyaXMvdXRzL2NvbW1vbi9mcy96ZnMvenZv bC5jICAgfCAxMTkgKysrKysrKysrLS0tLS0tLS0tLS0tDQogNCBmaWxlcyBj aGFuZ2VkLCA3MCBpbnNlcnRpb25zKCspLCA4MCBkZWxldGlvbnMoLSkNCg0K ZGlmZiAtLWdpdCBhL3N5cy9jZGRsL2NvbnRyaWIvb3BlbnNvbGFyaXMvdXRz L2NvbW1vbi9mcy96ZnMvc3lzL3pmc19pb2N0bC5oIGIvc3lzL2NkZGwvY29u dHJpYi9vcGVuc29sYXJpcy91dHMvY29tbW9uL2ZzL3pmcy9zeXMvemZzX2lv Y3RsLmgNCmluZGV4IGFmMmRlZjIuLjgyNzJjNGQgMTAwNjQ0DQotLS0gYS9z eXMvY2RkbC9jb250cmliL29wZW5zb2xhcmlzL3V0cy9jb21tb24vZnMvemZz L3N5cy96ZnNfaW9jdGwuaA0KKysrIGIvc3lzL2NkZGwvY29udHJpYi9vcGVu c29sYXJpcy91dHMvY29tbW9uL2ZzL3pmcy9zeXMvemZzX2lvY3RsLmgNCkBA IC0zODMsNiArMzgzLDcgQEAgZXh0ZXJuIHZvaWQgKnpmc2Rldl9nZXRfc29m dF9zdGF0ZShtaW5vcl90IG1pbm9yLA0KIGV4dGVybiBtaW5vcl90IHpmc2Rl dl9taW5vcl9hbGxvYyh2b2lkKTsNCiANCiBleHRlcm4gdm9pZCAqemZzZGV2 X3N0YXRlOw0KK2V4dGVybiBrbXV0ZXhfdCB6ZnNkZXZfc3RhdGVfbG9jazsN CiANCiAjZW5kaWYJLyogX0tFUk5FTCAqLw0KIA0KZGlmZiAtLWdpdCBhL3N5 cy9jZGRsL2NvbnRyaWIvb3BlbnNvbGFyaXMvdXRzL2NvbW1vbi9mcy96ZnMv dmRldl9nZW9tLmMgYi9zeXMvY2RkbC9jb250cmliL29wZW5zb2xhcmlzL3V0 cy9jb21tb24vZnMvemZzL3ZkZXZfZ2VvbS5jDQppbmRleCAxNTY4NWE1Li41 YzNlOWYzIDEwMDY0NA0KLS0tIGEvc3lzL2NkZGwvY29udHJpYi9vcGVuc29s YXJpcy91dHMvY29tbW9uL2ZzL3pmcy92ZGV2X2dlb20uYw0KKysrIGIvc3lz L2NkZGwvY29udHJpYi9vcGVuc29sYXJpcy91dHMvY29tbW9uL2ZzL3pmcy92 ZGV2X2dlb20uYw0KQEAgLTU4MSw3ICs1ODEsNyBAQCB2ZGV2X2dlb21fb3Bl bih2ZGV2X3QgKnZkLCB1aW50NjRfdCAqcHNpemUsIHVpbnQ2NF90ICptYXhf cHNpemUsDQogCXN0cnVjdCBnX3Byb3ZpZGVyICpwcDsNCiAJc3RydWN0IGdf Y29uc3VtZXIgKmNwOw0KIAlzaXplX3QgYnVmc2l6ZTsNCi0JaW50IGVycm9y Ow0KKwlpbnQgZXJyb3IsIGxvY2s7DQogDQogCS8qDQogCSAqIFdlIG11c3Qg aGF2ZSBhIHBhdGhuYW1lLCBhbmQgaXQgbXVzdCBiZSBhYnNvbHV0ZS4NCkBA IC01OTMsNiArNTkzLDEyIEBAIHZkZXZfZ2VvbV9vcGVuKHZkZXZfdCAqdmQs IHVpbnQ2NF90ICpwc2l6ZSwgdWludDY0X3QgKm1heF9wc2l6ZSwNCiANCiAJ dmQtPnZkZXZfdHNkID0gTlVMTDsNCiANCisJaWYgKG11dGV4X293bmVkKCZz cGFfbmFtZXNwYWNlX2xvY2spKSB7DQorCQltdXRleF9leGl0KCZzcGFfbmFt ZXNwYWNlX2xvY2spOw0KKwkJbG9jayA9IDE7DQorCX0gZWxzZSB7DQorCQls b2NrID0gMDsNCisJfQ0KIAlEUk9QX0dJQU5UKCk7DQogCWdfdG9wb2xvZ3lf bG9jaygpOw0KIAllcnJvciA9IDA7DQpAQCAtNjI0LDcgKzYzMCwxMSBAQCB2 ZGV2X2dlb21fb3Blbih2ZGV2X3QgKnZkLCB1aW50NjRfdCAqcHNpemUsIHVp bnQ2NF90ICptYXhfcHNpemUsDQogCSAgICAhSVNQMihjcC0+cHJvdmlkZXIt PnNlY3RvcnNpemUpKSB7DQogCQlaRlNfTE9HKDEsICJQcm92aWRlciAlcyBo YXMgdW5zdXBwb3J0ZWQgc2VjdG9yc2l6ZS4iLA0KIAkJICAgIHZkLT52ZGV2 X3BhdGgpOw0KKw0KKwkJZ190b3BvbG9neV9sb2NrKCk7DQogCQl2ZGV2X2dl b21fZGV0YWNoKGNwLCAwKTsNCisJCWdfdG9wb2xvZ3lfdW5sb2NrKCk7DQor DQogCQllcnJvciA9IEVJTlZBTDsNCiAJCWNwID0gTlVMTDsNCiAJfSBlbHNl IGlmIChjcC0+YWN3ID09IDAgJiYgKHNwYV9tb2RlKHZkLT52ZGV2X3NwYSkg JiBGV1JJVEUpICE9IDApIHsNCkBAIC02NDcsNiArNjU3LDggQEAgdmRldl9n ZW9tX29wZW4odmRldl90ICp2ZCwgdWludDY0X3QgKnBzaXplLCB1aW50NjRf dCAqbWF4X3BzaXplLA0KIAl9DQogCWdfdG9wb2xvZ3lfdW5sb2NrKCk7DQog CVBJQ0tVUF9HSUFOVCgpOw0KKwlpZiAobG9jaykNCisJCW11dGV4X2VudGVy KCZzcGFfbmFtZXNwYWNlX2xvY2spOw0KIAlpZiAoY3AgPT0gTlVMTCkgew0K IAkJdmQtPnZkZXZfc3RhdC52c19hdXggPSBWREVWX0FVWF9PUEVOX0ZBSUxF RDsNCiAJCXJldHVybiAoZXJyb3IpOw0KZGlmZiAtLWdpdCBhL3N5cy9jZGRs L2NvbnRyaWIvb3BlbnNvbGFyaXMvdXRzL2NvbW1vbi9mcy96ZnMvemZzX2lv Y3RsLmMgYi9zeXMvY2RkbC9jb250cmliL29wZW5zb2xhcmlzL3V0cy9jb21t b24vZnMvemZzL3pmc19pb2N0bC5jDQppbmRleCBlOWZiYTI2Li45MWJlY2Rl IDEwMDY0NA0KLS0tIGEvc3lzL2NkZGwvY29udHJpYi9vcGVuc29sYXJpcy91 dHMvY29tbW9uL2ZzL3pmcy96ZnNfaW9jdGwuYw0KKysrIGIvc3lzL2NkZGwv Y29udHJpYi9vcGVuc29sYXJpcy91dHMvY29tbW9uL2ZzL3pmcy96ZnNfaW9j dGwuYw0KQEAgLTU2MzUsNyArNTYzNSw3IEBAIHpmc2Rldl9taW5vcl9hbGxv Yyh2b2lkKQ0KIAlzdGF0aWMgbWlub3JfdCBsYXN0X21pbm9yOw0KIAltaW5v cl90IG07DQogDQotCUFTU0VSVChNVVRFWF9IRUxEKCZzcGFfbmFtZXNwYWNl X2xvY2spKTsNCisJQVNTRVJUKE1VVEVYX0hFTEQoJnpmc2Rldl9zdGF0ZV9s b2NrKSk7DQogDQogCWZvciAobSA9IGxhc3RfbWlub3IgKyAxOyBtICE9IGxh c3RfbWlub3I7IG0rKykgew0KIAkJaWYgKG0gPiBaRlNERVZfTUFYX01JTk9S KQ0KQEAgLTU2NTUsNyArNTY1NSw3IEBAIHpmc19jdGxkZXZfaW5pdChzdHJ1 Y3QgY2RldiAqZGV2cCkNCiAJbWlub3JfdCBtaW5vcjsNCiAJemZzX3NvZnRf c3RhdGVfdCAqenM7DQogDQotCUFTU0VSVChNVVRFWF9IRUxEKCZzcGFfbmFt ZXNwYWNlX2xvY2spKTsNCisJQVNTRVJUKE1VVEVYX0hFTEQoJnpmc2Rldl9z dGF0ZV9sb2NrKSk7DQogDQogCW1pbm9yID0gemZzZGV2X21pbm9yX2FsbG9j KCk7DQogCWlmIChtaW5vciA9PSAwKQ0KQEAgLTU2NzYsNyArNTY3Niw3IEBA IHpmc19jdGxkZXZfaW5pdChzdHJ1Y3QgY2RldiAqZGV2cCkNCiBzdGF0aWMg dm9pZA0KIHpmc19jdGxkZXZfZGVzdHJveSh6ZnNfb25leGl0X3QgKnpvLCBt aW5vcl90IG1pbm9yKQ0KIHsNCi0JQVNTRVJUKE1VVEVYX0hFTEQoJnNwYV9u YW1lc3BhY2VfbG9jaykpOw0KKwlBU1NFUlQoTVVURVhfSEVMRCgmemZzZGV2 X3N0YXRlX2xvY2spKTsNCiANCiAJemZzX29uZXhpdF9kZXN0cm95KHpvKTsN CiAJZGRpX3NvZnRfc3RhdGVfZnJlZSh6ZnNkZXZfc3RhdGUsIG1pbm9yKTsN CkBAIC01NzA2LDkgKzU3MDYsOSBAQCB6ZnNkZXZfb3BlbihzdHJ1Y3QgY2Rl diAqZGV2cCwgaW50IGZsYWcsIGludCBtb2RlLCBzdHJ1Y3QgdGhyZWFkICp0 ZCkNCiANCiAJLyogVGhpcyBpcyB0aGUgY29udHJvbCBkZXZpY2UuIEFsbG9j YXRlIGEgbmV3IG1pbm9yIGlmIHJlcXVlc3RlZC4gKi8NCiAJaWYgKGZsYWcg JiBGRVhDTCkgew0KLQkJbXV0ZXhfZW50ZXIoJnNwYV9uYW1lc3BhY2VfbG9j ayk7DQorCQltdXRleF9lbnRlcigmemZzZGV2X3N0YXRlX2xvY2spOw0KIAkJ ZXJyb3IgPSB6ZnNfY3RsZGV2X2luaXQoZGV2cCk7DQotCQltdXRleF9leGl0 KCZzcGFfbmFtZXNwYWNlX2xvY2spOw0KKwkJbXV0ZXhfZXhpdCgmemZzZGV2 X3N0YXRlX2xvY2spOw0KIAl9DQogDQogCXJldHVybiAoZXJyb3IpOw0KQEAg LTU3MjMsMTQgKzU3MjMsMTQgQEAgemZzZGV2X2Nsb3NlKHZvaWQgKmRhdGEp DQogCWlmIChtaW5vciA9PSAwKQ0KIAkJcmV0dXJuOw0KIA0KLQltdXRleF9l bnRlcigmc3BhX25hbWVzcGFjZV9sb2NrKTsNCisJbXV0ZXhfZW50ZXIoJnpm c2Rldl9zdGF0ZV9sb2NrKTsNCiAJem8gPSB6ZnNkZXZfZ2V0X3NvZnRfc3Rh dGUobWlub3IsIFpTU1RfQ1RMREVWKTsNCiAJaWYgKHpvID09IE5VTEwpIHsN Ci0JCW11dGV4X2V4aXQoJnNwYV9uYW1lc3BhY2VfbG9jayk7DQorCQltdXRl eF9leGl0KCZ6ZnNkZXZfc3RhdGVfbG9jayk7DQogCQlyZXR1cm47DQogCX0N CiAJemZzX2N0bGRldl9kZXN0cm95KHpvLCBtaW5vcik7DQotCW11dGV4X2V4 aXQoJnNwYV9uYW1lc3BhY2VfbG9jayk7DQorCW11dGV4X2V4aXQoJnpmc2Rl dl9zdGF0ZV9sb2NrKTsNCiB9DQogDQogc3RhdGljIGludA0KZGlmZiAtLWdp dCBhL3N5cy9jZGRsL2NvbnRyaWIvb3BlbnNvbGFyaXMvdXRzL2NvbW1vbi9m cy96ZnMvenZvbC5jIGIvc3lzL2NkZGwvY29udHJpYi9vcGVuc29sYXJpcy91 dHMvY29tbW9uL2ZzL3pmcy96dm9sLmMNCmluZGV4IDcyZDQ1MDIuLmFlYzUy MTkgMTAwNjQ0DQotLS0gYS9zeXMvY2RkbC9jb250cmliL29wZW5zb2xhcmlz L3V0cy9jb21tb24vZnMvemZzL3p2b2wuYw0KKysrIGIvc3lzL2NkZGwvY29u dHJpYi9vcGVuc29sYXJpcy91dHMvY29tbW9uL2ZzL3pmcy96dm9sLmMNCkBA IC0xMDQsMTEgKzEwNCwxMiBAQCBzdGF0aWMgY2hhciAqenZvbF90YWcgPSAi enZvbF90YWciOw0KICNkZWZpbmUJWlZPTF9EVU1QU0laRQkJImR1bXBzaXpl Ig0KIA0KIC8qDQotICogVGhlIHNwYV9uYW1lc3BhY2VfbG9jayBwcm90ZWN0 cyB0aGUgemZzZGV2X3N0YXRlIHN0cnVjdHVyZSBmcm9tIGJlaW5nDQotICog bW9kaWZpZWQgd2hpbGUgaXQncyBiZWluZyB1c2VkLCBlLmcuIGFuIG9wZW4g dGhhdCBjb21lcyBpbiBiZWZvcmUgYQ0KLSAqIGNyZWF0ZSBmaW5pc2hlcy4g IEl0IGFsc28gcHJvdGVjdHMgdGVtcG9yYXJ5IG9wZW5zIG9mIHRoZSBkYXRh c2V0IHNvIHRoYXQsDQorICogVGhpcyBsb2NrIHByb3RlY3RzIHRoZSB6ZnNk ZXZfc3RhdGUgc3RydWN0dXJlIGZyb20gYmVpbmcgbW9kaWZpZWQNCisgKiB3 aGlsZSBpdCdzIGJlaW5nIHVzZWQsIGUuZy4gYW4gb3BlbiB0aGF0IGNvbWVz IGluIGJlZm9yZSBhIGNyZWF0ZQ0KKyAqIGZpbmlzaGVzLiAgSXQgYWxzbyBw cm90ZWN0cyB0ZW1wb3Jhcnkgb3BlbnMgb2YgdGhlIGRhdGFzZXQgc28gdGhh dCwNCiAgKiBlLmcuLCBhbiBvcGVuIGRvZXNuJ3QgZ2V0IGEgc3B1cmlvdXMg RUJVU1kuDQogICovDQora211dGV4X3QgemZzZGV2X3N0YXRlX2xvY2s7DQog c3RhdGljIHVpbnQzMl90IHp2b2xfbWlub3JzOw0KIA0KIHR5cGVkZWYgc3Ry dWN0IHp2b2xfZXh0ZW50IHsNCkBAIC0yNDksNyArMjUwLDcgQEAgenZvbF9t aW5vcl9sb29rdXAoY29uc3QgY2hhciAqbmFtZSkNCiAJc3RydWN0IGdfZ2Vv bSAqZ3A7DQogCXp2b2xfc3RhdGVfdCAqenYgPSBOVUxMOw0KIA0KLQlBU1NF UlQoTVVURVhfSEVMRCgmc3BhX25hbWVzcGFjZV9sb2NrKSk7DQorCUFTU0VS VChNVVRFWF9IRUxEKCZ6ZnNkZXZfc3RhdGVfbG9jaykpOw0KIA0KIAlnX3Rv cG9sb2d5X2xvY2soKTsNCiAJTElTVF9GT1JFQUNIKGdwLCAmemZzX3p2b2xf Y2xhc3MuZ2VvbSwgZ2VvbSkgew0KQEAgLTQ2NSwxMSArNDY2LDExIEBAIHp2 b2xfbmFtZTJtaW5vcihjb25zdCBjaGFyICpuYW1lLCBtaW5vcl90ICptaW5v cikNCiB7DQogCXp2b2xfc3RhdGVfdCAqenY7DQogDQotCW11dGV4X2VudGVy KCZzcGFfbmFtZXNwYWNlX2xvY2spOw0KKwltdXRleF9lbnRlcigmemZzZGV2 X3N0YXRlX2xvY2spOw0KIAl6diA9IHp2b2xfbWlub3JfbG9va3VwKG5hbWUp Ow0KIAlpZiAobWlub3IgJiYgenYpDQogCQkqbWlub3IgPSB6di0+enZfbWlu b3I7DQotCW11dGV4X2V4aXQoJnNwYV9uYW1lc3BhY2VfbG9jayk7DQorCW11 dGV4X2V4aXQoJnpmc2Rldl9zdGF0ZV9sb2NrKTsNCiAJcmV0dXJuICh6diA/ IDAgOiAtMSk7DQogfQ0KICNlbmRpZgkvKiBzdW4gKi8NCkBAIC00ODksMTAg KzQ5MCwxMCBAQCB6dm9sX2NyZWF0ZV9taW5vcihjb25zdCBjaGFyICpuYW1l KQ0KIA0KIAlaRlNfTE9HKDEsICJDcmVhdGluZyBaVk9MICVzLi4uIiwgbmFt ZSk7DQogDQotCW11dGV4X2VudGVyKCZzcGFfbmFtZXNwYWNlX2xvY2spOw0K KwltdXRleF9lbnRlcigmemZzZGV2X3N0YXRlX2xvY2spOw0KIA0KIAlpZiAo enZvbF9taW5vcl9sb29rdXAobmFtZSkgIT0gTlVMTCkgew0KLQkJbXV0ZXhf ZXhpdCgmc3BhX25hbWVzcGFjZV9sb2NrKTsNCisJCW11dGV4X2V4aXQoJnpm c2Rldl9zdGF0ZV9sb2NrKTsNCiAJCXJldHVybiAoU0VUX0VSUk9SKEVFWElT VCkpOw0KIAl9DQogDQpAQCAtNTAwLDIwICs1MDEsMjAgQEAgenZvbF9jcmVh dGVfbWlub3IoY29uc3QgY2hhciAqbmFtZSkNCiAJZXJyb3IgPSBkbXVfb2Jq c2V0X293bihuYW1lLCBETVVfT1NUX1pWT0wsIEJfVFJVRSwgRlRBRywgJm9z KTsNCiANCiAJaWYgKGVycm9yKSB7DQotCQltdXRleF9leGl0KCZzcGFfbmFt ZXNwYWNlX2xvY2spOw0KKwkJbXV0ZXhfZXhpdCgmemZzZGV2X3N0YXRlX2xv Y2spOw0KIAkJcmV0dXJuIChlcnJvcik7DQogCX0NCiANCiAjaWZkZWYgc3Vu DQogCWlmICgobWlub3IgPSB6ZnNkZXZfbWlub3JfYWxsb2MoKSkgPT0gMCkg ew0KIAkJZG11X29ianNldF9kaXNvd24ob3MsIEZUQUcpOw0KLQkJbXV0ZXhf ZXhpdCgmc3BhX25hbWVzcGFjZV9sb2NrKTsNCisJCW11dGV4X2V4aXQoJnpm c2Rldl9zdGF0ZV9sb2NrKTsNCiAJCXJldHVybiAoU0VUX0VSUk9SKEVOWElP KSk7DQogCX0NCiANCiAJaWYgKGRkaV9zb2Z0X3N0YXRlX3phbGxvYyh6ZnNk ZXZfc3RhdGUsIG1pbm9yKSAhPSBERElfU1VDQ0VTUykgew0KIAkJZG11X29i anNldF9kaXNvd24ob3MsIEZUQUcpOw0KLQkJbXV0ZXhfZXhpdCgmc3BhX25h bWVzcGFjZV9sb2NrKTsNCisJCW11dGV4X2V4aXQoJnpmc2Rldl9zdGF0ZV9s b2NrKTsNCiAJCXJldHVybiAoU0VUX0VSUk9SKEVBR0FJTikpOw0KIAl9DQog CSh2b2lkKSBkZGlfcHJvcF91cGRhdGVfc3RyaW5nKG1pbm9yLCB6ZnNfZGlw LCBaVk9MX1BST1BfTkFNRSwNCkBAIC01MjUsNyArNTI2LDcgQEAgenZvbF9j cmVhdGVfbWlub3IoY29uc3QgY2hhciAqbmFtZSkNCiAJICAgIG1pbm9yLCBE RElfUFNFVURPLCAwKSA9PSBERElfRkFJTFVSRSkgew0KIAkJZGRpX3NvZnRf c3RhdGVfZnJlZSh6ZnNkZXZfc3RhdGUsIG1pbm9yKTsNCiAJCWRtdV9vYmpz ZXRfZGlzb3duKG9zLCBGVEFHKTsNCi0JCW11dGV4X2V4aXQoJnNwYV9uYW1l c3BhY2VfbG9jayk7DQorCQltdXRleF9leGl0KCZ6ZnNkZXZfc3RhdGVfbG9j ayk7DQogCQlyZXR1cm4gKFNFVF9FUlJPUihFQUdBSU4pKTsNCiAJfQ0KIA0K QEAgLTUzNiw3ICs1MzcsNyBAQCB6dm9sX2NyZWF0ZV9taW5vcihjb25zdCBj aGFyICpuYW1lKQ0KIAkJZGRpX3JlbW92ZV9taW5vcl9ub2RlKHpmc19kaXAs IGNocmJ1Zik7DQogCQlkZGlfc29mdF9zdGF0ZV9mcmVlKHpmc2Rldl9zdGF0 ZSwgbWlub3IpOw0KIAkJZG11X29ianNldF9kaXNvd24ob3MsIEZUQUcpOw0K LQkJbXV0ZXhfZXhpdCgmc3BhX25hbWVzcGFjZV9sb2NrKTsNCisJCW11dGV4 X2V4aXQoJnpmc2Rldl9zdGF0ZV9sb2NrKTsNCiAJCXJldHVybiAoU0VUX0VS Uk9SKEVBR0FJTikpOw0KIAl9DQogDQpAQCAtNTg3LDcgKzU4OCw3IEBAIHp2 b2xfY3JlYXRlX21pbm9yKGNvbnN0IGNoYXIgKm5hbWUpDQogDQogCXp2b2xf bWlub3JzKys7DQogDQotCW11dGV4X2V4aXQoJnNwYV9uYW1lc3BhY2VfbG9j ayk7DQorCW11dGV4X2V4aXQoJnpmc2Rldl9zdGF0ZV9sb2NrKTsNCiANCiAJ enZvbF9nZW9tX3J1bih6dik7DQogDQpAQCAtNjA5LDcgKzYxMCw3IEBAIHp2 b2xfcmVtb3ZlX3p2KHp2b2xfc3RhdGVfdCAqenYpDQogCW1pbm9yX3QgbWlu b3IgPSB6di0+enZfbWlub3I7DQogI2VuZGlmDQogDQotCUFTU0VSVChNVVRF WF9IRUxEKCZzcGFfbmFtZXNwYWNlX2xvY2spKTsNCisJQVNTRVJUKE1VVEVY X0hFTEQoJnpmc2Rldl9zdGF0ZV9sb2NrKSk7DQogCWlmICh6di0+enZfdG90 YWxfb3BlbnMgIT0gMCkNCiAJCXJldHVybiAoU0VUX0VSUk9SKEVCVVNZKSk7 DQogDQpAQCAtNjM1LDE1ICs2MzYsMTUgQEAgenZvbF9yZW1vdmVfbWlub3Io Y29uc3QgY2hhciAqbmFtZSkNCiAJenZvbF9zdGF0ZV90ICp6djsNCiAJaW50 IHJjOw0KIA0KLQltdXRleF9lbnRlcigmc3BhX25hbWVzcGFjZV9sb2NrKTsN CisJbXV0ZXhfZW50ZXIoJnpmc2Rldl9zdGF0ZV9sb2NrKTsNCiAJaWYgKCh6 diA9IHp2b2xfbWlub3JfbG9va3VwKG5hbWUpKSA9PSBOVUxMKSB7DQotCQlt dXRleF9leGl0KCZzcGFfbmFtZXNwYWNlX2xvY2spOw0KKwkJbXV0ZXhfZXhp dCgmemZzZGV2X3N0YXRlX2xvY2spOw0KIAkJcmV0dXJuIChTRVRfRVJST1Io RU5YSU8pKTsNCiAJfQ0KIAlnX3RvcG9sb2d5X2xvY2soKTsNCiAJcmMgPSB6 dm9sX3JlbW92ZV96dih6dik7DQogCWdfdG9wb2xvZ3lfdW5sb2NrKCk7DQot CW11dGV4X2V4aXQoJnNwYV9uYW1lc3BhY2VfbG9jayk7DQorCW11dGV4X2V4 aXQoJnpmc2Rldl9zdGF0ZV9sb2NrKTsNCiAJcmV0dXJuIChyYyk7DQogfQ0K IA0KQEAgLTc1NSw3ICs3NTYsNyBAQCB6dm9sX3VwZGF0ZV92b2xzaXplKG9i anNldF90ICpvcywgdWludDY0X3Qgdm9sc2l6ZSkNCiAJZG11X3R4X3QgKnR4 Ow0KIAlpbnQgZXJyb3I7DQogDQotCUFTU0VSVChNVVRFWF9IRUxEKCZzcGFf bmFtZXNwYWNlX2xvY2spKTsNCisJQVNTRVJUKE1VVEVYX0hFTEQoJnpmc2Rl dl9zdGF0ZV9sb2NrKSk7DQogDQogCXR4ID0gZG11X3R4X2NyZWF0ZShvcyk7 DQogCWRtdV90eF9ob2xkX3phcCh0eCwgWlZPTF9aQVBfT0JKLCBUUlVFLCBO VUxMKTsNCkBAIC03ODYsNyArNzg3LDcgQEAgenZvbF9yZW1vdmVfbWlub3Jz KGNvbnN0IGNoYXIgKm5hbWUpDQogCW5hbWVsZW4gPSBzdHJsZW4obmFtZSk7 DQogDQogCURST1BfR0lBTlQoKTsNCi0JbXV0ZXhfZW50ZXIoJnNwYV9uYW1l c3BhY2VfbG9jayk7DQorCW11dGV4X2VudGVyKCZ6ZnNkZXZfc3RhdGVfbG9j ayk7DQogCWdfdG9wb2xvZ3lfbG9jaygpOw0KIA0KIAlMSVNUX0ZPUkVBQ0hf U0FGRShncCwgJnpmc196dm9sX2NsYXNzLmdlb20sIGdlb20sIGdwdG1wKSB7 DQpAQCAtODA0LDcgKzgwNSw3IEBAIHp2b2xfcmVtb3ZlX21pbm9ycyhjb25z dCBjaGFyICpuYW1lKQ0KIAl9DQogDQogCWdfdG9wb2xvZ3lfdW5sb2NrKCk7 DQotCW11dGV4X2V4aXQoJnNwYV9uYW1lc3BhY2VfbG9jayk7DQorCW11dGV4 X2V4aXQoJnpmc2Rldl9zdGF0ZV9sb2NrKTsNCiAJUElDS1VQX0dJQU5UKCk7 DQogfQ0KIA0KQEAgLTgxOCwxMCArODE5LDEwIEBAIHp2b2xfc2V0X3ZvbHNp emUoY29uc3QgY2hhciAqbmFtZSwgbWFqb3JfdCBtYWosIHVpbnQ2NF90IHZv bHNpemUpDQogCXVpbnQ2NF90IG9sZF92b2xzaXplID0gMFVMTDsNCiAJdWlu dDY0X3QgcmVhZG9ubHk7DQogDQotCW11dGV4X2VudGVyKCZzcGFfbmFtZXNw YWNlX2xvY2spOw0KKwltdXRleF9lbnRlcigmemZzZGV2X3N0YXRlX2xvY2sp Ow0KIAl6diA9IHp2b2xfbWlub3JfbG9va3VwKG5hbWUpOw0KIAlpZiAoKGVy cm9yID0gZG11X29ianNldF9ob2xkKG5hbWUsIEZUQUcsICZvcykpICE9IDAp IHsNCi0JCW11dGV4X2V4aXQoJnNwYV9uYW1lc3BhY2VfbG9jayk7DQorCQlt dXRleF9leGl0KCZ6ZnNkZXZfc3RhdGVfbG9jayk7DQogCQlyZXR1cm4gKGVy cm9yKTsNCiAJfQ0KIA0KQEAgLTg4OCw3ICs4ODksNyBAQCB6dm9sX3NldF92 b2xzaXplKGNvbnN0IGNoYXIgKm5hbWUsIG1ham9yX3QgbWFqLCB1aW50NjRf dCB2b2xzaXplKQ0KIG91dDoNCiAJZG11X29ianNldF9yZWxlKG9zLCBGVEFH KTsNCiANCi0JbXV0ZXhfZXhpdCgmc3BhX25hbWVzcGFjZV9sb2NrKTsNCisJ bXV0ZXhfZXhpdCgmemZzZGV2X3N0YXRlX2xvY2spOw0KIA0KIAlyZXR1cm4g KGVycm9yKTsNCiB9DQpAQCAtODk5LDM2ICs5MDAsMTkgQEAgenZvbF9vcGVu KHN0cnVjdCBnX3Byb3ZpZGVyICpwcCwgaW50IGZsYWcsIGludCBjb3VudCkN CiB7DQogCXp2b2xfc3RhdGVfdCAqenY7DQogCWludCBlcnIgPSAwOw0KLQli b29sZWFuX3QgbG9ja2VkID0gQl9GQUxTRTsNCiANCi0JLyoNCi0JICogUHJv dGVjdCBhZ2FpbnN0IHJlY3Vyc2l2ZWx5IGVudGVyaW5nIHNwYV9uYW1lc3Bh Y2VfbG9jaw0KLQkgKiB3aGVuIHNwYV9vcGVuKCkgaXMgdXNlZCBmb3IgYSBw b29sIG9uIGEgKGxvY2FsKSBaVk9MKHMpLg0KLQkgKiBUaGlzIGlzIG5lZWRl ZCBzaW5jZSB3ZSByZXBsYWNlZCB1cHN0cmVhbSB6ZnNkZXZfc3RhdGVfbG9j aw0KLQkgKiB3aXRoIHNwYV9uYW1lc3BhY2VfbG9jayBpbiB0aGUgWlZPTCBj b2RlLg0KLQkgKiBXZSBhcmUgdXNpbmcgdGhlIHNhbWUgdHJpY2sgYXMgc3Bh X29wZW4oKS4NCi0JICogTm90ZSB0aGF0IGNhbGxzIGluIHp2b2xfZmlyc3Rf b3BlbiB3aGljaCBuZWVkIHRvIHJlc29sdmUNCi0JICogcG9vbCBuYW1lIHRv IGEgc3BhIG9iamVjdCB3aWxsIGVudGVyIHNwYV9vcGVuKCkNCi0JICogcmVj dXJzaXZlbHksIGJ1dCB0aGF0IGZ1bmN0aW9uIGFscmVhZHkgaGFzIGFsbCB0 aGUNCi0JICogbmVjZXNzYXJ5IHByb3RlY3Rpb24uDQotCSAqLw0KLQlpZiAo IU1VVEVYX0hFTEQoJnNwYV9uYW1lc3BhY2VfbG9jaykpIHsNCi0JCW11dGV4 X2VudGVyKCZzcGFfbmFtZXNwYWNlX2xvY2spOw0KLQkJbG9ja2VkID0gQl9U UlVFOw0KLQl9DQorCW11dGV4X2VudGVyKCZ6ZnNkZXZfc3RhdGVfbG9jayk7 DQogDQogCXp2ID0gcHAtPnByaXZhdGU7DQogCWlmICh6diA9PSBOVUxMKSB7 DQotCQlpZiAobG9ja2VkKQ0KLQkJCW11dGV4X2V4aXQoJnNwYV9uYW1lc3Bh Y2VfbG9jayk7DQorCQltdXRleF9leGl0KCZ6ZnNkZXZfc3RhdGVfbG9jayk7 DQogCQlyZXR1cm4gKFNFVF9FUlJPUihFTlhJTykpOw0KIAl9DQogDQogCWlm ICh6di0+enZfdG90YWxfb3BlbnMgPT0gMCkNCiAJCWVyciA9IHp2b2xfZmly c3Rfb3Blbih6dik7DQogCWlmIChlcnIpIHsNCi0JCWlmIChsb2NrZWQpDQot CQkJbXV0ZXhfZXhpdCgmc3BhX25hbWVzcGFjZV9sb2NrKTsNCisJCW11dGV4 X2V4aXQoJnpmc2Rldl9zdGF0ZV9sb2NrKTsNCiAJCXJldHVybiAoZXJyKTsN CiAJfQ0KIAlpZiAoKGZsYWcgJiBGV1JJVEUpICYmICh6di0+enZfZmxhZ3Mg JiBaVk9MX1JET05MWSkpIHsNCkBAIC05NTAsMTUgKzkzNCwxMyBAQCB6dm9s X29wZW4oc3RydWN0IGdfcHJvdmlkZXIgKnBwLCBpbnQgZmxhZywgaW50IGNv dW50KQ0KICNlbmRpZg0KIA0KIAl6di0+enZfdG90YWxfb3BlbnMgKz0gY291 bnQ7DQotCWlmIChsb2NrZWQpDQotCQltdXRleF9leGl0KCZzcGFfbmFtZXNw YWNlX2xvY2spOw0KKwltdXRleF9leGl0KCZ6ZnNkZXZfc3RhdGVfbG9jayk7 DQogDQogCXJldHVybiAoZXJyKTsNCiBvdXQ6DQogCWlmICh6di0+enZfdG90 YWxfb3BlbnMgPT0gMCkNCiAJCXp2b2xfbGFzdF9jbG9zZSh6dik7DQotCWlm IChsb2NrZWQpDQotCQltdXRleF9leGl0KCZzcGFfbmFtZXNwYWNlX2xvY2sp Ow0KKwltdXRleF9leGl0KCZ6ZnNkZXZfc3RhdGVfbG9jayk7DQogCXJldHVy biAoZXJyKTsNCiB9DQogDQpAQCAtOTY4LDE4ICs5NTAsMTIgQEAgenZvbF9j bG9zZShzdHJ1Y3QgZ19wcm92aWRlciAqcHAsIGludCBmbGFnLCBpbnQgY291 bnQpDQogew0KIAl6dm9sX3N0YXRlX3QgKnp2Ow0KIAlpbnQgZXJyb3IgPSAw Ow0KLQlib29sZWFuX3QgbG9ja2VkID0gQl9GQUxTRTsNCiANCi0JLyogU2Vl IGNvbW1lbnQgaW4genZvbF9vcGVuKCkuICovDQotCWlmICghTVVURVhfSEVM RCgmc3BhX25hbWVzcGFjZV9sb2NrKSkgew0KLQkJbXV0ZXhfZW50ZXIoJnNw YV9uYW1lc3BhY2VfbG9jayk7DQotCQlsb2NrZWQgPSBCX1RSVUU7DQotCX0N CisJbXV0ZXhfZW50ZXIoJnpmc2Rldl9zdGF0ZV9sb2NrKTsNCiANCiAJenYg PSBwcC0+cHJpdmF0ZTsNCiAJaWYgKHp2ID09IE5VTEwpIHsNCi0JCWlmIChs b2NrZWQpDQotCQkJbXV0ZXhfZXhpdCgmc3BhX25hbWVzcGFjZV9sb2NrKTsN CisJCW11dGV4X2V4aXQoJnpmc2Rldl9zdGF0ZV9sb2NrKTsNCiAJCXJldHVy biAoU0VUX0VSUk9SKEVOWElPKSk7DQogCX0NCiANCkBAIC0xMDAyLDggKzk3 OCw3IEBAIHp2b2xfY2xvc2Uoc3RydWN0IGdfcHJvdmlkZXIgKnBwLCBpbnQg ZmxhZywgaW50IGNvdW50KQ0KIAlpZiAoenYtPnp2X3RvdGFsX29wZW5zID09 IDApDQogCQl6dm9sX2xhc3RfY2xvc2UoenYpOw0KIA0KLQlpZiAobG9ja2Vk KQ0KLQkJbXV0ZXhfZXhpdCgmc3BhX25hbWVzcGFjZV9sb2NrKTsNCisJbXV0 ZXhfZXhpdCgmemZzZGV2X3N0YXRlX2xvY2spOw0KIAlyZXR1cm4gKGVycm9y KTsNCiB9DQogDQpAQCAtMTY1OCwxMiArMTYzMywxMiBAQCB6dm9sX2lvY3Rs KGRldl90IGRldiwgaW50IGNtZCwgaW50cHRyX3QgYXJnLCBpbnQgZmxhZywg Y3JlZF90ICpjciwgaW50ICpydmFscCkNCiAJaW50IGVycm9yID0gMDsNCiAJ cmxfdCAqcmw7DQogDQotCW11dGV4X2VudGVyKCZzcGFfbmFtZXNwYWNlX2xv Y2spOw0KKwltdXRleF9lbnRlcigmemZzZGV2X3N0YXRlX2xvY2spOw0KIA0K IAl6diA9IHpmc2Rldl9nZXRfc29mdF9zdGF0ZShnZXRtaW5vcihkZXYpLCBa U1NUX1pWT0wpOw0KIA0KIAlpZiAoenYgPT0gTlVMTCkgew0KLQkJbXV0ZXhf ZXhpdCgmc3BhX25hbWVzcGFjZV9sb2NrKTsNCisJCW11dGV4X2V4aXQoJnpm c2Rldl9zdGF0ZV9sb2NrKTsNCiAJCXJldHVybiAoU0VUX0VSUk9SKEVOWElP KSk7DQogCX0NCiAJQVNTRVJUKHp2LT56dl90b3RhbF9vcGVucyA+IDApOw0K QEAgLTE2NzcsNyArMTY1Miw3IEBAIHp2b2xfaW9jdGwoZGV2X3QgZGV2LCBp bnQgY21kLCBpbnRwdHJfdCBhcmcsIGludCBmbGFnLCBjcmVkX3QgKmNyLCBp bnQgKnJ2YWxwKQ0KIAkJZGtpLmRraV9jdHlwZSA9IERLQ19VTktOT1dOOw0K IAkJZGtpLmRraV91bml0ID0gZ2V0bWlub3IoZGV2KTsNCiAJCWRraS5ka2lf bWF4dHJhbnNmZXIgPSAxIDw8IChTUEFfTUFYQkxPQ0tTSElGVCAtIHp2LT56 dl9taW5fYnMpOw0KLQkJbXV0ZXhfZXhpdCgmc3BhX25hbWVzcGFjZV9sb2Nr KTsNCisJCW11dGV4X2V4aXQoJnpmc2Rldl9zdGF0ZV9sb2NrKTsNCiAJCWlm IChkZGlfY29weW91dCgmZGtpLCAodm9pZCAqKWFyZywgc2l6ZW9mIChka2kp LCBmbGFnKSkNCiAJCQllcnJvciA9IFNFVF9FUlJPUihFRkFVTFQpOw0KIAkJ cmV0dXJuIChlcnJvcik7DQpAQCAtMTY4Nyw3ICsxNjYyLDcgQEAgenZvbF9p b2N0bChkZXZfdCBkZXYsIGludCBjbWQsIGludHB0cl90IGFyZywgaW50IGZs YWcsIGNyZWRfdCAqY3IsIGludCAqcnZhbHApDQogCQlka20uZGtpX2xic2l6 ZSA9IDFVIDw8IHp2LT56dl9taW5fYnM7DQogCQlka20uZGtpX2NhcGFjaXR5 ID0genYtPnp2X3ZvbHNpemUgPj4genYtPnp2X21pbl9iczsNCiAJCWRrbS5k a2lfbWVkaWFfdHlwZSA9IERLX1VOS05PV047DQotCQltdXRleF9leGl0KCZz cGFfbmFtZXNwYWNlX2xvY2spOw0KKwkJbXV0ZXhfZXhpdCgmemZzZGV2X3N0 YXRlX2xvY2spOw0KIAkJaWYgKGRkaV9jb3B5b3V0KCZka20sICh2b2lkICop YXJnLCBzaXplb2YgKGRrbSksIGZsYWcpKQ0KIAkJCWVycm9yID0gU0VUX0VS Uk9SKEVGQVVMVCk7DQogCQlyZXR1cm4gKGVycm9yKTsNCkBAIC0xNjk3LDE0 ICsxNjcyLDE0IEBAIHp2b2xfaW9jdGwoZGV2X3QgZGV2LCBpbnQgY21kLCBp bnRwdHJfdCBhcmcsIGludCBmbGFnLCBjcmVkX3QgKmNyLCBpbnQgKnJ2YWxw KQ0KIAkJCXVpbnQ2NF90IHZzID0genYtPnp2X3ZvbHNpemU7DQogCQkJdWlu dDhfdCBicyA9IHp2LT56dl9taW5fYnM7DQogDQotCQkJbXV0ZXhfZXhpdCgm c3BhX25hbWVzcGFjZV9sb2NrKTsNCisJCQltdXRleF9leGl0KCZ6ZnNkZXZf c3RhdGVfbG9jayk7DQogCQkJZXJyb3IgPSB6dm9sX2dldGVmaSgodm9pZCAq KWFyZywgZmxhZywgdnMsIGJzKTsNCiAJCQlyZXR1cm4gKGVycm9yKTsNCiAJ CX0NCiANCiAJY2FzZSBES0lPQ0ZMVVNIV1JJVEVDQUNIRToNCiAJCWRrYyA9 IChzdHJ1Y3QgZGtfY2FsbGJhY2sgKilhcmc7DQotCQltdXRleF9leGl0KCZz cGFfbmFtZXNwYWNlX2xvY2spOw0KKwkJbXV0ZXhfZXhpdCgmemZzZGV2X3N0 YXRlX2xvY2spOw0KIAkJemlsX2NvbW1pdCh6di0+enZfemlsb2csIFpWT0xf T0JKKTsNCiAJCWlmICgoZmxhZyAmIEZLSU9DVEwpICYmIGRrYyAhPSBOVUxM ICYmIGRrYy0+ZGtjX2NhbGxiYWNrKSB7DQogCQkJKCpka2MtPmRrY19jYWxs YmFjaykoZGtjLT5ka2NfY29va2llLCBlcnJvcik7DQpAQCAtMTczMCwxMCAr MTcwNSwxMCBAQCB6dm9sX2lvY3RsKGRldl90IGRldiwgaW50IGNtZCwgaW50 cHRyX3QgYXJnLCBpbnQgZmxhZywgY3JlZF90ICpjciwgaW50ICpydmFscCkN CiAJCQl9DQogCQkJaWYgKHdjZSkgew0KIAkJCQl6di0+enZfZmxhZ3MgfD0g WlZPTF9XQ0U7DQotCQkJCW11dGV4X2V4aXQoJnNwYV9uYW1lc3BhY2VfbG9j ayk7DQorCQkJCW11dGV4X2V4aXQoJnpmc2Rldl9zdGF0ZV9sb2NrKTsNCiAJ CQl9IGVsc2Ugew0KIAkJCQl6di0+enZfZmxhZ3MgJj0gflpWT0xfV0NFOw0K LQkJCQltdXRleF9leGl0KCZzcGFfbmFtZXNwYWNlX2xvY2spOw0KKwkJCQlt dXRleF9leGl0KCZ6ZnNkZXZfc3RhdGVfbG9jayk7DQogCQkJCXppbF9jb21t aXQoenYtPnp2X3ppbG9nLCBaVk9MX09CSik7DQogCQkJfQ0KIAkJCXJldHVy biAoMCk7DQpAQCAtMTgyOCw3ICsxODAzLDcgQEAgenZvbF9pb2N0bChkZXZf dCBkZXYsIGludCBjbWQsIGludHB0cl90IGFyZywgaW50IGZsYWcsIGNyZWRf dCAqY3IsIGludCAqcnZhbHApDQogCQlicmVhazsNCiANCiAJfQ0KLQltdXRl eF9leGl0KCZzcGFfbmFtZXNwYWNlX2xvY2spOw0KKwltdXRleF9leGl0KCZ6 ZnNkZXZfc3RhdGVfbG9jayk7DQogCXJldHVybiAoZXJyb3IpOw0KIH0NCiAj ZW5kaWYJLyogc3VuICovDQpAQCAtMTg0NCwxMiArMTgxOSwxNCBAQCB6dm9s X2luaXQodm9pZCkNCiB7DQogCVZFUklGWShkZGlfc29mdF9zdGF0ZV9pbml0 KCZ6ZnNkZXZfc3RhdGUsIHNpemVvZiAoemZzX3NvZnRfc3RhdGVfdCksDQog CSAgICAxKSA9PSAwKTsNCisJbXV0ZXhfaW5pdCgmemZzZGV2X3N0YXRlX2xv Y2ssIE5VTEwsIE1VVEVYX0RFRkFVTFQsIE5VTEwpOw0KIAlaRlNfTE9HKDEs ICJaVk9MIEluaXRpYWxpemVkLiIpOw0KIH0NCiANCiB2b2lkDQogenZvbF9m aW5pKHZvaWQpDQogew0KKwltdXRleF9kZXN0cm95KCZ6ZnNkZXZfc3RhdGVf bG9jayk7DQogCWRkaV9zb2Z0X3N0YXRlX2ZpbmkoJnpmc2Rldl9zdGF0ZSk7 DQogCVpGU19MT0coMSwgIlpWT0wgRGVpbml0aWFsaXplZC4iKTsNCiB9DQpA QCAtMTg4OSw3ICsxODY2LDcgQEAgenZvbF9kdW1wX2luaXQoenZvbF9zdGF0 ZV90ICp6diwgYm9vbGVhbl90IHJlc2l6ZSkNCiAJdWludDY0X3QgdmVyc2lv biA9IHNwYV92ZXJzaW9uKHNwYSk7DQogCWVudW0gemlvX2NoZWNrc3VtIGNo ZWNrc3VtOw0KIA0KLQlBU1NFUlQoTVVURVhfSEVMRCgmc3BhX25hbWVzcGFj ZV9sb2NrKSk7DQorCUFTU0VSVChNVVRFWF9IRUxEKCZ6ZnNkZXZfc3RhdGVf bG9jaykpOw0KIAlBU1NFUlQodmQtPnZkZXZfb3BzID09ICZ2ZGV2X3Jvb3Rf b3BzKTsNCiANCiAJZXJyb3IgPSBkbXVfZnJlZV9sb25nX3JhbmdlKHp2LT56 dl9vYmpzZXQsIFpWT0xfT0JKLCAwLA0KQEAgLTI0MzcsNyArMjQxNCw3IEBA IHp2b2xfcmVuYW1lX21pbm9yKHN0cnVjdCBnX2dlb20gKmdwLCBjb25zdCBj aGFyICpuZXduYW1lKQ0KIAlzdHJ1Y3QgZ19wcm92aWRlciAqcHA7DQogCXp2 b2xfc3RhdGVfdCAqenY7DQogDQotCUFTU0VSVChNVVRFWF9IRUxEKCZzcGFf bmFtZXNwYWNlX2xvY2spKTsNCisJQVNTRVJUKE1VVEVYX0hFTEQoJnpmc2Rl dl9zdGF0ZV9sb2NrKSk7DQogCWdfdG9wb2xvZ3lfYXNzZXJ0KCk7DQogDQog CXBwID0gTElTVF9GSVJTVCgmZ3AtPnByb3ZpZGVyKTsNCkBAIC0yNDcxLDcg KzI0NDgsNyBAQCB6dm9sX3JlbmFtZV9taW5vcnMoY29uc3QgY2hhciAqb2xk bmFtZSwgY29uc3QgY2hhciAqbmV3bmFtZSkNCiAJbmV3bmFtZWxlbiA9IHN0 cmxlbihuZXduYW1lKTsNCiANCiAJRFJPUF9HSUFOVCgpOw0KLQltdXRleF9l bnRlcigmc3BhX25hbWVzcGFjZV9sb2NrKTsNCisJbXV0ZXhfZW50ZXIoJnpm c2Rldl9zdGF0ZV9sb2NrKTsNCiAJZ190b3BvbG9neV9sb2NrKCk7DQogDQog CUxJU1RfRk9SRUFDSChncCwgJnpmc196dm9sX2NsYXNzLmdlb20sIGdlb20p IHsNCkBAIC0yNDk0LDYgKzI0NzEsNiBAQCB6dm9sX3JlbmFtZV9taW5vcnMo Y29uc3QgY2hhciAqb2xkbmFtZSwgY29uc3QgY2hhciAqbmV3bmFtZSkNCiAJ fQ0KIA0KIAlnX3RvcG9sb2d5X3VubG9jaygpOw0KLQltdXRleF9leGl0KCZz cGFfbmFtZXNwYWNlX2xvY2spOw0KKwltdXRleF9leGl0KCZ6ZnNkZXZfc3Rh dGVfbG9jayk7DQogCVBJQ0tVUF9HSUFOVCgpOw0KIH0NCi0tIA0KMS44LjQu Mg0KDQo= --1030603365-686922855-1387467971=:4344 Content-Type: TEXT/x-diff; name=0002-ZFS-snapshot-handling-fix.patch Content-Transfer-Encoding: BASE64 Content-ID: Content-Description: Content-Disposition: attachment; filename=0002-ZFS-snapshot-handling-fix.patch RnJvbSA1N2Q1YTM4M2I1ODVjMzJjNzdhZjU0ZThjZGFjYWRkZjhjZTc1ODRm IE1vbiBTZXAgMTcgMDA6MDA6MDAgMjAwMQ0KRnJvbTogUmljaGFyZCBLb2pl ZHppbnN6a3kgPGtyaWNoeUBjZmxpbnV4Lmh1Pg0KRGF0ZTogV2VkLCAxOCBE ZWMgMjAxMyAyMjoxMToyMSArMDEwMA0KU3ViamVjdDogW1BBVENIIDIvMl0g WkZTIHNuYXBzaG90IGhhbmRsaW5nIGZpeA0KDQotLS0NCiAuLi4vY29tcGF0 L29wZW5zb2xhcmlzL2tlcm4vb3BlbnNvbGFyaXNfbG9va3VwLmMgICB8IDEz ICsrKy0tLQ0KIC4uLi9vcGVuc29sYXJpcy91dHMvY29tbW9uL2ZzL3pmcy96 ZnNfY3RsZGlyLmMgICAgIHwgNTMgKysrKysrKysrKysrKysrLS0tLS0tLQ0K IDIgZmlsZXMgY2hhbmdlZCwgNDIgaW5zZXJ0aW9ucygrKSwgMjQgZGVsZXRp b25zKC0pDQoNCmRpZmYgLS1naXQgYS9zeXMvY2RkbC9jb21wYXQvb3BlbnNv bGFyaXMva2Vybi9vcGVuc29sYXJpc19sb29rdXAuYyBiL3N5cy9jZGRsL2Nv bXBhdC9vcGVuc29sYXJpcy9rZXJuL29wZW5zb2xhcmlzX2xvb2t1cC5jDQpp bmRleCA5NDM4M2Q2Li40Y2FjMDUzIDEwMDY0NA0KLS0tIGEvc3lzL2NkZGwv Y29tcGF0L29wZW5zb2xhcmlzL2tlcm4vb3BlbnNvbGFyaXNfbG9va3VwLmMN CisrKyBiL3N5cy9jZGRsL2NvbXBhdC9vcGVuc29sYXJpcy9rZXJuL29wZW5z b2xhcmlzX2xvb2t1cC5jDQpAQCAtODEsNiArODEsOCBAQCB0cmF2ZXJzZSh2 bm9kZV90ICoqY3ZwcCwgaW50IGxrdHlwZSkNCiAJICogcHJvZ3Jlc3Mgb24g dGhpcyB2bm9kZS4NCiAJICovDQogDQorCXZuX2xvY2soY3ZwLCBsa3R5cGUp Ow0KKw0KIAlmb3IgKDs7KSB7DQogCQkvKg0KIAkJICogUmVhY2hlZCB0aGUg ZW5kIG9mIHRoZSBtb3VudCBjaGFpbj8NCkBAIC04OSwxMyArOTEsNyBAQCB0 cmF2ZXJzZSh2bm9kZV90ICoqY3ZwcCwgaW50IGxrdHlwZSkNCiAJCWlmICh2 ZnNwID09IE5VTEwpDQogCQkJYnJlYWs7DQogCQllcnJvciA9IHZmc19idXN5 KHZmc3AsIDApOw0KLQkJLyoNCi0JCSAqIHR2cCBpcyBOVUxMIGZvciAqY3Zw cCB2bm9kZSwgd2hpY2ggd2UgY2FuJ3QgdW5sb2NrLg0KLQkJICovDQotCQlp ZiAodHZwICE9IE5VTEwpDQotCQkJdnB1dChjdnApOw0KLQkJZWxzZQ0KLQkJ CXZyZWxlKGN2cCk7DQorCQlWT1BfVU5MT0NLKGN2cCwgMCk7DQogCQlpZiAo ZXJyb3IpDQogCQkJcmV0dXJuIChlcnJvcik7DQogDQpAQCAtMTA3LDYgKzEw Myw5IEBAIHRyYXZlcnNlKHZub2RlX3QgKipjdnBwLCBpbnQgbGt0eXBlKQ0K IAkJdmZzX3VuYnVzeSh2ZnNwKTsNCiAJCWlmIChlcnJvciAhPSAwKQ0KIAkJ CXJldHVybiAoZXJyb3IpOw0KKw0KKwkJVk5fUkVMRShjdnApOw0KKw0KIAkJ Y3ZwID0gdHZwOw0KIAl9DQogDQpkaWZmIC0tZ2l0IGEvc3lzL2NkZGwvY29u dHJpYi9vcGVuc29sYXJpcy91dHMvY29tbW9uL2ZzL3pmcy96ZnNfY3RsZGly LmMgYi9zeXMvY2RkbC9jb250cmliL29wZW5zb2xhcmlzL3V0cy9jb21tb24v ZnMvemZzL3pmc19jdGxkaXIuYw0KaW5kZXggMjhhYjFmYS4uZDM0NjRiNCAx MDA2NDQNCi0tLSBhL3N5cy9jZGRsL2NvbnRyaWIvb3BlbnNvbGFyaXMvdXRz L2NvbW1vbi9mcy96ZnMvemZzX2N0bGRpci5jDQorKysgYi9zeXMvY2RkbC9j b250cmliL29wZW5zb2xhcmlzL3V0cy9jb21tb24vZnMvemZzL3pmc19jdGxk aXIuYw0KQEAgLTExMiw2ICsxMTIsMjUgQEAgc25hcGVudHJ5X2NvbXBhcmUo Y29uc3Qgdm9pZCAqYSwgY29uc3Qgdm9pZCAqYikNCiAJCXJldHVybiAoMCk7 DQogfQ0KIA0KKy8qIFJldHVybiB0aGUgemZzY3RsX3NuYXBkaXJfdCBvYmpl Y3QgZnJvbSBjdXJyZW50IHZub2RlLCBmb2xsb3dpbmcNCisgKiB0aGUgbG9j ayBvcmRlcnMgaW4gemZzY3RsX3NuYXBkaXJfbG9va3VwKCkgdG8gYXZvaWQg ZGVhZGxvY2tzLg0KKyAqIE9uIHJldHVybiB0aGUgcGFzc2VkIGluIHZwIGlz IHVubG9ja2VkICovDQorc3RhdGljIHpmc2N0bF9zbmFwZGlyX3QgKg0KK3pm c2N0bF9zbmFwc2hvdF9nZXRfc25hcGRpcih2bm9kZV90ICp2cCwgdm5vZGVf dCAqKmR2cHApDQorew0KKwlnZnNfZGlyX3QgKmRwID0gdnAtPnZfZGF0YTsN CisJKmR2cHAgPSBkcC0+Z2ZzZF9maWxlLmdmc19wYXJlbnQ7DQorCXpmc2N0 bF9zbmFwZGlyX3QgKnNkcDsNCisNCisJVk5fSE9MRCgqZHZwcCk7DQorCVZP UF9VTkxPQ0sodnAsIDApOw0KKwl2bl9sb2NrKCpkdnBwLCBMS19TSEFSRUQg fCBMS19SRVRSWSB8IExLX0NBTlJFQ1VSU0UpOw0KKwlzZHAgPSAoKmR2cHAp LT52X2RhdGE7DQorCVZPUF9VTkxPQ0soKmR2cHAsIDApOw0KKw0KKwlyZXR1 cm4gKHNkcCk7DQorfQ0KKw0KICNpZmRlZiBzdW4NCiB2bm9kZW9wc190ICp6 ZnNjdGxfb3BzX3Jvb3Q7DQogdm5vZGVvcHNfdCAqemZzY3RsX29wc19zbmFw ZGlyOw0KQEAgLTEwMTMsNiArMTAzMiw4IEBAIHpmc2N0bF9zbmFwZGlyX2xv b2t1cChhcCkNCiAJCQkgKiBUaGUgc25hcHNob3Qgd2FzIHVubW91bnRlZCBi ZWhpbmQgb3VyIGJhY2tzLA0KIAkJCSAqIHRyeSB0byByZW1vdW50IGl0Lg0K IAkJCSAqLw0KKwkJCVZPUF9VTkxPQ0soKnZwcCwgMCk7DQorCQkJVk5fSE9M RCgqdnBwKTsNCiAJCQlWRVJJRlkoemZzY3RsX3NuYXBzaG90X3puYW1lKGR2 cCwgbm0sIE1BWE5BTUVMRU4sIHNuYXBuYW1lKSA9PSAwKTsNCiAJCQlnb3Rv IGRvbW91bnQ7DQogCQl9IGVsc2Ugew0KQEAgLTEwNjQsNyArMTA4NSw2IEBA IHpmc2N0bF9zbmFwZGlyX2xvb2t1cChhcCkNCiAJc2VwLT5zZV9uYW1lID0g a21lbV9hbGxvYyhzdHJsZW4obm0pICsgMSwgS01fU0xFRVApOw0KIAkodm9p ZCkgc3RyY3B5KHNlcC0+c2VfbmFtZSwgbm0pOw0KIAkqdnBwID0gc2VwLT5z ZV9yb290ID0gemZzY3RsX3NuYXBzaG90X21rbm9kZShkdnAsIGRtdV9vYmpz ZXRfaWQoc25hcCkpOw0KLQlWTl9IT0xEKCp2cHApOw0KIAlhdmxfaW5zZXJ0 KCZzZHAtPnNkX3NuYXBzLCBzZXAsIHdoZXJlKTsNCiANCiAJZG11X29ianNl dF9yZWxlKHNuYXAsIEZUQUcpOw0KQEAgLTEwNzUsNiArMTA5NSw3IEBAIGRv bW91bnQ6DQogCSh2b2lkKSBzbnByaW50Zihtb3VudHBvaW50LCBtb3VudHBv aW50X2xlbiwNCiAJICAgICIlcy8iIFpGU19DVExESVJfTkFNRSAiL3NuYXBz aG90LyVzIiwNCiAJICAgIGR2cC0+dl92ZnNwLT5tbnRfc3RhdC5mX21udG9u bmFtZSwgbm0pOw0KKwlWTl9IT0xEKCp2cHApOw0KIAllcnIgPSBtb3VudF9z bmFwc2hvdChjdXJ0aHJlYWQsIHZwcCwgInpmcyIsIG1vdW50cG9pbnQsIHNu YXBuYW1lLCAwKTsNCiAJa21lbV9mcmVlKG1vdW50cG9pbnQsIG1vdW50cG9p bnRfbGVuKTsNCiAJaWYgKGVyciA9PSAwKSB7DQpAQCAtMTQ2NCwxNiArMTQ4 NSwxOCBAQCB6ZnNjdGxfc25hcHNob3RfaW5hY3RpdmUoYXApDQogCWludCBs b2NrZWQ7DQogCXZub2RlX3QgKmR2cDsNCiANCi0JaWYgKHZwLT52X2NvdW50 ID4gMCkNCi0JCWdvdG8gZW5kOw0KLQ0KLQlWRVJJRlkoZ2ZzX2Rpcl9sb29r dXAodnAsICIuLiIsICZkdnAsIGNyLCAwLCBOVUxMLCBOVUxMKSA9PSAwKTsN Ci0Jc2RwID0gZHZwLT52X2RhdGE7DQotCVZPUF9VTkxPQ0soZHZwLCAwKTsN CisJc2RwID0gemZzY3RsX3NuYXBzaG90X2dldF9zbmFwZGlyKHZwLCAmZHZw KTsNCiANCiAJaWYgKCEobG9ja2VkID0gTVVURVhfSEVMRCgmc2RwLT5zZF9s b2NrKSkpDQogCQltdXRleF9lbnRlcigmc2RwLT5zZF9sb2NrKTsNCiANCisJ dm5fbG9jayh2cCwgTEtfRVhDTFVTSVZFIHwgTEtfUkVUUlkpOw0KKw0KKwlp ZiAodnAtPnZfY291bnQgPiAwKSB7DQorCQltdXRleF9leGl0KCZzZHAtPnNk X2xvY2spOw0KKwkJcmV0dXJuICgwKTsNCisJfQ0KKw0KIAlBU1NFUlQoIXZu X2lzbW50cHQodnApKTsNCiANCiAJc2VwID0gYXZsX2ZpcnN0KCZzZHAtPnNk X3NuYXBzKTsNCkBAIC0xNDk0LDcgKzE1MTcsNiBAQCB6ZnNjdGxfc25hcHNo b3RfaW5hY3RpdmUoYXApDQogCQltdXRleF9leGl0KCZzZHAtPnNkX2xvY2sp Ow0KIAlWTl9SRUxFKGR2cCk7DQogDQotZW5kOg0KIAkvKg0KIAkgKiBEaXNw b3NlIG9mIHRoZSB2bm9kZSBmb3IgdGhlIHNuYXBzaG90IG1vdW50IHBvaW50 Lg0KIAkgKiBUaGlzIGlzIHNhZmUgdG8gZG8gYmVjYXVzZSBvbmNlIHRoaXMg ZW50cnkgaGFzIGJlZW4gcmVtb3ZlZA0KQEAgLTE1OTUsMjAgKzE2MTcsMTcg QEAgemZzY3RsX3NuYXBzaG90X2xvb2t1cChhcCkNCiBzdGF0aWMgaW50DQog emZzY3RsX3NuYXBzaG90X3ZwdG9jbnAoc3RydWN0IHZvcF92cHRvY25wX2Fy Z3MgKmFwKQ0KIHsNCi0JemZzdmZzX3QgKnpmc3ZmcyA9IGFwLT5hX3ZwLT52 X3Zmc3AtPnZmc19kYXRhOw0KLQl2bm9kZV90ICpkdnAsICp2cDsNCisJdm5v ZGVfdCAqZHZwLCAqdnAgPSBhcC0+YV92cDsNCiAJemZzY3RsX3NuYXBkaXJf dCAqc2RwOw0KIAl6ZnNfc25hcGVudHJ5X3QgKnNlcDsNCi0JaW50IGVycm9y Ow0KKwlpbnQgZXJyb3IgPSAwOw0KIA0KLQlBU1NFUlQoemZzdmZzLT56X2N0 bGRpciAhPSBOVUxMKTsNCi0JZXJyb3IgPSB6ZnNjdGxfcm9vdF9sb29rdXAo emZzdmZzLT56X2N0bGRpciwgInNuYXBzaG90IiwgJmR2cCwNCi0JICAgIE5V TEwsIDAsIE5VTEwsIGtjcmVkLCBOVUxMLCBOVUxMLCBOVUxMKTsNCi0JaWYg KGVycm9yICE9IDApDQotCQlyZXR1cm4gKGVycm9yKTsNCi0Jc2RwID0gZHZw LT52X2RhdGE7DQorCXNkcCA9IHpmc2N0bF9zbmFwc2hvdF9nZXRfc25hcGRp cih2cCwgJmR2cCk7DQogDQogCW11dGV4X2VudGVyKCZzZHAtPnNkX2xvY2sp Ow0KKw0KKwl2bl9sb2NrKHZwLCBMS19TSEFSRUQgfCBMS19SRVRSWSk7DQor DQogCXNlcCA9IGF2bF9maXJzdCgmc2RwLT5zZF9zbmFwcyk7DQogCXdoaWxl IChzZXAgIT0gTlVMTCkgew0KIAkJdnAgPSBzZXAtPnNlX3Jvb3Q7DQotLSAN CjEuOC40LjINCg0K --1030603365-686922855-1387467971=:4344-- --0ntfKIWw70PvrIHh-- --MnLPg7ZWsaic7Fhd Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (FreeBSD) iEYEARECAAYFAlK0SRAACgkQForvXbEpPzSYmQCgmJrdom5npeVE3XL/3wAyPbav II8An2XzK7AlWBJMOau8BGBtMc88M+hi =qMz1 -----END PGP SIGNATURE----- --MnLPg7ZWsaic7Fhd-- From owner-zfs-devel@FreeBSD.ORG Fri Dec 20 18:15:08 2013 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 5FB11F2B; Fri, 20 Dec 2013 18:15:08 +0000 (UTC) Received: from mail-we0-x232.google.com (mail-we0-x232.google.com [IPv6:2a00:1450:400c:c03::232]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id BE5AA1CA2; Fri, 20 Dec 2013 18:15:07 +0000 (UTC) Received: by mail-we0-f178.google.com with SMTP id u57so2797762wes.23 for ; Fri, 20 Dec 2013 10:15:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type:content-transfer-encoding; bh=2PO8oCbk8M+bMofo6yrHoAjW55wHRzj6BEzht2PlpKQ=; b=FPe8n26RftNNhI/rM/OiyE5f3oCytCF7FrzOwMBDLrxGoWBgUxC7zVEE7FU2m68aPx AL3sryWZfASzWAF5yKlqANFZaV+TqJNmyAUxacGN/WwOdh7O+NIX9BZge8I2hGZ+O6XQ /8I1cSp5j4LMaf5QObv6Wz8HsB7ff+5iL29R6hzHJY5Z8p+Gckdzy3GpENztsEU9NWRh LHS3ZXGVsmqZglBYNj6Bqrh/4paxktBl1QnbjnyGvnlxolqn4OCUpVXm2OoXOGNbQvzu zhsYKGY8ws/a/3/az3KhbzS6nb3HM1UT2FiXXUhummxjNuC9stLmX1bVcel2NQGvZJ1z HAzA== MIME-Version: 1.0 X-Received: by 10.194.61.211 with SMTP id s19mr3451250wjr.73.1387563306201; Fri, 20 Dec 2013 10:15:06 -0800 (PST) Sender: asomers@gmail.com Received: by 10.194.55.101 with HTTP; Fri, 20 Dec 2013 10:15:06 -0800 (PST) In-Reply-To: <20131220134136.GD1658@garage.freebsd.pl> References: <20131220134136.GD1658@garage.freebsd.pl> Date: Fri, 20 Dec 2013 11:15:06 -0700 X-Google-Sender-Auth: i-_80M2tOI_-sp4b7B9WtN8EzsM Message-ID: Subject: Re: [krichy@tvnetwork.hu: Re: kern/184677 / ZFS snapshot handling deadlocks] From: Alan Somers To: Pawel Jakub Dawidek Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: "zfs-devel@freebsd.org" X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Dec 2013 18:15:08 -0000 On Fri, Dec 20, 2013 at 6:41 AM, Pawel Jakub Dawidek wrot= e: > Guys, > > can someone please look into this report? It is very detailed and even > have a patch. I can't get to it right now, but I also remember someone > was going to rework the locking in vdev_geom.c. Did that happen? Maybe > some regression was introduced? I don't have time to thoroughly review the patch, but I'll put it through our test suite. > > -- > Pawel Jakub Dawidek http://www.wheelsystems.com > FreeBSD committer http://www.FreeBSD.org > Am I Evil? Yes, I Am! http://mobter.com > > > ---------- Forwarded message ---------- > From: krichy@tvnetwork.hu > To: freebsd-fs@freebsd.org > Cc: pawel@dawidek.net > Date: Thu, 19 Dec 2013 16:46:11 +0100 (CET) > Subject: Re: kern/184677 / ZFS snapshot handling deadlocks > Dear devs, > > I am attaching a more clear patch/fix for my snapshot handling issues (00= 02), but I would be happy if some ZFS expert would comment it. I am trying = to solve it at least for two weeks now, and an ACK or a NACK would be nice = from someone. Also a commit is reverted since it also caused deadlocks. I'v= e read its comments, which also eliminates deadlocks, but I did not find an= y reference how to produce that deadlock. In my view reverting that makes m= y issues disappear, but I dont know what new cases will it rise. > > I've rewritten traverse() to make more like upstream, added two extra VN_= HOLD()s to snapdir_lookup() when traverse returned the same vnode what was = passed to it (I dont know even that upon creating a snapshot vnode why is t= hat extra two holds needed for the vnode.) And unfortunately, due to FreeBS= D calls vop_inactive callbacks with vnodes locked, that could also cause de= adlocks, so zfsctl_snapshot_inactive() and zfsctl_snapshot_vptocnp() has be= en rewritten to work that around. > > After this, one may also get a deadlock, when a simple access would call = into zfsctl_snapshot_lookup(). The documentation says, that those vnodes sh= ould always be covered, but after some stress test, sometimes we hit that c= all, and that can cause again deadlocks. In our environment I've just uncom= mented that callback, which returns ENODIR on some calls, but at least not = a deadlock. > > The attached script can be used to reproduce my cases (would one confirm = that?), and after the patches applied, they disappear (confirm?). > > Thanks, > > > Kojedzinszky Richard > Euronet Magyarorszag Informatikai Zrt. > > On Tue, 17 Dec 2013, krichy@tvnetwork.hu wrote: > >> Date: Tue, 17 Dec 2013 14:50:16 +0100 (CET) >> From: krichy@tvnetwork.hu >> To: pjd@freebsd.org >> Cc: freebsd-fs@freebsd.org >> Subject: Re: kern/184677 (fwd) >> >> Dear devs, >> >> I will sum up my experience regarding the issue: >> >> The sympton is that a concurrent 'zfs send -R' and some activity on the = snapshot dir (or in the snapshot) may cause a deadlock. >> >> After investigating the problem, I found that zfs send umounts the snaps= hots, and that causes the deadlock, so later I tested only with concurrent = umount and the "activity". More later I found that listing the snapshots in= .zfs/snapshot/ and unounting them can cause the found deadlock, so I used = them for the tests. But for my surprise, instead of a deadlock, a recursive= lock panic has arised. >> >> The vnode for the ".zfs/snapshot/" directory contains zfs's zfsctl_snapd= ir_t structure (sdp). This contains a tree of mounted snapshots, and each e= ntry (sep) contains the vnode of entry on which the snapshot is mounted on = top (se_root). The strange is that the se_root member does not hold a refer= ence for the vnode, just a simple pointer to it. >> >> Upon entry lookup (zfsctl_snapdir_lookup()) the "snapshot" vnode is lock= ed, the zfsctl_snapdir_t's tree is locked, and searched for the mount if it= exists already. If it founds no entry, does the mount. In the case of an e= ntry was found, the se_root member contains the vnode which the snapshot is= mounted on. Thus, a reference is taken for it, and the traverse() call wil= l resolve to the real root vnode of the mounted snapshot, returning it as l= ocked. (Examining the traverse() code I've found that it did not follow Fre= eBSD's lock order recommendation described in sys/kern/vfs_subr.c.) >> >> On the other way, when an umount is issued, the se_root vnode looses its= last reference (as only the mountpoint holds one for it), it goes through = the vinactive() path, to zfsctl_snapshot_inactive(). In FreeBSD this is cal= led with a locked vnode, so this is a deadlock race condition. While zfsctl= _snapdir_lookup() holds the mutex for the sdp tree, and traverse() tries to= acquire the se_root, zfsctl_snapshot_inactive() holds the lock on se_root = while tries to access the sdp lock. >> >> The zfsctl_snapshot_inactive() has an if statement checking the v_usecou= nt, which is incremented in zfsctl_snapdir_lookup(), but in that context it= is not covered by VI_LOCK. And it seems to me that FreeBSD's vinactive() p= ath assumes that the vnode remains inactive (as opposed to illumos, at leas= t how i read the code). So zfsctl_snapshot_inactive() must free resources w= hile in a locked state. I was a bit confused, and probably that is why the = previously posted patch is as is. >> >> Maybe if I had some clues on the directions of this problem, I could hav= e worked more for a nicer, shorter solution. >> >> Please someone comment on my post. >> >> Regards, >> >> >> >> Kojedzinszky Richard >> Euronet Magyarorszag Informatikai Zrt. >> >> On Mon, 16 Dec 2013, krichy@tvnetwork.hu wrote: >> >>> Date: Mon, 16 Dec 2013 16:52:16 +0100 (CET) >>> From: krichy@tvnetwork.hu >>> To: pjd@freebsd.org >>> Cc: freebsd-fs@freebsd.org >>> Subject: Re: kern/184677 (fwd) >>> >>> Dear PJD, >>> >>> I am a happy FreeBSD user, I am sure you've read my previous posts rega= rding some issues in ZFS. Please give some advice for me, where to look for= solutions, or how could I help to resolve those issues. >>> >>> Regards, >>> Kojedzinszky Richard >>> Euronet Magyarorszag Informatikai Zrt. >>> >>> ---------- Forwarded message ---------- >>> Date: Mon, 16 Dec 2013 15:23:06 +0100 (CET) >>> From: krichy@tvnetwork.hu >>> To: freebsd-fs@freebsd.org >>> Subject: Re: kern/184677 >>> >>> >>> Seems that pjd did a change which eliminated the zfsdev_state_lock on F= ri Aug 12 07:04:16 2011 +0000, which might introduced a new deadlock situat= ion. Any comments on this? >>> >>> >>> Kojedzinszky Richard >>> Euronet Magyarorszag Informatikai Zrt. >>> >>> On Mon, 16 Dec 2013, krichy@tvnetwork.hu wrote: >>> >>>> Date: Mon, 16 Dec 2013 11:08:11 +0100 (CET) >>>> From: krichy@tvnetwork.hu >>>> To: freebsd-fs@freebsd.org >>>> Subject: kern/184677 >>>> >>>> Dear devs, >>>> >>>> I've attached a patch, which makes the recursive lockmgr disappear, an= d makes the reported bug to disappear. I dont know if I followed any guidel= ines well, or not, but at least it works for me. Please some ZFS/FreeBSD fs= expert review it, and fix it where it needed. >>>> >>>> But unfortunately, my original problem is still not solved, maybe the = same as Ryan's: http://lists.freebsd.org/pipermail/freebsd-fs/2013-December= /018707.html >>>> >>>> Tracing the problem down is that zfsctl_snapdir_lookup() tries to acqu= ire spa_namespace_lock while when finishing a zfs send -R does a zfsdev_clo= se(), and that also holds the same mutex. And this causes a deadlock scenar= io. I looked at illumos's code, and for some reason they use another mutex = on zfsdev_close(), which therefore may not deadlock with zfsctl_snapdir_loo= kup(). But I am still investigating the problem. >>>> >>>> I would like to help making ZFS more stable on freebsd also with its w= hole functionality. I would be very thankful if some expert would give some= advice, how to solve these bugs. PJD, Steven, Xin? >>>> >>>> Thanks in advance, >>>> >>>> >>>> Kojedzinszky Richard >>>> Euronet Magyarorszag Informatikai Zrt. >>> >>> _______________________________________________ >>> freebsd-fs@freebsd.org mailing list >>> http://lists.freebsd.org/mailman/listinfo/freebsd-fs >>> To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" >>> > From owner-zfs-devel@FreeBSD.ORG Fri Dec 20 18:33:56 2013 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 62724ED9 for ; Fri, 20 Dec 2013 18:33:56 +0000 (UTC) Received: from mail-vb0-f48.google.com (mail-vb0-f48.google.com [209.85.212.48]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 18F281E6B for ; Fri, 20 Dec 2013 18:33:55 +0000 (UTC) Received: by mail-vb0-f48.google.com with SMTP id f13so1578089vbg.7 for ; Fri, 20 Dec 2013 10:33:49 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=ZjVMW8dMeT+V5QMtlPpPAPdoabedYOpMXDtsO5NWtuA=; b=IzAyYImSEKO1XPzKG1seZb6CRmAuxidFewRbW4Vl3rNqyGV0duhP5iiV67Xl3X+232 5uTLV+ejW+/g80lqK8kA5crpUEUpu2RZ8C4BmOSXcviKrqE9bk35WGc7WXFoHolAEIRq ugGTnAh9UHdWcml+97j3NWUdJBfSYe2srOVbVI9jUFzScnC+GPdlvCy8PHywv6kvrtvi OwdEtqPe+LOZhPCzTkkxTGZ7tFksjgbmBgEBFyxC8DqEt6Fsf4aJf8n/DXBih5x5Tv9W cMFLsjHcZwCyBAHUxZKLzo+lK4FsKxQpcBugU/aByMlWRnU+9bFJ2ytR77afqYyrxXKv BHzg== X-Gm-Message-State: ALoCoQn5hsts3O7vabG39PWwkbXqNnTHN0ZUQkYxZiylHDeax2UrqHMoQmP75ct4HWduNvn4BrOl MIME-Version: 1.0 X-Received: by 10.52.231.130 with SMTP id tg2mr4923290vdc.16.1387564429536; Fri, 20 Dec 2013 10:33:49 -0800 (PST) Received: by 10.58.197.39 with HTTP; Fri, 20 Dec 2013 10:33:49 -0800 (PST) In-Reply-To: References: <20131220134136.GD1658@garage.freebsd.pl> Date: Fri, 20 Dec 2013 11:33:49 -0700 Message-ID: Subject: Re: [krichy@tvnetwork.hu: Re: kern/184677 / ZFS snapshot handling deadlocks] From: Will Andrews To: Alan Somers Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: "zfs-devel@freebsd.org" X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Dec 2013 18:33:56 -0000 There's a good chance it's already fixed in SpectraBSD ZFS. I fixed a very similar bug (panic on snapshot unmount) back in July/August that involved fixing issues related to the GFS/snapdir port. The underlying bug was that we don't move process refcounts to the proper vnode. It sounds like the reported issue is different, but caused by the same underlying bugs. I was planning to port the patch (SpectraBSD/stable c/l 974581) to FreeBSD, but haven't had time yet. --Will. On Fri, Dec 20, 2013 at 11:15 AM, Alan Somers wrote: > On Fri, Dec 20, 2013 at 6:41 AM, Pawel Jakub Dawidek wr= ote: >> Guys, >> >> can someone please look into this report? It is very detailed and even >> have a patch. I can't get to it right now, but I also remember someone >> was going to rework the locking in vdev_geom.c. Did that happen? Maybe >> some regression was introduced? > > I don't have time to thoroughly review the patch, but I'll put it > through our test suite. > >> >> -- >> Pawel Jakub Dawidek http://www.wheelsystems.com >> FreeBSD committer http://www.FreeBSD.org >> Am I Evil? Yes, I Am! http://mobter.com >> >> >> ---------- Forwarded message ---------- >> From: krichy@tvnetwork.hu >> To: freebsd-fs@freebsd.org >> Cc: pawel@dawidek.net >> Date: Thu, 19 Dec 2013 16:46:11 +0100 (CET) >> Subject: Re: kern/184677 / ZFS snapshot handling deadlocks >> Dear devs, >> >> I am attaching a more clear patch/fix for my snapshot handling issues (0= 002), but I would be happy if some ZFS expert would comment it. I am trying= to solve it at least for two weeks now, and an ACK or a NACK would be nice= from someone. Also a commit is reverted since it also caused deadlocks. I'= ve read its comments, which also eliminates deadlocks, but I did not find a= ny reference how to produce that deadlock. In my view reverting that makes = my issues disappear, but I dont know what new cases will it rise. >> >> I've rewritten traverse() to make more like upstream, added two extra VN= _HOLD()s to snapdir_lookup() when traverse returned the same vnode what was= passed to it (I dont know even that upon creating a snapshot vnode why is = that extra two holds needed for the vnode.) And unfortunately, due to FreeB= SD calls vop_inactive callbacks with vnodes locked, that could also cause d= eadlocks, so zfsctl_snapshot_inactive() and zfsctl_snapshot_vptocnp() has b= een rewritten to work that around. >> >> After this, one may also get a deadlock, when a simple access would call= into zfsctl_snapshot_lookup(). The documentation says, that those vnodes s= hould always be covered, but after some stress test, sometimes we hit that = call, and that can cause again deadlocks. In our environment I've just unco= mmented that callback, which returns ENODIR on some calls, but at least not= a deadlock. >> >> The attached script can be used to reproduce my cases (would one confirm= that?), and after the patches applied, they disappear (confirm?). >> >> Thanks, >> >> >> Kojedzinszky Richard >> Euronet Magyarorszag Informatikai Zrt. >> >> On Tue, 17 Dec 2013, krichy@tvnetwork.hu wrote: >> >>> Date: Tue, 17 Dec 2013 14:50:16 +0100 (CET) >>> From: krichy@tvnetwork.hu >>> To: pjd@freebsd.org >>> Cc: freebsd-fs@freebsd.org >>> Subject: Re: kern/184677 (fwd) >>> >>> Dear devs, >>> >>> I will sum up my experience regarding the issue: >>> >>> The sympton is that a concurrent 'zfs send -R' and some activity on the= snapshot dir (or in the snapshot) may cause a deadlock. >>> >>> After investigating the problem, I found that zfs send umounts the snap= shots, and that causes the deadlock, so later I tested only with concurrent= umount and the "activity". More later I found that listing the snapshots i= n .zfs/snapshot/ and unounting them can cause the found deadlock, so I used= them for the tests. But for my surprise, instead of a deadlock, a recursiv= e lock panic has arised. >>> >>> The vnode for the ".zfs/snapshot/" directory contains zfs's zfsctl_snap= dir_t structure (sdp). This contains a tree of mounted snapshots, and each = entry (sep) contains the vnode of entry on which the snapshot is mounted on= top (se_root). The strange is that the se_root member does not hold a refe= rence for the vnode, just a simple pointer to it. >>> >>> Upon entry lookup (zfsctl_snapdir_lookup()) the "snapshot" vnode is loc= ked, the zfsctl_snapdir_t's tree is locked, and searched for the mount if i= t exists already. If it founds no entry, does the mount. In the case of an = entry was found, the se_root member contains the vnode which the snapshot i= s mounted on. Thus, a reference is taken for it, and the traverse() call wi= ll resolve to the real root vnode of the mounted snapshot, returning it as = locked. (Examining the traverse() code I've found that it did not follow Fr= eeBSD's lock order recommendation described in sys/kern/vfs_subr.c.) >>> >>> On the other way, when an umount is issued, the se_root vnode looses it= s last reference (as only the mountpoint holds one for it), it goes through= the vinactive() path, to zfsctl_snapshot_inactive(). In FreeBSD this is ca= lled with a locked vnode, so this is a deadlock race condition. While zfsct= l_snapdir_lookup() holds the mutex for the sdp tree, and traverse() tries t= o acquire the se_root, zfsctl_snapshot_inactive() holds the lock on se_root= while tries to access the sdp lock. >>> >>> The zfsctl_snapshot_inactive() has an if statement checking the v_useco= unt, which is incremented in zfsctl_snapdir_lookup(), but in that context i= t is not covered by VI_LOCK. And it seems to me that FreeBSD's vinactive() = path assumes that the vnode remains inactive (as opposed to illumos, at lea= st how i read the code). So zfsctl_snapshot_inactive() must free resources = while in a locked state. I was a bit confused, and probably that is why the= previously posted patch is as is. >>> >>> Maybe if I had some clues on the directions of this problem, I could ha= ve worked more for a nicer, shorter solution. >>> >>> Please someone comment on my post. >>> >>> Regards, >>> >>> >>> >>> Kojedzinszky Richard >>> Euronet Magyarorszag Informatikai Zrt. >>> >>> On Mon, 16 Dec 2013, krichy@tvnetwork.hu wrote: >>> >>>> Date: Mon, 16 Dec 2013 16:52:16 +0100 (CET) >>>> From: krichy@tvnetwork.hu >>>> To: pjd@freebsd.org >>>> Cc: freebsd-fs@freebsd.org >>>> Subject: Re: kern/184677 (fwd) >>>> >>>> Dear PJD, >>>> >>>> I am a happy FreeBSD user, I am sure you've read my previous posts reg= arding some issues in ZFS. Please give some advice for me, where to look fo= r solutions, or how could I help to resolve those issues. >>>> >>>> Regards, >>>> Kojedzinszky Richard >>>> Euronet Magyarorszag Informatikai Zrt. >>>> >>>> ---------- Forwarded message ---------- >>>> Date: Mon, 16 Dec 2013 15:23:06 +0100 (CET) >>>> From: krichy@tvnetwork.hu >>>> To: freebsd-fs@freebsd.org >>>> Subject: Re: kern/184677 >>>> >>>> >>>> Seems that pjd did a change which eliminated the zfsdev_state_lock on = Fri Aug 12 07:04:16 2011 +0000, which might introduced a new deadlock situa= tion. Any comments on this? >>>> >>>> >>>> Kojedzinszky Richard >>>> Euronet Magyarorszag Informatikai Zrt. >>>> >>>> On Mon, 16 Dec 2013, krichy@tvnetwork.hu wrote: >>>> >>>>> Date: Mon, 16 Dec 2013 11:08:11 +0100 (CET) >>>>> From: krichy@tvnetwork.hu >>>>> To: freebsd-fs@freebsd.org >>>>> Subject: kern/184677 >>>>> >>>>> Dear devs, >>>>> >>>>> I've attached a patch, which makes the recursive lockmgr disappear, a= nd makes the reported bug to disappear. I dont know if I followed any guide= lines well, or not, but at least it works for me. Please some ZFS/FreeBSD f= s expert review it, and fix it where it needed. >>>>> >>>>> But unfortunately, my original problem is still not solved, maybe the= same as Ryan's: http://lists.freebsd.org/pipermail/freebsd-fs/2013-Decembe= r/018707.html >>>>> >>>>> Tracing the problem down is that zfsctl_snapdir_lookup() tries to acq= uire spa_namespace_lock while when finishing a zfs send -R does a zfsdev_cl= ose(), and that also holds the same mutex. And this causes a deadlock scena= rio. I looked at illumos's code, and for some reason they use another mutex= on zfsdev_close(), which therefore may not deadlock with zfsctl_snapdir_lo= okup(). But I am still investigating the problem. >>>>> >>>>> I would like to help making ZFS more stable on freebsd also with its = whole functionality. I would be very thankful if some expert would give som= e advice, how to solve these bugs. PJD, Steven, Xin? >>>>> >>>>> Thanks in advance, >>>>> >>>>> >>>>> Kojedzinszky Richard >>>>> Euronet Magyarorszag Informatikai Zrt. >>>> >>>> _______________________________________________ >>>> freebsd-fs@freebsd.org mailing list >>>> http://lists.freebsd.org/mailman/listinfo/freebsd-fs >>>> To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" >>>> >> > _______________________________________________ > zfs-devel@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/zfs-devel > To unsubscribe, send any mail to "zfs-devel-unsubscribe@freebsd.org" From owner-zfs-devel@FreeBSD.ORG Fri Dec 20 21:05:31 2013 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A695A903; Fri, 20 Dec 2013 21:05:31 +0000 (UTC) Received: from mail-wi0-x229.google.com (mail-wi0-x229.google.com [IPv6:2a00:1450:400c:c05::229]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 0E7291A61; Fri, 20 Dec 2013 21:05:30 +0000 (UTC) Received: by mail-wi0-f169.google.com with SMTP id j9so619990wiv.0 for ; Fri, 20 Dec 2013 13:05:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type:content-transfer-encoding; bh=pi5EG8Q0W33qe5O9lG4d6Vm2LeRH2ALZTDbOYgDlTdM=; b=PIO8/yG3DjU0GDRy0QWZe1H/MA8SvymuGhdei5eFu8WmicqXkWSVe3SnSqx2DSWJXu IsZJqGLMvbP7kq5Bd5F0CxFrtuKhsVmR6h6+HPj0Sgu/2jYKQsBb5sUdRb6tPlj18jh3 ADxPxff/nj3HZomazKOFfxTc5I2Dr8VFWkql8LforIFtYXlw9nP/xnX50vGLghy3fm81 OJOvwl/BCYNFtM0pc+HI5V0sSN+VpLVdZhScD03496VsTiB0DIF0Z7UfBqVyfQqhFQfT BD+Qv6BW/vo0jKyB9q3HMYDYJdtczoV//Ez76MPGh8VTzJqH0yzKIw0N89NF4DKfiv0R wv0g== MIME-Version: 1.0 X-Received: by 10.180.11.34 with SMTP id n2mr4791506wib.40.1387573529346; Fri, 20 Dec 2013 13:05:29 -0800 (PST) Sender: asomers@gmail.com Received: by 10.194.55.101 with HTTP; Fri, 20 Dec 2013 13:05:29 -0800 (PST) In-Reply-To: References: <20131220134136.GD1658@garage.freebsd.pl> Date: Fri, 20 Dec 2013 14:05:29 -0700 X-Google-Sender-Auth: b1EFFTANotFOq3imzCnXCxytnLI Message-ID: Subject: Re: [krichy@tvnetwork.hu: Re: kern/184677 / ZFS snapshot handling deadlocks] From: Alan Somers To: Will Andrews Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: "zfs-devel@freebsd.org" X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Dec 2013 21:05:31 -0000 On Fri, Dec 20, 2013 at 11:33 AM, Will Andrews wrote: > There's a good chance it's already fixed in SpectraBSD ZFS. I fixed a > very similar bug (panic on snapshot unmount) back in July/August that > involved fixing issues related to the GFS/snapdir port. The > underlying bug was that we don't move process refcounts to the proper > vnode. It sounds like the reported issue is different, but caused by > the same underlying bugs. Definitely not fixed. I just reproduced the panic using Krichy's script. Next, I'll try to integrate the patch. > > I was planning to port the patch (SpectraBSD/stable c/l 974581) to > FreeBSD, but haven't had time yet. > > --Will. > > On Fri, Dec 20, 2013 at 11:15 AM, Alan Somers wrote= : >> On Fri, Dec 20, 2013 at 6:41 AM, Pawel Jakub Dawidek w= rote: >>> Guys, >>> >>> can someone please look into this report? It is very detailed and even >>> have a patch. I can't get to it right now, but I also remember someone >>> was going to rework the locking in vdev_geom.c. Did that happen? Maybe >>> some regression was introduced? >> >> I don't have time to thoroughly review the patch, but I'll put it >> through our test suite. >> >>> >>> -- >>> Pawel Jakub Dawidek http://www.wheelsystems.com >>> FreeBSD committer http://www.FreeBSD.org >>> Am I Evil? Yes, I Am! http://mobter.com >>> >>> >>> ---------- Forwarded message ---------- >>> From: krichy@tvnetwork.hu >>> To: freebsd-fs@freebsd.org >>> Cc: pawel@dawidek.net >>> Date: Thu, 19 Dec 2013 16:46:11 +0100 (CET) >>> Subject: Re: kern/184677 / ZFS snapshot handling deadlocks >>> Dear devs, >>> >>> I am attaching a more clear patch/fix for my snapshot handling issues (= 0002), but I would be happy if some ZFS expert would comment it. I am tryin= g to solve it at least for two weeks now, and an ACK or a NACK would be nic= e from someone. Also a commit is reverted since it also caused deadlocks. I= 've read its comments, which also eliminates deadlocks, but I did not find = any reference how to produce that deadlock. In my view reverting that makes= my issues disappear, but I dont know what new cases will it rise. >>> >>> I've rewritten traverse() to make more like upstream, added two extra V= N_HOLD()s to snapdir_lookup() when traverse returned the same vnode what wa= s passed to it (I dont know even that upon creating a snapshot vnode why is= that extra two holds needed for the vnode.) And unfortunately, due to Free= BSD calls vop_inactive callbacks with vnodes locked, that could also cause = deadlocks, so zfsctl_snapshot_inactive() and zfsctl_snapshot_vptocnp() has = been rewritten to work that around. >>> >>> After this, one may also get a deadlock, when a simple access would cal= l into zfsctl_snapshot_lookup(). The documentation says, that those vnodes = should always be covered, but after some stress test, sometimes we hit that= call, and that can cause again deadlocks. In our environment I've just unc= ommented that callback, which returns ENODIR on some calls, but at least no= t a deadlock. >>> >>> The attached script can be used to reproduce my cases (would one confir= m that?), and after the patches applied, they disappear (confirm?). >>> >>> Thanks, >>> >>> >>> Kojedzinszky Richard >>> Euronet Magyarorszag Informatikai Zrt. >>> >>> On Tue, 17 Dec 2013, krichy@tvnetwork.hu wrote: >>> >>>> Date: Tue, 17 Dec 2013 14:50:16 +0100 (CET) >>>> From: krichy@tvnetwork.hu >>>> To: pjd@freebsd.org >>>> Cc: freebsd-fs@freebsd.org >>>> Subject: Re: kern/184677 (fwd) >>>> >>>> Dear devs, >>>> >>>> I will sum up my experience regarding the issue: >>>> >>>> The sympton is that a concurrent 'zfs send -R' and some activity on th= e snapshot dir (or in the snapshot) may cause a deadlock. >>>> >>>> After investigating the problem, I found that zfs send umounts the sna= pshots, and that causes the deadlock, so later I tested only with concurren= t umount and the "activity". More later I found that listing the snapshots = in .zfs/snapshot/ and unounting them can cause the found deadlock, so I use= d them for the tests. But for my surprise, instead of a deadlock, a recursi= ve lock panic has arised. >>>> >>>> The vnode for the ".zfs/snapshot/" directory contains zfs's zfsctl_sna= pdir_t structure (sdp). This contains a tree of mounted snapshots, and each= entry (sep) contains the vnode of entry on which the snapshot is mounted o= n top (se_root). The strange is that the se_root member does not hold a ref= erence for the vnode, just a simple pointer to it. >>>> >>>> Upon entry lookup (zfsctl_snapdir_lookup()) the "snapshot" vnode is lo= cked, the zfsctl_snapdir_t's tree is locked, and searched for the mount if = it exists already. If it founds no entry, does the mount. In the case of an= entry was found, the se_root member contains the vnode which the snapshot = is mounted on. Thus, a reference is taken for it, and the traverse() call w= ill resolve to the real root vnode of the mounted snapshot, returning it as= locked. (Examining the traverse() code I've found that it did not follow F= reeBSD's lock order recommendation described in sys/kern/vfs_subr.c.) >>>> >>>> On the other way, when an umount is issued, the se_root vnode looses i= ts last reference (as only the mountpoint holds one for it), it goes throug= h the vinactive() path, to zfsctl_snapshot_inactive(). In FreeBSD this is c= alled with a locked vnode, so this is a deadlock race condition. While zfsc= tl_snapdir_lookup() holds the mutex for the sdp tree, and traverse() tries = to acquire the se_root, zfsctl_snapshot_inactive() holds the lock on se_roo= t while tries to access the sdp lock. >>>> >>>> The zfsctl_snapshot_inactive() has an if statement checking the v_usec= ount, which is incremented in zfsctl_snapdir_lookup(), but in that context = it is not covered by VI_LOCK. And it seems to me that FreeBSD's vinactive()= path assumes that the vnode remains inactive (as opposed to illumos, at le= ast how i read the code). So zfsctl_snapshot_inactive() must free resources= while in a locked state. I was a bit confused, and probably that is why th= e previously posted patch is as is. >>>> >>>> Maybe if I had some clues on the directions of this problem, I could h= ave worked more for a nicer, shorter solution. >>>> >>>> Please someone comment on my post. >>>> >>>> Regards, >>>> >>>> >>>> >>>> Kojedzinszky Richard >>>> Euronet Magyarorszag Informatikai Zrt. >>>> >>>> On Mon, 16 Dec 2013, krichy@tvnetwork.hu wrote: >>>> >>>>> Date: Mon, 16 Dec 2013 16:52:16 +0100 (CET) >>>>> From: krichy@tvnetwork.hu >>>>> To: pjd@freebsd.org >>>>> Cc: freebsd-fs@freebsd.org >>>>> Subject: Re: kern/184677 (fwd) >>>>> >>>>> Dear PJD, >>>>> >>>>> I am a happy FreeBSD user, I am sure you've read my previous posts re= garding some issues in ZFS. Please give some advice for me, where to look f= or solutions, or how could I help to resolve those issues. >>>>> >>>>> Regards, >>>>> Kojedzinszky Richard >>>>> Euronet Magyarorszag Informatikai Zrt. >>>>> >>>>> ---------- Forwarded message ---------- >>>>> Date: Mon, 16 Dec 2013 15:23:06 +0100 (CET) >>>>> From: krichy@tvnetwork.hu >>>>> To: freebsd-fs@freebsd.org >>>>> Subject: Re: kern/184677 >>>>> >>>>> >>>>> Seems that pjd did a change which eliminated the zfsdev_state_lock on= Fri Aug 12 07:04:16 2011 +0000, which might introduced a new deadlock situ= ation. Any comments on this? >>>>> >>>>> >>>>> Kojedzinszky Richard >>>>> Euronet Magyarorszag Informatikai Zrt. >>>>> >>>>> On Mon, 16 Dec 2013, krichy@tvnetwork.hu wrote: >>>>> >>>>>> Date: Mon, 16 Dec 2013 11:08:11 +0100 (CET) >>>>>> From: krichy@tvnetwork.hu >>>>>> To: freebsd-fs@freebsd.org >>>>>> Subject: kern/184677 >>>>>> >>>>>> Dear devs, >>>>>> >>>>>> I've attached a patch, which makes the recursive lockmgr disappear, = and makes the reported bug to disappear. I dont know if I followed any guid= elines well, or not, but at least it works for me. Please some ZFS/FreeBSD = fs expert review it, and fix it where it needed. >>>>>> >>>>>> But unfortunately, my original problem is still not solved, maybe th= e same as Ryan's: http://lists.freebsd.org/pipermail/freebsd-fs/2013-Decemb= er/018707.html >>>>>> >>>>>> Tracing the problem down is that zfsctl_snapdir_lookup() tries to ac= quire spa_namespace_lock while when finishing a zfs send -R does a zfsdev_c= lose(), and that also holds the same mutex. And this causes a deadlock scen= ario. I looked at illumos's code, and for some reason they use another mute= x on zfsdev_close(), which therefore may not deadlock with zfsctl_snapdir_l= ookup(). But I am still investigating the problem. >>>>>> >>>>>> I would like to help making ZFS more stable on freebsd also with its= whole functionality. I would be very thankful if some expert would give so= me advice, how to solve these bugs. PJD, Steven, Xin? >>>>>> >>>>>> Thanks in advance, >>>>>> >>>>>> >>>>>> Kojedzinszky Richard >>>>>> Euronet Magyarorszag Informatikai Zrt. >>>>> >>>>> _______________________________________________ >>>>> freebsd-fs@freebsd.org mailing list >>>>> http://lists.freebsd.org/mailman/listinfo/freebsd-fs >>>>> To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" >>>>> >>> >> _______________________________________________ >> zfs-devel@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/zfs-devel >> To unsubscribe, send any mail to "zfs-devel-unsubscribe@freebsd.org" From owner-zfs-devel@FreeBSD.ORG Tue Dec 24 17:21:46 2013 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 28D68F97; Tue, 24 Dec 2013 17:21:46 +0000 (UTC) Received: from mail-wi0-x234.google.com (mail-wi0-x234.google.com [IPv6:2a00:1450:400c:c05::234]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 63D981298; Tue, 24 Dec 2013 17:21:45 +0000 (UTC) Received: by mail-wi0-f180.google.com with SMTP id hm19so7482528wib.7 for ; Tue, 24 Dec 2013 09:21:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type:content-transfer-encoding; bh=NQalhp0u08PUd6Zk5uf5V51sRl0OIVE36Vq/pBwUOd8=; b=uzfk1tz3ZM7R2N7PxblxDe1rf4l1ePLU2Zd2wMavJJzhpDABe6HyPILWrVXGtyVyVS OnG/IJeX2+cFEkXUuDuQxuK/iynD1qRPXl8CJPn+w5JuVtGlZS9ViWnLsqt65h6t1jDU mUlSke5azN2GbtZSt141+Qa5nr3z2juC1vJSelFekEa2i8ivP+iSVYOj4dTb0bjFe4tJ MYHcydvNKECh9cU+HjsG5v//SUTiqPsnwxj6r+9SupQJSzleviJLp4253ZYjNqvRP1dZ uUkFULYqUhEkhinMWprq3WhEzKKdfXwjz24iz76iwkNopA2I5eWDPdiyro1rF3IOZzv1 nxwQ== MIME-Version: 1.0 X-Received: by 10.180.105.199 with SMTP id go7mr23248052wib.53.1387905702977; Tue, 24 Dec 2013 09:21:42 -0800 (PST) Sender: asomers@gmail.com Received: by 10.194.164.6 with HTTP; Tue, 24 Dec 2013 09:21:42 -0800 (PST) In-Reply-To: References: <20131220134136.GD1658@garage.freebsd.pl> Date: Tue, 24 Dec 2013 10:21:42 -0700 X-Google-Sender-Auth: e_wM4DNYMGs3Ewyrm-QeFSJTcdg Message-ID: Subject: Re: [krichy@tvnetwork.hu: Re: kern/184677 / ZFS snapshot handling deadlocks] From: Alan Somers To: Alan Somers Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: Will Andrews , "zfs-devel@freebsd.org" X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Dec 2013 17:21:46 -0000 On Fri, Dec 20, 2013 at 2:05 PM, Alan Somers wrote: > On Fri, Dec 20, 2013 at 11:33 AM, Will Andrews wrote: >> There's a good chance it's already fixed in SpectraBSD ZFS. I fixed a >> very similar bug (panic on snapshot unmount) back in July/August that >> involved fixing issues related to the GFS/snapdir port. The >> underlying bug was that we don't move process refcounts to the proper >> vnode. It sounds like the reported issue is different, but caused by >> the same underlying bugs. > > Definitely not fixed. I just reproduced the panic using Krichy's > script. Next, I'll try to integrate the patch. > >> >> I was planning to port the patch (SpectraBSD/stable c/l 974581) to >> FreeBSD, but haven't had time yet. Krichy's patch conflicts with Will's, and Will is on vacation. He'll be able to work on it after he gets back on 2-Jan. -Alan >> >> --Will. >> >> On Fri, Dec 20, 2013 at 11:15 AM, Alan Somers wrot= e: >>> On Fri, Dec 20, 2013 at 6:41 AM, Pawel Jakub Dawidek = wrote: >>>> Guys, >>>> >>>> can someone please look into this report? It is very detailed and even >>>> have a patch. I can't get to it right now, but I also remember someone >>>> was going to rework the locking in vdev_geom.c. Did that happen? Maybe >>>> some regression was introduced? >>> >>> I don't have time to thoroughly review the patch, but I'll put it >>> through our test suite. >>> >>>> >>>> -- >>>> Pawel Jakub Dawidek http://www.wheelsystems.com >>>> FreeBSD committer http://www.FreeBSD.org >>>> Am I Evil? Yes, I Am! http://mobter.com >>>> >>>> >>>> ---------- Forwarded message ---------- >>>> From: krichy@tvnetwork.hu >>>> To: freebsd-fs@freebsd.org >>>> Cc: pawel@dawidek.net >>>> Date: Thu, 19 Dec 2013 16:46:11 +0100 (CET) >>>> Subject: Re: kern/184677 / ZFS snapshot handling deadlocks >>>> Dear devs, >>>> >>>> I am attaching a more clear patch/fix for my snapshot handling issues = (0002), but I would be happy if some ZFS expert would comment it. I am tryi= ng to solve it at least for two weeks now, and an ACK or a NACK would be ni= ce from someone. Also a commit is reverted since it also caused deadlocks. = I've read its comments, which also eliminates deadlocks, but I did not find= any reference how to produce that deadlock. In my view reverting that make= s my issues disappear, but I dont know what new cases will it rise. >>>> >>>> I've rewritten traverse() to make more like upstream, added two extra = VN_HOLD()s to snapdir_lookup() when traverse returned the same vnode what w= as passed to it (I dont know even that upon creating a snapshot vnode why i= s that extra two holds needed for the vnode.) And unfortunately, due to Fre= eBSD calls vop_inactive callbacks with vnodes locked, that could also cause= deadlocks, so zfsctl_snapshot_inactive() and zfsctl_snapshot_vptocnp() has= been rewritten to work that around. >>>> >>>> After this, one may also get a deadlock, when a simple access would ca= ll into zfsctl_snapshot_lookup(). The documentation says, that those vnodes= should always be covered, but after some stress test, sometimes we hit tha= t call, and that can cause again deadlocks. In our environment I've just un= commented that callback, which returns ENODIR on some calls, but at least n= ot a deadlock. >>>> >>>> The attached script can be used to reproduce my cases (would one confi= rm that?), and after the patches applied, they disappear (confirm?). >>>> >>>> Thanks, >>>> >>>> >>>> Kojedzinszky Richard >>>> Euronet Magyarorszag Informatikai Zrt. >>>> >>>> On Tue, 17 Dec 2013, krichy@tvnetwork.hu wrote: >>>> >>>>> Date: Tue, 17 Dec 2013 14:50:16 +0100 (CET) >>>>> From: krichy@tvnetwork.hu >>>>> To: pjd@freebsd.org >>>>> Cc: freebsd-fs@freebsd.org >>>>> Subject: Re: kern/184677 (fwd) >>>>> >>>>> Dear devs, >>>>> >>>>> I will sum up my experience regarding the issue: >>>>> >>>>> The sympton is that a concurrent 'zfs send -R' and some activity on t= he snapshot dir (or in the snapshot) may cause a deadlock. >>>>> >>>>> After investigating the problem, I found that zfs send umounts the sn= apshots, and that causes the deadlock, so later I tested only with concurre= nt umount and the "activity". More later I found that listing the snapshots= in .zfs/snapshot/ and unounting them can cause the found deadlock, so I us= ed them for the tests. But for my surprise, instead of a deadlock, a recurs= ive lock panic has arised. >>>>> >>>>> The vnode for the ".zfs/snapshot/" directory contains zfs's zfsctl_sn= apdir_t structure (sdp). This contains a tree of mounted snapshots, and eac= h entry (sep) contains the vnode of entry on which the snapshot is mounted = on top (se_root). The strange is that the se_root member does not hold a re= ference for the vnode, just a simple pointer to it. >>>>> >>>>> Upon entry lookup (zfsctl_snapdir_lookup()) the "snapshot" vnode is l= ocked, the zfsctl_snapdir_t's tree is locked, and searched for the mount if= it exists already. If it founds no entry, does the mount. In the case of a= n entry was found, the se_root member contains the vnode which the snapshot= is mounted on. Thus, a reference is taken for it, and the traverse() call = will resolve to the real root vnode of the mounted snapshot, returning it a= s locked. (Examining the traverse() code I've found that it did not follow = FreeBSD's lock order recommendation described in sys/kern/vfs_subr.c.) >>>>> >>>>> On the other way, when an umount is issued, the se_root vnode looses = its last reference (as only the mountpoint holds one for it), it goes throu= gh the vinactive() path, to zfsctl_snapshot_inactive(). In FreeBSD this is = called with a locked vnode, so this is a deadlock race condition. While zfs= ctl_snapdir_lookup() holds the mutex for the sdp tree, and traverse() tries= to acquire the se_root, zfsctl_snapshot_inactive() holds the lock on se_ro= ot while tries to access the sdp lock. >>>>> >>>>> The zfsctl_snapshot_inactive() has an if statement checking the v_use= count, which is incremented in zfsctl_snapdir_lookup(), but in that context= it is not covered by VI_LOCK. And it seems to me that FreeBSD's vinactive(= ) path assumes that the vnode remains inactive (as opposed to illumos, at l= east how i read the code). So zfsctl_snapshot_inactive() must free resource= s while in a locked state. I was a bit confused, and probably that is why t= he previously posted patch is as is. >>>>> >>>>> Maybe if I had some clues on the directions of this problem, I could = have worked more for a nicer, shorter solution. >>>>> >>>>> Please someone comment on my post. >>>>> >>>>> Regards, >>>>> >>>>> >>>>> >>>>> Kojedzinszky Richard >>>>> Euronet Magyarorszag Informatikai Zrt. >>>>> >>>>> On Mon, 16 Dec 2013, krichy@tvnetwork.hu wrote: >>>>> >>>>>> Date: Mon, 16 Dec 2013 16:52:16 +0100 (CET) >>>>>> From: krichy@tvnetwork.hu >>>>>> To: pjd@freebsd.org >>>>>> Cc: freebsd-fs@freebsd.org >>>>>> Subject: Re: kern/184677 (fwd) >>>>>> >>>>>> Dear PJD, >>>>>> >>>>>> I am a happy FreeBSD user, I am sure you've read my previous posts r= egarding some issues in ZFS. Please give some advice for me, where to look = for solutions, or how could I help to resolve those issues. >>>>>> >>>>>> Regards, >>>>>> Kojedzinszky Richard >>>>>> Euronet Magyarorszag Informatikai Zrt. >>>>>> >>>>>> ---------- Forwarded message ---------- >>>>>> Date: Mon, 16 Dec 2013 15:23:06 +0100 (CET) >>>>>> From: krichy@tvnetwork.hu >>>>>> To: freebsd-fs@freebsd.org >>>>>> Subject: Re: kern/184677 >>>>>> >>>>>> >>>>>> Seems that pjd did a change which eliminated the zfsdev_state_lock o= n Fri Aug 12 07:04:16 2011 +0000, which might introduced a new deadlock sit= uation. Any comments on this? >>>>>> >>>>>> >>>>>> Kojedzinszky Richard >>>>>> Euronet Magyarorszag Informatikai Zrt. >>>>>> >>>>>> On Mon, 16 Dec 2013, krichy@tvnetwork.hu wrote: >>>>>> >>>>>>> Date: Mon, 16 Dec 2013 11:08:11 +0100 (CET) >>>>>>> From: krichy@tvnetwork.hu >>>>>>> To: freebsd-fs@freebsd.org >>>>>>> Subject: kern/184677 >>>>>>> >>>>>>> Dear devs, >>>>>>> >>>>>>> I've attached a patch, which makes the recursive lockmgr disappear,= and makes the reported bug to disappear. I dont know if I followed any gui= delines well, or not, but at least it works for me. Please some ZFS/FreeBSD= fs expert review it, and fix it where it needed. >>>>>>> >>>>>>> But unfortunately, my original problem is still not solved, maybe t= he same as Ryan's: http://lists.freebsd.org/pipermail/freebsd-fs/2013-Decem= ber/018707.html >>>>>>> >>>>>>> Tracing the problem down is that zfsctl_snapdir_lookup() tries to a= cquire spa_namespace_lock while when finishing a zfs send -R does a zfsdev_= close(), and that also holds the same mutex. And this causes a deadlock sce= nario. I looked at illumos's code, and for some reason they use another mut= ex on zfsdev_close(), which therefore may not deadlock with zfsctl_snapdir_= lookup(). But I am still investigating the problem. >>>>>>> >>>>>>> I would like to help making ZFS more stable on freebsd also with it= s whole functionality. I would be very thankful if some expert would give s= ome advice, how to solve these bugs. PJD, Steven, Xin? >>>>>>> >>>>>>> Thanks in advance, >>>>>>> >>>>>>> >>>>>>> Kojedzinszky Richard >>>>>>> Euronet Magyarorszag Informatikai Zrt. >>>>>> >>>>>> _______________________________________________ >>>>>> freebsd-fs@freebsd.org mailing list >>>>>> http://lists.freebsd.org/mailman/listinfo/freebsd-fs >>>>>> To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org= " >>>>>> >>>> >>> _______________________________________________ >>> zfs-devel@freebsd.org mailing list >>> http://lists.freebsd.org/mailman/listinfo/zfs-devel >>> To unsubscribe, send any mail to "zfs-devel-unsubscribe@freebsd.org" From owner-zfs-devel@FreeBSD.ORG Sat Dec 28 08:01:20 2013 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 43FBCD6B for ; Sat, 28 Dec 2013 08:01:20 +0000 (UTC) Received: from pi.nmdps.net (pi.nmdps.net [IPv6:2a01:be00:10:201:0:80:0:1]) by mx1.freebsd.org (Postfix) with ESMTP id 06944181D for ; Sat, 28 Dec 2013 08:01:19 +0000 (UTC) Received: from pi.nmdps.net (pi.nmdps.net [109.61.102.5]) (Authenticated sender: krichy@cflinux.hu) by pi.nmdps.net (Postfix) with ESMTPSA id 88AA41676 for ; Sat, 28 Dec 2013 09:01:17 +0100 (CET) Date: Sat, 28 Dec 2013 09:01:15 +0100 (CET) From: Richard Kojedzinszky X-X-Sender: krichy@pi.nmdps.net To: zfs-devel@freebsd.org Subject: snapshot rename does not unmount it Message-ID: User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; format=flowed; charset=ISO-8859-15 Content-Transfer-Encoding: 8BIT X-Content-Filtered-By: Mailman/MimeDel 2.1.17 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 28 Dec 2013 08:01:20 -0000 Dear zfs-devel list, Gerrit Khn reported that renaming a snapshot, and after listing them in .zfs/snapshot/ dir shows some error. Attached a script which will demonstrate it. I found that (with git bisect) that https://github.com/freebsd/freebsd/commit/7c87858955593be19a80e57b7353b09f5587ae9b is causing the wrong behaviour, but unfortunately that commit is too big for me to understand. Thanks in advance, Kojedzinszky Richard From owner-zfs-devel@FreeBSD.ORG Sun Dec 29 22:46:09 2013 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id BA31F980 for ; Sun, 29 Dec 2013 22:46:09 +0000 (UTC) Received: from pi.nmdps.net (pi.nmdps.net [IPv6:2a01:be00:10:201:0:80:0:1]) by mx1.freebsd.org (Postfix) with ESMTP id 7DDCD1EE9 for ; Sun, 29 Dec 2013 22:46:09 +0000 (UTC) Received: from pi.nmdps.net (pi.nmdps.net [109.61.102.5]) (Authenticated sender: krichy@cflinux.hu) by pi.nmdps.net (Postfix) with ESMTPSA id BF4B51809 for ; Sun, 29 Dec 2013 23:45:58 +0100 (CET) Date: Sun, 29 Dec 2013 23:45:56 +0100 (CET) From: Richard Kojedzinszky X-X-Sender: krichy@pi.nmdps.net To: zfs-devel@freebsd.org Subject: ZFS/GFS locking fixes Message-ID: User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; format=flowed; charset=US-ASCII X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 29 Dec 2013 22:46:09 -0000 Dear devs, Maybe PJD has forwarded my conversation with him, I've made to fixes for my zfs/gfs locking issues. They can be found here: https://github.com/rkojedzinszky/freebsd/commits/releng/9.2-zfs While this solves most of my discovered issues, one still remained. Commit https://github.com/rkojedzinszky/freebsd/commit/1d8972b3f353f986eb5b85bc108b1c0d946d3218 introduced another deadlock possibility: When 'zfs send -R' tries to exit, it calls zfsdev_close(), which acquires spa_namespace_lock, which then invokes zfs_unmount_snap(), which goes to zfsctl_snapshot_inactive() which will lock the .zfs/snapshot's sdp->sd_lock. The same time, when zfsctl_snapdir_lookup() is running, holding the same directory's sdp->sd_lock, tries to mount a snapshot, which somewhere tries to acquire spa_namespace_lock, and they got into a deadlock. This problem also can be used to DoS a system, as an administrator may have set up to backup its system using zfs send, and a normal user can initiate the other process (the mount). What could be the solution? Thanks in advance, Kojedzinszky Richard From owner-zfs-devel@FreeBSD.ORG Thu Jan 2 10:22:39 2014 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C0BAE668 for ; Thu, 2 Jan 2014 10:22:39 +0000 (UTC) Received: from pi.nmdps.net (pi.nmdps.net [109.61.102.5]) by mx1.freebsd.org (Postfix) with ESMTP id 7C98E18FD for ; Thu, 2 Jan 2014 10:22:38 +0000 (UTC) Received: from pi.nmdps.net (localhost [127.0.0.1]) (Authenticated sender: krichy@cflinux.hu) by pi.nmdps.net (Postfix) with ESMTPSA id 3089B1683; Thu, 2 Jan 2014 11:22:31 +0100 (CET) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Date: Thu, 02 Jan 2014 11:22:28 +0100 From: krichy@cflinux.hu To: Richard Kojedzinszky Subject: Re: ZFS/GFS locking fixes In-Reply-To: References: Message-ID: <69bde436f914a58d0d29550ae08a9e52@cflinux.hu> X-Sender: krichy@cflinux.hu User-Agent: Roundcube Webmail/0.9.5 Cc: zfs-devel@freebsd.org X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Jan 2014 10:22:39 -0000 Dear list, Is there any progress regarding my issues/patches? Thanks, 2013-12-29 23:45 időpontban Richard Kojedzinszky ezt írta: > Dear devs, > > Maybe PJD has forwarded my conversation with him, I've made to fixes > for my zfs/gfs locking issues. They can be found here: > https://github.com/rkojedzinszky/freebsd/commits/releng/9.2-zfs > > While this solves most of my discovered issues, one still remained. > Commit > https://github.com/rkojedzinszky/freebsd/commit/1d8972b3f353f986eb5b85bc108b1c0d946d3218 > introduced another deadlock possibility: > When 'zfs send -R' tries to exit, it calls zfsdev_close(), which > acquires spa_namespace_lock, which then invokes zfs_unmount_snap(), > which goes to > zfsctl_snapshot_inactive() which will lock the .zfs/snapshot's > sdp->sd_lock. The same time, when zfsctl_snapdir_lookup() is running, > holding the same directory's sdp->sd_lock, tries to mount a snapshot, > which somewhere tries to acquire spa_namespace_lock, and they got into > a deadlock. > > This problem also can be used to DoS a system, as an administrator may > have set up to backup its system using zfs send, and a normal user can > initiate the other process (the mount). > > What could be the solution? > > Thanks in advance, > Kojedzinszky Richard From owner-zfs-devel@FreeBSD.ORG Thu Jan 2 14:05:23 2014 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id F32ED34A for ; Thu, 2 Jan 2014 14:05:22 +0000 (UTC) Received: from mail-ie0-f170.google.com (mail-ie0-f170.google.com [209.85.223.170]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id BA32E1D33 for ; Thu, 2 Jan 2014 14:05:22 +0000 (UTC) Received: by mail-ie0-f170.google.com with SMTP id qd12so14917399ieb.29 for ; Thu, 02 Jan 2014 06:05:16 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:date:from:to:cc:message-id:in-reply-to :references:subject:mime-version:content-type; bh=7hA1kgW2HVudoPNZchC4yweLA4oe63AbXTP7pS9jmt4=; b=G8Jmbeg9Ss8ssef63FcZX8jLJwavAOkFAjtANQNdmt9tdzqJlfHh8Yx8k4VMyCaja4 9UiVa1otgXq2ZWUXc8/nWLO3IL0c6z/IHxUy6IlLAHmt4Vb2t+GqV7quh8tAFKls9+Wg Mhm1x118d2kQ/XJVuEKDj9qmR9QryBg0X79GD6RyIaXr1vWLQTkx19EjetsMVerSuEcS QH1cQTseHrAJFkE7nqprrXc1/msswYehx53n3fS8HO69f9SwuS5paEHvQY4FYVw+1tZ+ dhOAwlov+7pnpMMoG3PRP7Z/6YVyRiSEhwZs2ekbPfuoIT9WAjXNhSskMXV9HgEpcIUk Z9Kg== X-Gm-Message-State: ALoCoQmFM2uo+roIJ/Uw/USD7ZbrZ8Hy2I2Pz/zNHAiF0ulQHfrs2dGQs2DYetIos4zF9+cPNbwc X-Received: by 10.43.81.200 with SMTP id zz8mr58359674icb.29.1388671516704; Thu, 02 Jan 2014 06:05:16 -0800 (PST) Received: from scorpius.firepipe.net (207-225-98-3.dia.static.qwest.net. [207.225.98.3]) by mx.google.com with ESMTPSA id l7sm71854571igx.2.2014.01.02.06.05.15 for (version=TLSv1.2 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Thu, 02 Jan 2014 06:05:15 -0800 (PST) Date: Thu, 2 Jan 2014 07:05:14 -0700 From: Will Andrews To: krichy@cflinux.hu Message-ID: In-Reply-To: <69bde436f914a58d0d29550ae08a9e52@cflinux.hu> References: <69bde436f914a58d0d29550ae08a9e52@cflinux.hu> Subject: Re: ZFS/GFS locking fixes X-Mailer: Airmail (223) MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Content-Disposition: inline X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: zfs-devel@freebsd.org X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Jan 2014 14:05:23 -0000 Hi, I=E2=80=99ve been on vacation for the last 2 weeks. =C2=A0I=E2=80=99ll ta= ke a look at it today, work on merging your changes with mine, and then p= ort them forward to =46reeBSD/head. Thanks for the report and your patch. =E2=80=94Will. =46rom:=C2=A0krichy=40cflinux.hu krichy=40cflinux.hu Reply:=C2=A0krichy=40cflinux.hu krichy=40cflinux.hu Date:=C2=A0January 2, 2014 at 3:22:48 AM To:=C2=A0Richard Kojedzinszky krichy=40cflinux.hu Subject:=C2=A0 Re: Z=46S/G=46S locking fixes =20 Dear list, Is there any progress regarding my issues/patches=3F Thanks, 2013-12-29 23:45 id=C5=91pontban Richard Kojedzinszky ezt =C3=ADrta: > Dear devs, > =20 > Maybe PJD has forwarded my conversation with him, I've made to fixes > for my zfs/gfs locking issues. They can be found here: > https://github.com/rkojedzinszky/freebsd/commits/releng/9.2-zfs > =20 > While this solves most of my discovered issues, one still remained. > Commit > https://github.com/rkojedzinszky/freebsd/commit/1d8972b3f353f986eb5b85b= c108b1c0d946d3218 > introduced another deadlock possibility: > When 'zfs send -R' tries to exit, it calls zfsdev=5Fclose(), which > acquires spa=5Fnamespace=5Flock, which then invokes zfs=5Funmount=5Fsna= p(), > which goes to > zfsctl=5Fsnapshot=5Finactive() which will lock the .zfs/snapshot's > sdp->sd=5Flock. The same time, when zfsctl=5Fsnapdir=5Flookup() is runn= ing, > holding the same directory's sdp->sd=5Flock, tries to mount a snapshot,= > which somewhere tries to acquire spa=5Fnamespace=5Flock, and they got i= nto > a deadlock. > =20 > This problem also can be used to DoS a system, as an administrator may > have set up to backup its system using zfs send, and a normal user can > initiate the other process (the mount). > =20 > What could be the solution=3F > =20 > Thanks in advance, > Kojedzinszky Richard =5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F= =5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F zfs-devel=40freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/zfs-devel To unsubscribe, send any mail to =22zfs-devel-unsubscribe=40freebsd.org=22= --=C2=A0 Will Andrews Sent with Airmail From owner-zfs-devel@FreeBSD.ORG Tue Jan 7 09:53:52 2014 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id F34E4831 for ; Tue, 7 Jan 2014 09:53:51 +0000 (UTC) Received: from smtp1.multiplay.co.uk (smtp1.multiplay.co.uk [85.236.96.35]) by mx1.freebsd.org (Postfix) with ESMTP id B773014C7 for ; Tue, 7 Jan 2014 09:53:51 +0000 (UTC) Received: by smtp1.multiplay.co.uk (Postfix, from userid 65534) id 94E9520E7088B; Tue, 7 Jan 2014 09:53:44 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.multiplay.co.uk X-Spam-Level: X-Spam-Status: No, score=-1.4 required=8.0 tests=ALL_TRUSTED,AWL,BAYES_00, STOX_REPLY_TYPE,STOX_REPLY_TYPE_WITHOUT_QUOTES autolearn=no version=3.3.1 Received: from r2d2 (82-69-141-170.dsl.in-addr.zen.co.uk [82.69.141.170]) by smtp1.multiplay.co.uk (Postfix) with ESMTPA id 4D2E320E70886 for ; Tue, 7 Jan 2014 09:53:43 +0000 (UTC) Message-ID: <71C4CE091BDC441C84664005D82EEED4@multiplay.co.uk> From: "Steven Hartland" To: Subject: zvol locking fixes Date: Tue, 7 Jan 2014 09:53:40 -0000 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="iso-8859-1"; reply-type=original Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.5931 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.6157 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 07 Jan 2014 09:53:52 -0000 Hi guys, zvol locking fixes have been mentioned a few times before now, during which the guys at SpectraLogic mentioned they had some in depth changes which fix the problems and where working on getting them into the tree. I've been sitting on a number of PR's for a long time now, for which I have some fixes namely kern/181235 & kern/178999 awaiting these fixes but to my knowledge they have yet to be commited. Could we please get an update on this as if its not going to make it into the tree in the near future I'm going to commit my fixes which at least help with some of the common cases. Regards Steve From owner-zfs-devel@FreeBSD.ORG Sun Jan 12 15:46:05 2014 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id CE91EA29; Sun, 12 Jan 2014 15:46:05 +0000 (UTC) Received: from salmon.maths.tcd.ie (salmon.maths.tcd.ie [IPv6:2001:770:10:300::86e2:510b]) by mx1.freebsd.org (Postfix) with SMTP id 062C41ACE; Sun, 12 Jan 2014 15:46:04 +0000 (UTC) Received: from walton.maths.tcd.ie ([IPv6:2001:770:10:300::86e2:510a] helo=walton.maths.tcd.ie) by salmon.maths.tcd.ie with SMTP id ; 12 Jan 2014 15:46:02 +0000 (GMT) Received: from walton.maths.tcd.ie (localhost [IPv6:::1]) by walton.maths.tcd.ie (Postfix) with ESMTP id 9551573044; Sun, 12 Jan 2014 15:46:02 +0000 (GMT) To: zfs-devel@freebsd.org Subject: zfs panic on i386 From: David Malone Date: Sun, 12 Jan 2014 15:46:02 +0000 Message-Id: <20140112154602.9551573044@walton.maths.tcd.ie> X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 12 Jan 2014 15:46:05 -0000 I have an i386 machine running -current booting from a zfs root. I tried to update it in December, and it panics while trying to mount the zfs filesystem on an assertion: solaris assert: idx < sizeof(rt->rt_histogram)/sizeof(*rt->rt_histogram) (0xffffffffffffffff < 0x40) This is in range_tree_stat_incr(). I did some digging, and it seems that a large object is being added to the range tree, range_tree_stat_incr() does this: uint64_t size = rs->rs_end - rs->rs_start; int idx = highbit(size) - 1; But the highbit code on i386 only deals with unsigned longs on i386, and so returns zero, leaving idx set to -1. If I fix the highbit code to work for uint64_t (see below), then the machine can boot successfully. David. @@ -380,17 +380,18 @@ * High order bit is 31 (or 63 in _LP64 kernel). */ static __inline int -highbit(ulong_t i) +/*highbit(ulong_t i)*/ +highbit(uint64_t i) { register int h = 1; if (i == 0) return (0); -#ifdef _LP64 +/*#ifdef _LP64 */ if (i & 0xffffffff00000000ul) { h += 32; i >>= 32; } -#endif +/*#endif*/ if (i & 0xffff0000) { h += 16; i >>= 16; } From owner-zfs-devel@FreeBSD.ORG Mon Jan 13 07:03:24 2014 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B162BA6E for ; Mon, 13 Jan 2014 07:03:24 +0000 (UTC) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 0A56019F5 for ; Mon, 13 Jan 2014 07:03:23 +0000 (UTC) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id JAA21011; Mon, 13 Jan 2014 09:03:12 +0200 (EET) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1W2bYF-0005uy-P5; Mon, 13 Jan 2014 09:03:11 +0200 Message-ID: <52D38F77.6080809@FreeBSD.org> Date: Mon, 13 Jan 2014 09:02:15 +0200 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.2.0 MIME-Version: 1.0 To: David Malone , zfs-devel@FreeBSD.org Subject: Re: zfs panic on i386 References: <20140112154602.9551573044@walton.maths.tcd.ie> In-Reply-To: <20140112154602.9551573044@walton.maths.tcd.ie> X-Enigmail-Version: 1.6 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 13 Jan 2014 07:03:24 -0000 on 12/01/2014 17:46 David Malone said the following: > I have an i386 machine running -current booting from a zfs root. I > tried to update it in December, and it panics while trying to mount > the zfs filesystem on an assertion: > > solaris assert: idx < sizeof(rt->rt_histogram)/sizeof(*rt->rt_histogram) (0xffffffffffffffff < 0x40) > > This is in range_tree_stat_incr(). I did some digging, and it seems > that a large object is being added to the range tree, > range_tree_stat_incr() does this: > > uint64_t size = rs->rs_end - rs->rs_start; > int idx = highbit(size) - 1; > > But the highbit code on i386 only deals with unsigned longs on i386, > and so returns zero, leaving idx set to -1. If I fix the highbit > code to work for uint64_t (see below), then the machine can boot > successfully. David, thank you very much for the analysis and the patch. I discussed the issue with the upstream and they are aware of the problem. We are probably going to end up with a slightly different fix with explicit highbit64() function. Thanks again! > @@ -380,17 +380,18 @@ > * High order bit is 31 (or 63 in _LP64 kernel). > */ > static __inline int > -highbit(ulong_t i) > +/*highbit(ulong_t i)*/ > +highbit(uint64_t i) > { > register int h = 1; > > if (i == 0) > return (0); > -#ifdef _LP64 > +/*#ifdef _LP64 */ > if (i & 0xffffffff00000000ul) { > h += 32; i >>= 32; > } > -#endif > +/*#endif*/ > if (i & 0xffff0000) { > h += 16; i >>= 16; > } > -- Andriy Gapon From owner-zfs-devel@FreeBSD.ORG Mon Jan 13 09:43:18 2014 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 31AFEBEE; Mon, 13 Jan 2014 09:43:18 +0000 (UTC) Received: from salmon.maths.tcd.ie (salmon.maths.tcd.ie [IPv6:2001:770:10:300::86e2:510b]) by mx1.freebsd.org (Postfix) with SMTP id 9F50E1626; Mon, 13 Jan 2014 09:43:17 +0000 (UTC) Received: from walton.maths.tcd.ie ([IPv6:2001:770:10:300::86e2:510a] helo=walton.maths.tcd.ie) by salmon.maths.tcd.ie with SMTP id ; 13 Jan 2014 09:43:16 +0000 (GMT) Received: from walton.maths.tcd.ie (localhost [IPv6:::1]) by walton.maths.tcd.ie (Postfix) with ESMTP id 6516573044; Mon, 13 Jan 2014 09:43:16 +0000 (GMT) To: Andriy Gapon Subject: Re: zfs panic on i386 From: David Malone In-reply-to: <52D38F77.6080809@FreeBSD.org> Date: Mon, 13 Jan 2014 09:43:16 +0000 Message-Id: <20140113094316.6516573044@walton.maths.tcd.ie> Cc: zfs-devel@FreeBSD.org X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 13 Jan 2014 09:43:18 -0000 > thank you very much for the analysis and the patch. > I discussed the issue with the upstream and they are aware of the problem. > We are probably going to end up with a slightly different fix with explicit > highbit64() function. Great - thanks. I'll keep an eye out for the fix. David. From owner-zfs-devel@FreeBSD.ORG Tue Jan 14 11:47:15 2014 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A8D30E96; Tue, 14 Jan 2014 11:47:15 +0000 (UTC) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id A3BBB1CE7; Tue, 14 Jan 2014 11:47:14 +0000 (UTC) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id NAA18720; Tue, 14 Jan 2014 13:47:12 +0200 (EET) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1W32Se-000Ai8-41; Tue, 14 Jan 2014 13:47:12 +0200 Message-ID: <52D5239B.6000203@FreeBSD.org> Date: Tue, 14 Jan 2014 13:46:35 +0200 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.2.0 MIME-Version: 1.0 To: krichy@tvnetwork.hu Subject: Re: kern/184677 / ZFS snapshot handling deadlocks References: <20131220134405.GE1658@garage.freebsd.pl> In-Reply-To: X-Enigmail-Version: 1.6 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Mailman-Approved-At: Tue, 14 Jan 2014 13:08:48 +0000 Cc: freebsd-fs@FreeBSD.org X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 14 Jan 2014 11:47:15 -0000 on 25/12/2013 07:22 krichy@tvnetwork.hu said the following: > I've made a new patch again, which fixes most of my earlier issues. Mainly, I've > enabled shared vnode locks for GFS vnodes - is it acceptable? And that way, > deadlock cases reduced much, and also the patch is more clear (at least to me). > One thing still remains, the spa_namespace_lock race I mentioned before, I dont > know how to handle that. > > Waiting for comments on this. Richard, first of all, apologies for the delay with a reply and for not having any comment on the essence of your patch... I do have the following meta-comment. - working with FreeBSD VFS is hard - porting code that was written for a very different VFS model to FreeBSD VFS is even harder - doing the same for the code that plays various tricks, like .zfs support code does, is harder again - reviewing somebody else's changes to that kind of code is harder still than making such changes This is quite an unfortunate situation. I am not much surprised by the lack of followups to your analysis and patches. I am saying this as someone who spent some time analyzing and trying to fix the .zfs code and ZFS<->VFS interaction in general. See e.g. http://thread.gmane.org/gmane.os.freebsd.devel.file-systems/18924/focus=19056 My opinion is that .zfs code breaks several fundamental FreeBSD VFS contracts. The most glaring violation is that VOP_INACTIVE makes a vnode invalid. I think that trying to fix .zfs code by patching individual problems here and there is an uphill battle. I think that the same also applies to ZFS ZPL code but in a less obvious way. The code in many cases just pretends to play by VFS rules by satisfying most obvious assertions, but it does not really try to obey VFS contracts. For example, almost all locks in znode are mostly redundant given VFS vnode locking. But in some cases they are not sufficient precisely because VFS expects its locking to be used rather than ZFS internal locking. The most obvious example is interaction of VOP_RENAME with other VOPs. In any case, thanks for your work! I am trying to find some time to review it. -- Andriy Gapon From owner-zfs-devel@FreeBSD.ORG Mon Jan 20 19:21:28 2014 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 75D9E571 for ; Mon, 20 Jan 2014 19:21:28 +0000 (UTC) Received: from pi.nmdps.net (pi.nmdps.net [109.61.102.5]) by mx1.freebsd.org (Postfix) with ESMTP id D23F311B7 for ; Mon, 20 Jan 2014 19:21:27 +0000 (UTC) Received: from pi.nmdps.net (localhost [127.0.0.1]) (Authenticated sender: krichy@cflinux.hu) by pi.nmdps.net (Postfix) with ESMTPSA id 6313E123C for ; Mon, 20 Jan 2014 20:21:26 +0100 (CET) MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=_6d066b626d245eaf36608aa45d442590" Date: Mon, 20 Jan 2014 20:21:24 +0100 From: krichy@cflinux.hu To: zfs-devel@freebsd.org Subject: Fwd: Re: ZFS .zfs DoS Message-ID: X-Sender: krichy@cflinux.hu User-Agent: Roundcube Webmail/0.9.5 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 20 Jan 2014 19:21:28 -0000 --=_6d066b626d245eaf36608aa45d442590 Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset=UTF-8; format=flowed -------- Eredeti üzenet -------- Tárgy: Re: ZFS .zfs DoS Dátum: 2014-01-20 16:30 Feladó: krichy@cflinux.hu Címzett: Richard Kojedzinszky Másolat: freebsd-fs@freebsd.org, freebsd-security@freebsd.org Dear users, I've worked out a patch for my known issues, please somebody test them, and give recommendations, fixes. Regards, 2014-01-17 03:11 időpontban Richard Kojedzinszky ezt írta: > Dear users, > > For a long time now I've been investigating problems relating FreeBSD > ZFS .zfs handling, and found that I am not enough to fix issues. Until > fixes arrive, unfortunately a regular user can DoS a FreeBSD system > which has ZFS filesystems with the attached script. While the script > expects a snapshot argument to be given, actually the first test case > does not need that, only a mounted zfs filesystem is enough. For more > of the tests a snapshot may be needed, and later ones need root > account also. > > I would recommend that until this gets rewritten or fixed at all, one > should disable access to .zfs at all with someting like I've attached. > > Regards, > Kojedzinszky Richard --=_6d066b626d245eaf36608aa45d442590 Content-Transfer-Encoding: base64 Content-Type: text/x-diff; name=gfs-4.patch Content-Disposition: attachment; filename=gfs-4.patch; size=11842 Y29tbWl0IGY1NmQ2NTk2Yjc5YzliYTc2ODUxZWU2YmVhMjI1ZjIyY2M5ZjBhMjYKQXV0aG9yOiBS aWNoYXJkIEtvamVkemluc3preSA8a3JpY2h5QGNmbGludXguaHU+CkRhdGU6ICAgRnJpIEphbiAx NyAyMjo1NzozMyAyMDE0ICswMTAwCgogICAgWkZTL0dGUyBoYW5kbGluZyBmaXhlcwoKZGlmZiAt LWdpdCBhL3N5cy9jZGRsL2NvbXBhdC9vcGVuc29sYXJpcy9rZXJuL29wZW5zb2xhcmlzX2xvb2t1 cC5jIGIvc3lzL2NkZGwvY29tcGF0L29wZW5zb2xhcmlzL2tlcm4vb3BlbnNvbGFyaXNfbG9va3Vw LmMKaW5kZXggOTQzODNkNi4uNGNhYzA1MyAxMDA2NDQKLS0tIGEvc3lzL2NkZGwvY29tcGF0L29w ZW5zb2xhcmlzL2tlcm4vb3BlbnNvbGFyaXNfbG9va3VwLmMKKysrIGIvc3lzL2NkZGwvY29tcGF0 L29wZW5zb2xhcmlzL2tlcm4vb3BlbnNvbGFyaXNfbG9va3VwLmMKQEAgLTgxLDYgKzgxLDggQEAg dHJhdmVyc2Uodm5vZGVfdCAqKmN2cHAsIGludCBsa3R5cGUpCiAJICogcHJvZ3Jlc3Mgb24gdGhp cyB2bm9kZS4KIAkgKi8KIAorCXZuX2xvY2soY3ZwLCBsa3R5cGUpOworCiAJZm9yICg7Oykgewog CQkvKgogCQkgKiBSZWFjaGVkIHRoZSBlbmQgb2YgdGhlIG1vdW50IGNoYWluPwpAQCAtODksMTMg KzkxLDcgQEAgdHJhdmVyc2Uodm5vZGVfdCAqKmN2cHAsIGludCBsa3R5cGUpCiAJCWlmICh2ZnNw ID09IE5VTEwpCiAJCQlicmVhazsKIAkJZXJyb3IgPSB2ZnNfYnVzeSh2ZnNwLCAwKTsKLQkJLyoK LQkJICogdHZwIGlzIE5VTEwgZm9yICpjdnBwIHZub2RlLCB3aGljaCB3ZSBjYW4ndCB1bmxvY2su Ci0JCSAqLwotCQlpZiAodHZwICE9IE5VTEwpCi0JCQl2cHV0KGN2cCk7Ci0JCWVsc2UKLQkJCXZy ZWxlKGN2cCk7CisJCVZPUF9VTkxPQ0soY3ZwLCAwKTsKIAkJaWYgKGVycm9yKQogCQkJcmV0dXJu IChlcnJvcik7CiAKQEAgLTEwNyw2ICsxMDMsOSBAQCB0cmF2ZXJzZSh2bm9kZV90ICoqY3ZwcCwg aW50IGxrdHlwZSkKIAkJdmZzX3VuYnVzeSh2ZnNwKTsKIAkJaWYgKGVycm9yICE9IDApCiAJCQly ZXR1cm4gKGVycm9yKTsKKworCQlWTl9SRUxFKGN2cCk7CisKIAkJY3ZwID0gdHZwOwogCX0KIApk aWZmIC0tZ2l0IGEvc3lzL2NkZGwvY29tcGF0L29wZW5zb2xhcmlzL2tlcm4vb3BlbnNvbGFyaXNf dmZzLmMgYi9zeXMvY2RkbC9jb21wYXQvb3BlbnNvbGFyaXMva2Vybi9vcGVuc29sYXJpc192ZnMu YwppbmRleCBhMjUzMmY4Li5jMzAyYTU0IDEwMDY0NAotLS0gYS9zeXMvY2RkbC9jb21wYXQvb3Bl bnNvbGFyaXMva2Vybi9vcGVuc29sYXJpc192ZnMuYworKysgYi9zeXMvY2RkbC9jb21wYXQvb3Bl bnNvbGFyaXMva2Vybi9vcGVuc29sYXJpc192ZnMuYwpAQCAtMTk0LDEwICsxOTQsOCBAQCBtb3Vu dF9zbmFwc2hvdChrdGhyZWFkX3QgKnRkLCB2bm9kZV90ICoqdnBwLCBjb25zdCBjaGFyICpmc3R5 cGUsIGNoYXIgKmZzcGF0aCwKIAkJVklfTE9DSyh2cCk7CiAJCXZwLT52X2lmbGFnICY9IH5WSV9N T1VOVDsKIAkJVklfVU5MT0NLKHZwKTsKLQkJdnJlbGUodnApOwogCQl2ZnNfdW5idXN5KG1wKTsK IAkJdmZzX21vdW50X2Rlc3Ryb3kobXApOwotCQkqdnBwID0gTlVMTDsKIAkJcmV0dXJuIChlcnJv cik7CiAJfQogCkBAIC0yMjgsNyArMjI2LDcgQEAgbW91bnRfc25hcHNob3Qoa3RocmVhZF90ICp0 ZCwgdm5vZGVfdCAqKnZwcCwgY29uc3QgY2hhciAqZnN0eXBlLCBjaGFyICpmc3BhdGgsCiAJdmZz X2V2ZW50X3NpZ25hbChOVUxMLCBWUV9NT1VOVCwgMCk7CiAJaWYgKFZGU19ST09UKG1wLCBMS19F WENMVVNJVkUsICZtdnApKQogCQlwYW5pYygibW91bnQ6IGxvc3QgbW91bnQiKTsKLQl2cHV0KHZw KTsKKwlWT1BfVU5MT0NLKHZwLCAwKTsKIAl2ZnNfdW5idXN5KG1wKTsKIAkqdnBwID0gbXZwOwog CXJldHVybiAoMCk7CmRpZmYgLS1naXQgYS9zeXMvY2RkbC9jb250cmliL29wZW5zb2xhcmlzL3V0 cy9jb21tb24vZnMvZ2ZzLmMgYi9zeXMvY2RkbC9jb250cmliL29wZW5zb2xhcmlzL3V0cy9jb21t b24vZnMvZ2ZzLmMKaW5kZXggNTk5NDRhMS4uMjllYzQ1NCAxMDA2NDQKLS0tIGEvc3lzL2NkZGwv Y29udHJpYi9vcGVuc29sYXJpcy91dHMvY29tbW9uL2ZzL2dmcy5jCisrKyBiL3N5cy9jZGRsL2Nv bnRyaWIvb3BlbnNvbGFyaXMvdXRzL2NvbW1vbi9mcy9nZnMuYwpAQCAtNDQ4LDcgKzQ0OCw2IEBA IGdmc19sb29rdXBfZG90KHZub2RlX3QgKip2cHAsIHZub2RlX3QgKmR2cCwgdm5vZGVfdCAqcHZw LCBjb25zdCBjaGFyICpubSkKIAkJCVZOX0hPTEQocHZwKTsKIAkJCSp2cHAgPSBwdnA7CiAJCX0K LQkJdm5fbG9jaygqdnBwLCBMS19FWENMVVNJVkUgfCBMS19SRVRSWSk7CiAJCXJldHVybiAoMCk7 CiAJfQogCkBAIC00ODUsNiArNDg0LDcgQEAgZ2ZzX2ZpbGVfY3JlYXRlKHNpemVfdCBzaXplLCB2 bm9kZV90ICpwdnAsIHZmc190ICp2ZnNwLCB2bm9kZW9wc190ICpvcHMpCiAJZnAgPSBrbWVtX3ph bGxvYyhzaXplLCBLTV9TTEVFUCk7CiAJZXJyb3IgPSBnZXRuZXd2bm9kZSgiemZzIiwgdmZzcCwg b3BzLCAmdnApOwogCUFTU0VSVChlcnJvciA9PSAwKTsKKwlWTl9MT0NLX0FTSEFSRSh2cCk7CiAJ dm5fbG9jayh2cCwgTEtfRVhDTFVTSVZFIHwgTEtfUkVUUlkpOwogCXZwLT52X2RhdGEgPSAoY2Fk ZHJfdClmcDsKIApAQCAtNDk2LDkgKzQ5Niw5IEBAIGdmc19maWxlX2NyZWF0ZShzaXplX3Qgc2l6 ZSwgdm5vZGVfdCAqcHZwLCB2ZnNfdCAqdmZzcCwgdm5vZGVvcHNfdCAqb3BzKQogCWZwLT5nZnNf c2l6ZSA9IHNpemU7CiAJZnAtPmdmc190eXBlID0gR0ZTX0ZJTEU7CiAKLQl2cC0+dl92ZmxhZyB8 PSBWVl9GT1JDRUlOU01ROworCXZwLT52X3ZmbGFnIHw9IFZWX0ZPUkNFSU5TTVEgfCBWVl9JTlNN UUhFQUQ7CiAJZXJyb3IgPSBpbnNtbnRxdWUodnAsIHZmc3ApOwotCXZwLT52X3ZmbGFnICY9IH5W Vl9GT1JDRUlOU01ROworCXZwLT52X3ZmbGFnICY9IH4oVlZfRk9SQ0VJTlNNUSB8IFZWX0lOU01R SEVBRCk7CiAJS0FTU0VSVChlcnJvciA9PSAwLCAoImluc21udHF1ZSgpIGZhaWxlZDogZXJyb3Ig JWQiLCBlcnJvcikpOwogCiAJLyoKQEAgLTYzNywxMiArNjM3LDcgQEAgZ2ZzX2ZpbGVfaW5hY3Rp dmUodm5vZGVfdCAqdnApCiAJaWYgKGZwLT5nZnNfcGFyZW50ID09IE5VTEwgfHwgKHZwLT52X2Zs YWcgJiBWX1hBVFRSRElSKSkKIAkJZ290byBmb3VuZDsKIAotCS8qCi0JICogWFhYIGNvcGUgd2l0 aCBhIEZyZWVCU0Qtc3BlY2lmaWMgcmFjZSB3aGVyZWluIHRoZSBwYXJlbnQncwotCSAqIHNuYXBz aG90IGRhdGEgY2FuIGJlIGZyZWVkIGJlZm9yZSB0aGUgcGFyZW50IGlzCi0JICovCi0JaWYgKChk cCA9IGZwLT5nZnNfcGFyZW50LT52X2RhdGEpID09IE5VTEwpCi0JCXJldHVybiAoTlVMTCk7CisJ ZHAgPSBmcC0+Z2ZzX3BhcmVudC0+dl9kYXRhOwogCiAJLyoKIAkgKiBGaXJzdCwgc2VlIGlmIHRo aXMgdm5vZGUgaXMgY2FjaGVkIGluIHRoZSBwYXJlbnQuCkBAIC02NjksMzcgKzY2NCw0NCBAQCBm b3VuZDoKIAlpZiAodnAtPnZfZmxhZyAmIFZfWEFUVFJESVIpCiAJCVZJX0xPQ0soZnAtPmdmc19w YXJlbnQpOwogI2VuZGlmCi0JVklfTE9DSyh2cCk7Ci0JLyoKLQkgKiBSZWFsbHkgcmVtb3ZlIHRo aXMgdm5vZGUKLQkgKi8KLQlkYXRhID0gdnAtPnZfZGF0YTsKLQlpZiAoZ2UgIT0gTlVMTCkgewor CWlmICh2cC0+dl9jb3VudCA9PSAwIHx8IHZwLT52X2lmbGFnICYgVklfRE9PTUVEKSB7CiAJCS8q Ci0JCSAqIElmIHRoaXMgd2FzIGEgc3RhdGljYWxseSBjYWNoZWQgZW50cnksIHNpbXBseSBzZXQg dGhlCi0JCSAqIGNhY2hlZCB2bm9kZSB0byBOVUxMLgorCQkgKiBSZWFsbHkgcmVtb3ZlIHRoaXMg dm5vZGUKIAkJICovCi0JCWdlLT5nZnNlX3Zub2RlID0gTlVMTDsKLQl9Ci0JVklfVU5MT0NLKHZw KTsKKwkJZGF0YSA9IHZwLT52X2RhdGE7CisJCWlmIChnZSAhPSBOVUxMKSB7CisJCQkvKgorCQkJ ICogSWYgdGhpcyB3YXMgYSBzdGF0aWNhbGx5IGNhY2hlZCBlbnRyeSwgc2ltcGx5IHNldCB0aGUK KwkJCSAqIGNhY2hlZCB2bm9kZSB0byBOVUxMLgorCQkJICovCisJCQlnZS0+Z2ZzZV92bm9kZSA9 IE5VTEw7CisJCX0KKyNpZmRlZiBUT0RPCisJCWlmICh2cC0+dl9mbGFnICYgVl9YQVRUUkRJUikK KwkJCVZJX1VOTE9DSyhmcC0+Z2ZzX3BhcmVudCk7CisjZW5kaWYKIAotCS8qCi0JICogRnJlZSB2 bm9kZSBhbmQgcmVsZWFzZSBwYXJlbnQKLQkgKi8KLQlpZiAoZnAtPmdmc19wYXJlbnQpIHsKLQkJ aWYgKGRwKQotCQkJZ2ZzX2Rpcl91bmxvY2soZHApOwotCQlWT1BfVU5MT0NLKHZwLCAwKTsKLQkJ Vk5fUkVMRShmcC0+Z2ZzX3BhcmVudCk7Ci0JCXZuX2xvY2sodnAsIExLX0VYQ0xVU0lWRSB8IExL X1JFVFJZKTsKKwkJLyoKKwkJICogRnJlZSB2bm9kZSBhbmQgcmVsZWFzZSBwYXJlbnQKKwkJICov CisJCWlmIChmcC0+Z2ZzX3BhcmVudCkgeworCQkJaWYgKGRwKQorCQkJCWdmc19kaXJfdW5sb2Nr KGRwKTsKKwkJCVZOX1JFTEUoZnAtPmdmc19wYXJlbnQpOworCQl9IGVsc2UgeworCQkJQVNTRVJU KHZwLT52X3Zmc3AgIT0gTlVMTCk7CisJCQlWRlNfUkVMRSh2cC0+dl92ZnNwKTsKKwkJfQogCX0g ZWxzZSB7Ci0JCUFTU0VSVCh2cC0+dl92ZnNwICE9IE5VTEwpOwotCQlWRlNfUkVMRSh2cC0+dl92 ZnNwKTsKLQl9CisJCWRhdGEgPSBOVUxMOwogI2lmZGVmIFRPRE8KLQlpZiAodnAtPnZfZmxhZyAm IFZfWEFUVFJESVIpCi0JCVZJX1VOTE9DSyhmcC0+Z2ZzX3BhcmVudCk7CisJCWlmICh2cC0+dl9m bGFnICYgVl9YQVRUUkRJUikKKwkJCVZJX1VOTE9DSyhmcC0+Z2ZzX3BhcmVudCk7CiAjZW5kaWYK KwkJaWYgKGRwKQorCQkJZ2ZzX2Rpcl91bmxvY2soZHApOworCX0KKwogCXJldHVybiAoZGF0YSk7 CiB9CiAKQEAgLTEyMzAsMTYgKzEyMzIsMTUgQEAgZ2ZzX3ZvcF9pbmFjdGl2ZShhcCkKIHsKIAl2 bm9kZV90ICp2cCA9IGFwLT5hX3ZwOwogCWdmc19maWxlX3QgKmZwID0gdnAtPnZfZGF0YTsKKwl2 b2lkICpkYXRhOwogCiAJaWYgKGZwLT5nZnNfdHlwZSA9PSBHRlNfRElSKQotCQlnZnNfZGlyX2lu YWN0aXZlKHZwKTsKKwkJZGF0YSA9IGdmc19kaXJfaW5hY3RpdmUodnApOwogCWVsc2UKLQkJZ2Zz X2ZpbGVfaW5hY3RpdmUodnApOworCQlkYXRhID0gZ2ZzX2ZpbGVfaW5hY3RpdmUodnApOwogCi0J VklfTE9DSyh2cCk7Ci0JdnAtPnZfZGF0YSA9IE5VTEw7Ci0JVklfVU5MT0NLKHZwKTsKLQlrbWVt X2ZyZWUoZnAsIGZwLT5nZnNfc2l6ZSk7CisJaWYgKGRhdGEgIT0gTlVMTCkKKwkJa21lbV9mcmVl KGRhdGEsIGZwLT5nZnNfc2l6ZSk7CiAKIAlyZXR1cm4gKDApOwogfQpkaWZmIC0tZ2l0IGEvc3lz L2NkZGwvY29udHJpYi9vcGVuc29sYXJpcy91dHMvY29tbW9uL2ZzL3pmcy96ZnNfY3RsZGlyLmMg Yi9zeXMvY2RkbC9jb250cmliL29wZW5zb2xhcmlzL3V0cy9jb21tb24vZnMvemZzL3pmc19jdGxk aXIuYwppbmRleCAyOGFiMWZhLi4xNWE1NWQyIDEwMDY0NAotLS0gYS9zeXMvY2RkbC9jb250cmli L29wZW5zb2xhcmlzL3V0cy9jb21tb24vZnMvemZzL3pmc19jdGxkaXIuYworKysgYi9zeXMvY2Rk bC9jb250cmliL29wZW5zb2xhcmlzL3V0cy9jb21tb24vZnMvemZzL3pmc19jdGxkaXIuYwpAQCAt NjEyLDcgKzYxMiw3IEBAIHpmc2N0bF9mcmVlYnNkX3Jvb3RfbG9va3VwKGFwKQogCiAJZXJyID0g emZzY3RsX3Jvb3RfbG9va3VwKGR2cCwgbm0sIHZwcCwgTlVMTCwgMCwgTlVMTCwgY3IsIE5VTEws IE5VTEwsIE5VTEwpOwogCWlmIChlcnIgPT0gMCAmJiAobm1bMF0gIT0gJy4nIHx8IG5tWzFdICE9 ICdcMCcpKQotCQl2bl9sb2NrKCp2cHAsIExLX0VYQ0xVU0lWRSB8IExLX1JFVFJZKTsKKwkJZXJy ID0gdm5fbG9jaygqdnBwLCBhcC0+YV9jbnAtPmNuX2xrZmxhZ3MpOwogCXJldHVybiAoZXJyKTsK IH0KIApAQCAtOTc1LDggKzk3NSwxMSBAQCB6ZnNjdGxfc25hcGRpcl9sb29rdXAoYXApCiAJWkZT X0VOVEVSKHpmc3Zmcyk7CiAKIAlpZiAoZ2ZzX2xvb2t1cF9kb3QodnBwLCBkdnAsIHpmc3Zmcy0+ el9jdGxkaXIsIG5tKSA9PSAwKSB7CisJCWVyciA9IDA7CisJCWlmIChubVswXSAhPSAnLicgfHwg bm1bMV0gIT0gJ1wwJykKKwkJCWVyciA9IHZuX2xvY2soKnZwcCwgYXAtPmFfY25wLT5jbl9sa2Zs YWdzKTsKIAkJWkZTX0VYSVQoemZzdmZzKTsKLQkJcmV0dXJuICgwKTsKKwkJcmV0dXJuIChlcnIp OwogCX0KIAogCWlmIChmbGFncyAmIEZJR05PUkVDQVNFKSB7CkBAIC0xMDA0LDcgKzEwMDcsNyBA QCB6ZnNjdGxfc25hcGRpcl9sb29rdXAoYXApCiAJaWYgKChzZXAgPSBhdmxfZmluZCgmc2RwLT5z ZF9zbmFwcywgJnNlYXJjaCwgJndoZXJlKSkgIT0gTlVMTCkgewogCQkqdnBwID0gc2VwLT5zZV9y b290OwogCQlWTl9IT0xEKCp2cHApOwotCQllcnIgPSB0cmF2ZXJzZSh2cHAsIExLX0VYQ0xVU0lW RSB8IExLX1JFVFJZKTsKKwkJZXJyID0gdHJhdmVyc2UodnBwLCBhcC0+YV9jbnAtPmNuX2xrZmxh Z3MpOwogCQlpZiAoZXJyICE9IDApIHsKIAkJCVZOX1JFTEUoKnZwcCk7CiAJCQkqdnBwID0gTlVM TDsKQEAgLTEwMTMsNiArMTAxNiw4IEBAIHpmc2N0bF9zbmFwZGlyX2xvb2t1cChhcCkKIAkJCSAq IFRoZSBzbmFwc2hvdCB3YXMgdW5tb3VudGVkIGJlaGluZCBvdXIgYmFja3MsCiAJCQkgKiB0cnkg dG8gcmVtb3VudCBpdC4KIAkJCSAqLworCQkJVk5fSE9MRCgqdnBwKTsKKwkJCVZPUF9VTkxPQ0so KnZwcCwgMCk7CiAJCQlWRVJJRlkoemZzY3RsX3NuYXBzaG90X3puYW1lKGR2cCwgbm0sIE1BWE5B TUVMRU4sIHNuYXBuYW1lKSA9PSAwKTsKIAkJCWdvdG8gZG9tb3VudDsKIAkJfSBlbHNlIHsKQEAg LTEwNjQsNyArMTA2OSw2IEBAIHpmc2N0bF9zbmFwZGlyX2xvb2t1cChhcCkKIAlzZXAtPnNlX25h bWUgPSBrbWVtX2FsbG9jKHN0cmxlbihubSkgKyAxLCBLTV9TTEVFUCk7CiAJKHZvaWQpIHN0cmNw eShzZXAtPnNlX25hbWUsIG5tKTsKIAkqdnBwID0gc2VwLT5zZV9yb290ID0gemZzY3RsX3NuYXBz aG90X21rbm9kZShkdnAsIGRtdV9vYmpzZXRfaWQoc25hcCkpOwotCVZOX0hPTEQoKnZwcCk7CiAJ YXZsX2luc2VydCgmc2RwLT5zZF9zbmFwcywgc2VwLCB3aGVyZSk7CiAKIAlkbXVfb2Jqc2V0X3Jl bGUoc25hcCwgRlRBRyk7CkBAIC0xMDkxLDcgKzEwOTUsNiBAQCBkb21vdW50OgogCW11dGV4X2V4 aXQoJnNkcC0+c2RfbG9jayk7CiAJWkZTX0VYSVQoemZzdmZzKTsKIAotI2lmZGVmIGlsbHVtb3MK IAkvKgogCSAqIElmIHdlIGhhZCBhbiBlcnJvciwgZHJvcCBvdXIgaG9sZCBvbiB0aGUgdm5vZGUg YW5kCiAJICogemZzY3RsX3NuYXBzaG90X2luYWN0aXZlKCkgd2lsbCBjbGVhbiB1cC4KQEAgLTEx MDAsMTAgKzExMDMsNiBAQCBkb21vdW50OgogCQlWTl9SRUxFKCp2cHApOwogCQkqdnBwID0gTlVM TDsKIAl9Ci0jZWxzZQotCWlmIChlcnIgIT0gMCkKLQkJKnZwcCA9IE5VTEw7Ci0jZW5kaWYKIAly ZXR1cm4gKGVycik7CiB9CiAKQEAgLTExMzAsOCArMTEyOSwxMSBAQCB6ZnNjdGxfc2hhcmVzX2xv b2t1cChhcCkKIAlzdHJsY3B5KG5tLCBjbnAtPmNuX25hbWVwdHIsIGNucC0+Y25fbmFtZWxlbiAr IDEpOwogCiAJaWYgKGdmc19sb29rdXBfZG90KHZwcCwgZHZwLCB6ZnN2ZnMtPnpfY3RsZGlyLCBu bSkgPT0gMCkgeworCQllcnJvciA9IDA7CisJCWlmIChubVswXSAhPSAnLicgfHwgbm1bMV0gIT0g J1wwJykKKwkJCWVycm9yID0gdm5fbG9jaygqdnBwLCBhcC0+YV9jbnAtPmNuX2xrZmxhZ3MpOwog CQlaRlNfRVhJVCh6ZnN2ZnMpOwotCQlyZXR1cm4gKDApOworCQlyZXR1cm4gKGVycm9yKTsKIAl9 CiAKIAlpZiAoemZzdmZzLT56X3NoYXJlc19kaXIgPT0gMCkgewpAQCAtMTM0NCwyMiArMTM0Niwx NSBAQCB6ZnNjdGxfc25hcGRpcl9pbmFjdGl2ZShhcCkKIAl2bm9kZV90ICp2cCA9IGFwLT5hX3Zw OwogCXpmc2N0bF9zbmFwZGlyX3QgKnNkcCA9IHZwLT52X2RhdGE7CiAJemZzX3NuYXBlbnRyeV90 ICpzZXA7Ci0KLQkvKgotCSAqIE9uIGZvcmNlZCB1bm1vdW50IHdlIGhhdmUgdG8gZnJlZSBzbmFw c2hvdHMgZnJvbSBoZXJlLgotCSAqLwotCW11dGV4X2VudGVyKCZzZHAtPnNkX2xvY2spOwotCXdo aWxlICgoc2VwID0gYXZsX2ZpcnN0KCZzZHAtPnNkX3NuYXBzKSkgIT0gTlVMTCkgewotCQlhdmxf cmVtb3ZlKCZzZHAtPnNkX3NuYXBzLCBzZXApOwotCQlrbWVtX2ZyZWUoc2VwLT5zZV9uYW1lLCBz dHJsZW4oc2VwLT5zZV9uYW1lKSArIDEpOwotCQlrbWVtX2ZyZWUoc2VwLCBzaXplb2YgKHpmc19z bmFwZW50cnlfdCkpOworCXZvaWQgKnByaXZhdGU7CisKKwlwcml2YXRlID0gZ2ZzX2Rpcl9pbmFj dGl2ZSh2cCk7CisJaWYgKHByaXZhdGUgIT0gTlVMTCkgeworCQlBU1NFUlQoYXZsX251bW5vZGVz KCZzZHAtPnNkX3NuYXBzKSA9PSAwKTsKKwkJbXV0ZXhfZGVzdHJveSgmc2RwLT5zZF9sb2NrKTsK KwkJYXZsX2Rlc3Ryb3koJnNkcC0+c2Rfc25hcHMpOworCQlrbWVtX2ZyZWUocHJpdmF0ZSwgc2l6 ZW9mICh6ZnNjdGxfc25hcGRpcl90KSk7CiAJfQotCW11dGV4X2V4aXQoJnNkcC0+c2RfbG9jayk7 Ci0JZ2ZzX2Rpcl9pbmFjdGl2ZSh2cCk7Ci0JQVNTRVJUKGF2bF9udW1ub2Rlcygmc2RwLT5zZF9z bmFwcykgPT0gMCk7Ci0JbXV0ZXhfZGVzdHJveSgmc2RwLT5zZF9sb2NrKTsKLQlhdmxfZGVzdHJv eSgmc2RwLT5zZF9zbmFwcyk7Ci0Ja21lbV9mcmVlKHNkcCwgc2l6ZW9mICh6ZnNjdGxfc25hcGRp cl90KSk7CiAKIAlyZXR1cm4gKDApOwogfQpAQCAtMTQ0MSw3ICsxNDM2LDYgQEAgemZzY3RsX3Nu YXBzaG90X21rbm9kZSh2bm9kZV90ICpwdnAsIHVpbnQ2NF90IG9ianNldCkKIAogCXZwID0gZ2Zz X2Rpcl9jcmVhdGUoc2l6ZW9mICh6ZnNjdGxfbm9kZV90KSwgcHZwLCBwdnAtPnZfdmZzcCwKIAkg ICAgJnpmc2N0bF9vcHNfc25hcHNob3QsIE5VTEwsIE5VTEwsIE1BWE5BTUVMRU4sIE5VTEwsIE5V TEwpOwotCVZOX0hPTEQodnApOwogCXpjcCA9IHZwLT52X2RhdGE7CiAJemNwLT56Y19pZCA9IG9i anNldDsKIAlWT1BfVU5MT0NLKHZwLCAwKTsKQEAgLTE0NjIsMTggKzE0NTYsMjUgQEAgemZzY3Rs X3NuYXBzaG90X2luYWN0aXZlKGFwKQogCXpmc2N0bF9zbmFwZGlyX3QgKnNkcDsKIAl6ZnNfc25h cGVudHJ5X3QgKnNlcCwgKm5leHQ7CiAJaW50IGxvY2tlZDsKLQl2bm9kZV90ICpkdnA7CisJZ2Zz X2Rpcl90ICpkcCA9IHZwLT52X2RhdGE7CisJdm5vZGVfdCAqZHZwID0gZHAtPmdmc2RfZmlsZS5n ZnNfcGFyZW50OwogCi0JaWYgKHZwLT52X2NvdW50ID4gMCkKLQkJZ290byBlbmQ7Ci0KLQlWRVJJ RlkoZ2ZzX2Rpcl9sb29rdXAodnAsICIuLiIsICZkdnAsIGNyLCAwLCBOVUxMLCBOVUxMKSA9PSAw KTsKKwlWTl9IT0xEKGR2cCk7CisJVk9QX1VOTE9DSyh2cCwgMCk7CiAJc2RwID0gZHZwLT52X2Rh dGE7Ci0JVk9QX1VOTE9DSyhkdnAsIDApOwogCiAJaWYgKCEobG9ja2VkID0gTVVURVhfSEVMRCgm c2RwLT5zZF9sb2NrKSkpCiAJCW11dGV4X2VudGVyKCZzZHAtPnNkX2xvY2spOwogCisJdm5fbG9j ayh2cCwgTEtfRVhDTFVTSVZFIHwgTEtfUkVUUlkpOworCisJaWYgKHZwLT52X2NvdW50ID4gMCkg eworCQlpZiAoIWxvY2tlZCkKKwkJCW11dGV4X2V4aXQoJnNkcC0+c2RfbG9jayk7CisJCVZOX1JF TEUoZHZwKTsKKwkJcmV0dXJuKDApOworCX0KKwogCUFTU0VSVCghdm5faXNtbnRwdCh2cCkpOwog CiAJc2VwID0gYXZsX2ZpcnN0KCZzZHAtPnNkX3NuYXBzKTsKQEAgLTE0OTQsNyArMTQ5NSw2IEBA IHpmc2N0bF9zbmFwc2hvdF9pbmFjdGl2ZShhcCkKIAkJbXV0ZXhfZXhpdCgmc2RwLT5zZF9sb2Nr KTsKIAlWTl9SRUxFKGR2cCk7CiAKLWVuZDoKIAkvKgogCSAqIERpc3Bvc2Ugb2YgdGhlIHZub2Rl IGZvciB0aGUgc25hcHNob3QgbW91bnQgcG9pbnQuCiAJICogVGhpcyBpcyBzYWZlIHRvIGRvIGJl Y2F1c2Ugb25jZSB0aGlzIGVudHJ5IGhhcyBiZWVuIHJlbW92ZWQKQEAgLTE1ODgsNyArMTU4OCw3 IEBAIHpmc2N0bF9zbmFwc2hvdF9sb29rdXAoYXApCiAJZXJyb3IgPSB6ZnNjdGxfcm9vdF9sb29r dXAoemZzdmZzLT56X2N0bGRpciwgInNuYXBzaG90IiwgdnBwLAogCSAgICBOVUxMLCAwLCBOVUxM LCBjciwgTlVMTCwgTlVMTCwgTlVMTCk7CiAJaWYgKGVycm9yID09IDApCi0JCXZuX2xvY2soKnZw cCwgTEtfRVhDTFVTSVZFIHwgTEtfUkVUUlkpOworCQllcnJvciA9IHZuX2xvY2soKnZwcCwgYXAt PmFfY25wLT5jbl9sa2ZsYWdzKTsKIAlyZXR1cm4gKGVycm9yKTsKIH0KIApkaWZmIC0tZ2l0IGEv c3lzL2NkZGwvY29udHJpYi9vcGVuc29sYXJpcy91dHMvY29tbW9uL2ZzL3pmcy96ZnNfdmZzb3Bz LmMgYi9zeXMvY2RkbC9jb250cmliL29wZW5zb2xhcmlzL3V0cy9jb21tb24vZnMvemZzL3pmc192 ZnNvcHMuYwppbmRleCA4ZWI4OTUzLi44ZWE3NjYxIDEwMDY0NAotLS0gYS9zeXMvY2RkbC9jb250 cmliL29wZW5zb2xhcmlzL3V0cy9jb21tb24vZnMvemZzL3pmc192ZnNvcHMuYworKysgYi9zeXMv Y2RkbC9jb250cmliL29wZW5zb2xhcmlzL3V0cy9jb21tb24vZnMvemZzL3pmc192ZnNvcHMuYwpA QCAtMjAzNiwxMiArMjAzNiw3IEBAIHpmc191bW91bnQodmZzX3QgKnZmc3AsIGludCBmZmxhZykK IAkgKi8KIAlpZiAoemZzdmZzLT56X2N0bGRpciAhPSBOVUxMKQogCQl6ZnNjdGxfZGVzdHJveSh6 ZnN2ZnMpOwotCWlmICh6ZnN2ZnMtPnpfaXNzbmFwKSB7Ci0JCXZub2RlX3QgKnN2cCA9IHZmc3At Pm1udF92bm9kZWNvdmVyZWQ7CiAKLQkJaWYgKHN2cC0+dl9jb3VudCA+PSAyKQotCQkJVk5fUkVM RShzdnApOwotCX0KIAl6ZnNfZnJlZXZmcyh2ZnNwKTsKIAogCXJldHVybiAoMCk7CmRpZmYgLS1n aXQgYS9zeXMva2Vybi92ZnNfc3Vici5jIGIvc3lzL2tlcm4vdmZzX3N1YnIuYwppbmRleCA5MWM2 NGEzLi4yOWI2Mzk1IDEwMDY0NAotLS0gYS9zeXMva2Vybi92ZnNfc3Vici5jCisrKyBiL3N5cy9r ZXJuL3Zmc19zdWJyLmMKQEAgLTExOTgsNyArMTE5OCwxMCBAQCBpbnNtbnRxdWUxKHN0cnVjdCB2 bm9kZSAqdnAsIHN0cnVjdCBtb3VudCAqbXAsCiAJfQogCXZwLT52X21vdW50ID0gbXA7CiAJTU5U X1JFRihtcCk7Ci0JVEFJTFFfSU5TRVJUX1RBSUwoJm1wLT5tbnRfbnZub2RlbGlzdCwgdnAsIHZf bm1udHZub2Rlcyk7CisJaWYgKHZwLT52X3ZmbGFnICYgVlZfSU5TTVFIRUFEKQorCQlUQUlMUV9J TlNFUlRfSEVBRCgmbXAtPm1udF9udm5vZGVsaXN0LCB2cCwgdl9ubW50dm5vZGVzKTsKKwllbHNl CisJCVRBSUxRX0lOU0VSVF9UQUlMKCZtcC0+bW50X252bm9kZWxpc3QsIHZwLCB2X25tbnR2bm9k ZXMpOwogCVZOQVNTRVJUKG1wLT5tbnRfbnZub2RlbGlzdHNpemUgPj0gMCwgdnAsCiAJCSgibmVn IG1vdW50IHBvaW50IHZub2RlIGxpc3Qgc2l6ZSIpKTsKIAltcC0+bW50X252bm9kZWxpc3RzaXpl Kys7CmRpZmYgLS1naXQgYS9zeXMvc3lzL3Zub2RlLmggYi9zeXMvc3lzL3Zub2RlLmgKaW5kZXgg YjBjYmNjMC4uOGRmYWUzYSAxMDA2NDQKLS0tIGEvc3lzL3N5cy92bm9kZS5oCisrKyBiL3N5cy9z eXMvdm5vZGUuaApAQCAtMjUzLDYgKzI1Myw3IEBAIHN0cnVjdCB4dm5vZGUgewogI2RlZmluZQlW Vl9ERUxFVEVECTB4MDQwMAkvKiBzaG91bGQgYmUgcmVtb3ZlZCAqLwogI2RlZmluZQlWVl9NRAkJ MHgwODAwCS8qIHZub2RlIGJhY2tzIHRoZSBtZCBkZXZpY2UgKi8KICNkZWZpbmUJVlZfRk9SQ0VJ TlNNUQkweDEwMDAJLyogZm9yY2UgdGhlIGluc21udHF1ZSB0byBzdWNjZWVkICovCisjZGVmaW5l CVZWX0lOU01RSEVBRAkweDIwMDAJLyogaW5zZXJ0IGluc3RlYWQgb2YgYXBwZW5kaW5nIHRvIG1u dF9udm5vZGVsaXN0ICovCiAKIC8qCiAgKiBWbm9kZSBhdHRyaWJ1dGVzLiAgQSBmaWVsZCB2YWx1 ZSBvZiBWTk9WQUwgcmVwcmVzZW50cyBhIGZpZWxkIHdob3NlIHZhbHVlCg== --=_6d066b626d245eaf36608aa45d442590-- From owner-zfs-devel@FreeBSD.ORG Tue Jan 21 15:58:03 2014 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 31FF6C16 for ; Tue, 21 Jan 2014 15:58:03 +0000 (UTC) Received: from pi.nmdps.net (pi.nmdps.net [IPv6:2a01:be00:10:201:0:80:0:1]) by mx1.freebsd.org (Postfix) with ESMTP id 59FDD1B6D for ; Tue, 21 Jan 2014 15:58:02 +0000 (UTC) Received: from pi.nmdps.net (localhost [127.0.0.1]) (Authenticated sender: krichy@cflinux.hu) by pi.nmdps.net (Postfix) with ESMTPSA id 3CC70110C for ; Tue, 21 Jan 2014 16:58:00 +0100 (CET) MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=_7b2dab942a25523d4e82e8dd34ed90f6" Date: Tue, 21 Jan 2014 16:57:57 +0100 From: krichy@cflinux.hu To: zfs-devel@freebsd.org Subject: Re: Fwd: Re: ZFS .zfs DoS In-Reply-To: References: Message-ID: X-Sender: krichy@cflinux.hu User-Agent: Roundcube Webmail/0.9.5 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 21 Jan 2014 15:58:03 -0000 --=_7b2dab942a25523d4e82e8dd34ed90f6 Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset=UTF-8; format=flowed A new version of the ZFS/GFS patch, fixing possible leaks. Please someone comment. 2014-01-20 20:21 időpontban krichy@cflinux.hu ezt írta: > -------- Eredeti üzenet -------- > Tárgy: Re: ZFS .zfs DoS > Dátum: 2014-01-20 16:30 > Feladó: krichy@cflinux.hu > Címzett: Richard Kojedzinszky > Másolat: freebsd-fs@freebsd.org, freebsd-security@freebsd.org > > Dear users, > > I've worked out a patch for my known issues, please somebody test > them, and give recommendations, fixes. > > Regards, > > 2014-01-17 03:11 időpontban Richard Kojedzinszky ezt írta: >> Dear users, >> >> For a long time now I've been investigating problems relating FreeBSD >> ZFS .zfs handling, and found that I am not enough to fix issues. Until >> fixes arrive, unfortunately a regular user can DoS a FreeBSD system >> which has ZFS filesystems with the attached script. While the script >> expects a snapshot argument to be given, actually the first test case >> does not need that, only a mounted zfs filesystem is enough. For more >> of the tests a snapshot may be needed, and later ones need root >> account also. >> >> I would recommend that until this gets rewritten or fixed at all, one >> should disable access to .zfs at all with someting like I've attached. >> >> Regards, >> Kojedzinszky Richard --=_7b2dab942a25523d4e82e8dd34ed90f6 Content-Transfer-Encoding: base64 Content-Type: text/x-diff; name=gfs-4.patch Content-Disposition: attachment; filename=gfs-4.patch; size=10845 Y29tbWl0IDEyYWUxNGRlNDUyMmNiODA1Yzg3MjNkM2U2NWQwNzNhN2Y4YmU4OTEKQXV0aG9yOiBS aWNoYXJkIEtvamVkemluc3preSA8a3JpY2h5QGNmbGludXguaHU+CkRhdGU6ICAgRnJpIEphbiAx NyAyMjo1NzozMyAyMDE0ICswMTAwCgogICAgWkZTL0dGUyBoYW5kbGluZyBmaXhlcwoKZGlmZiAt LWdpdCBhL3N5cy9jZGRsL2NvbXBhdC9vcGVuc29sYXJpcy9rZXJuL29wZW5zb2xhcmlzX2xvb2t1 cC5jIGIvc3lzL2NkZGwvY29tcGF0L29wZW5zb2xhcmlzL2tlcm4vb3BlbnNvbGFyaXNfbG9va3Vw LmMKaW5kZXggOTQzODNkNi4uNGNhYzA1MyAxMDA2NDQKLS0tIGEvc3lzL2NkZGwvY29tcGF0L29w ZW5zb2xhcmlzL2tlcm4vb3BlbnNvbGFyaXNfbG9va3VwLmMKKysrIGIvc3lzL2NkZGwvY29tcGF0 L29wZW5zb2xhcmlzL2tlcm4vb3BlbnNvbGFyaXNfbG9va3VwLmMKQEAgLTgxLDYgKzgxLDggQEAg dHJhdmVyc2Uodm5vZGVfdCAqKmN2cHAsIGludCBsa3R5cGUpCiAJICogcHJvZ3Jlc3Mgb24gdGhp cyB2bm9kZS4KIAkgKi8KIAorCXZuX2xvY2soY3ZwLCBsa3R5cGUpOworCiAJZm9yICg7Oykgewog CQkvKgogCQkgKiBSZWFjaGVkIHRoZSBlbmQgb2YgdGhlIG1vdW50IGNoYWluPwpAQCAtODksMTMg KzkxLDcgQEAgdHJhdmVyc2Uodm5vZGVfdCAqKmN2cHAsIGludCBsa3R5cGUpCiAJCWlmICh2ZnNw ID09IE5VTEwpCiAJCQlicmVhazsKIAkJZXJyb3IgPSB2ZnNfYnVzeSh2ZnNwLCAwKTsKLQkJLyoK LQkJICogdHZwIGlzIE5VTEwgZm9yICpjdnBwIHZub2RlLCB3aGljaCB3ZSBjYW4ndCB1bmxvY2su Ci0JCSAqLwotCQlpZiAodHZwICE9IE5VTEwpCi0JCQl2cHV0KGN2cCk7Ci0JCWVsc2UKLQkJCXZy ZWxlKGN2cCk7CisJCVZPUF9VTkxPQ0soY3ZwLCAwKTsKIAkJaWYgKGVycm9yKQogCQkJcmV0dXJu IChlcnJvcik7CiAKQEAgLTEwNyw2ICsxMDMsOSBAQCB0cmF2ZXJzZSh2bm9kZV90ICoqY3ZwcCwg aW50IGxrdHlwZSkKIAkJdmZzX3VuYnVzeSh2ZnNwKTsKIAkJaWYgKGVycm9yICE9IDApCiAJCQly ZXR1cm4gKGVycm9yKTsKKworCQlWTl9SRUxFKGN2cCk7CisKIAkJY3ZwID0gdHZwOwogCX0KIApk aWZmIC0tZ2l0IGEvc3lzL2NkZGwvY29tcGF0L29wZW5zb2xhcmlzL2tlcm4vb3BlbnNvbGFyaXNf dmZzLmMgYi9zeXMvY2RkbC9jb21wYXQvb3BlbnNvbGFyaXMva2Vybi9vcGVuc29sYXJpc192ZnMu YwppbmRleCBhMjUzMmY4Li5jMzAyYTU0IDEwMDY0NAotLS0gYS9zeXMvY2RkbC9jb21wYXQvb3Bl bnNvbGFyaXMva2Vybi9vcGVuc29sYXJpc192ZnMuYworKysgYi9zeXMvY2RkbC9jb21wYXQvb3Bl bnNvbGFyaXMva2Vybi9vcGVuc29sYXJpc192ZnMuYwpAQCAtMTk0LDEwICsxOTQsOCBAQCBtb3Vu dF9zbmFwc2hvdChrdGhyZWFkX3QgKnRkLCB2bm9kZV90ICoqdnBwLCBjb25zdCBjaGFyICpmc3R5 cGUsIGNoYXIgKmZzcGF0aCwKIAkJVklfTE9DSyh2cCk7CiAJCXZwLT52X2lmbGFnICY9IH5WSV9N T1VOVDsKIAkJVklfVU5MT0NLKHZwKTsKLQkJdnJlbGUodnApOwogCQl2ZnNfdW5idXN5KG1wKTsK IAkJdmZzX21vdW50X2Rlc3Ryb3kobXApOwotCQkqdnBwID0gTlVMTDsKIAkJcmV0dXJuIChlcnJv cik7CiAJfQogCkBAIC0yMjgsNyArMjI2LDcgQEAgbW91bnRfc25hcHNob3Qoa3RocmVhZF90ICp0 ZCwgdm5vZGVfdCAqKnZwcCwgY29uc3QgY2hhciAqZnN0eXBlLCBjaGFyICpmc3BhdGgsCiAJdmZz X2V2ZW50X3NpZ25hbChOVUxMLCBWUV9NT1VOVCwgMCk7CiAJaWYgKFZGU19ST09UKG1wLCBMS19F WENMVVNJVkUsICZtdnApKQogCQlwYW5pYygibW91bnQ6IGxvc3QgbW91bnQiKTsKLQl2cHV0KHZw KTsKKwlWT1BfVU5MT0NLKHZwLCAwKTsKIAl2ZnNfdW5idXN5KG1wKTsKIAkqdnBwID0gbXZwOwog CXJldHVybiAoMCk7CmRpZmYgLS1naXQgYS9zeXMvY2RkbC9jb250cmliL29wZW5zb2xhcmlzL3V0 cy9jb21tb24vZnMvZ2ZzLmMgYi9zeXMvY2RkbC9jb250cmliL29wZW5zb2xhcmlzL3V0cy9jb21t b24vZnMvZ2ZzLmMKaW5kZXggNTk5NDRhMS4uNTY3ZWUzOCAxMDA2NDQKLS0tIGEvc3lzL2NkZGwv Y29udHJpYi9vcGVuc29sYXJpcy91dHMvY29tbW9uL2ZzL2dmcy5jCisrKyBiL3N5cy9jZGRsL2Nv bnRyaWIvb3BlbnNvbGFyaXMvdXRzL2NvbW1vbi9mcy9nZnMuYwpAQCAtNDQ4LDcgKzQ0OCw2IEBA IGdmc19sb29rdXBfZG90KHZub2RlX3QgKip2cHAsIHZub2RlX3QgKmR2cCwgdm5vZGVfdCAqcHZw LCBjb25zdCBjaGFyICpubSkKIAkJCVZOX0hPTEQocHZwKTsKIAkJCSp2cHAgPSBwdnA7CiAJCX0K LQkJdm5fbG9jaygqdnBwLCBMS19FWENMVVNJVkUgfCBMS19SRVRSWSk7CiAJCXJldHVybiAoMCk7 CiAJfQogCkBAIC00ODUsNiArNDg0LDcgQEAgZ2ZzX2ZpbGVfY3JlYXRlKHNpemVfdCBzaXplLCB2 bm9kZV90ICpwdnAsIHZmc190ICp2ZnNwLCB2bm9kZW9wc190ICpvcHMpCiAJZnAgPSBrbWVtX3ph bGxvYyhzaXplLCBLTV9TTEVFUCk7CiAJZXJyb3IgPSBnZXRuZXd2bm9kZSgiemZzIiwgdmZzcCwg b3BzLCAmdnApOwogCUFTU0VSVChlcnJvciA9PSAwKTsKKwlWTl9MT0NLX0FTSEFSRSh2cCk7CiAJ dm5fbG9jayh2cCwgTEtfRVhDTFVTSVZFIHwgTEtfUkVUUlkpOwogCXZwLT52X2RhdGEgPSAoY2Fk ZHJfdClmcDsKIApAQCAtNjM3LDEyICs2MzcsNyBAQCBnZnNfZmlsZV9pbmFjdGl2ZSh2bm9kZV90 ICp2cCkKIAlpZiAoZnAtPmdmc19wYXJlbnQgPT0gTlVMTCB8fCAodnAtPnZfZmxhZyAmIFZfWEFU VFJESVIpKQogCQlnb3RvIGZvdW5kOwogCi0JLyoKLQkgKiBYWFggY29wZSB3aXRoIGEgRnJlZUJT RC1zcGVjaWZpYyByYWNlIHdoZXJlaW4gdGhlIHBhcmVudCdzCi0JICogc25hcHNob3QgZGF0YSBj YW4gYmUgZnJlZWQgYmVmb3JlIHRoZSBwYXJlbnQgaXMKLQkgKi8KLQlpZiAoKGRwID0gZnAtPmdm c19wYXJlbnQtPnZfZGF0YSkgPT0gTlVMTCkKLQkJcmV0dXJuIChOVUxMKTsKKwlkcCA9IGZwLT5n ZnNfcGFyZW50LT52X2RhdGE7CiAKIAkvKgogCSAqIEZpcnN0LCBzZWUgaWYgdGhpcyB2bm9kZSBp cyBjYWNoZWQgaW4gdGhlIHBhcmVudC4KQEAgLTY2OSwzNyArNjY0LDQ0IEBAIGZvdW5kOgogCWlm ICh2cC0+dl9mbGFnICYgVl9YQVRUUkRJUikKIAkJVklfTE9DSyhmcC0+Z2ZzX3BhcmVudCk7CiAj ZW5kaWYKLQlWSV9MT0NLKHZwKTsKLQkvKgotCSAqIFJlYWxseSByZW1vdmUgdGhpcyB2bm9kZQot CSAqLwotCWRhdGEgPSB2cC0+dl9kYXRhOwotCWlmIChnZSAhPSBOVUxMKSB7CisJaWYgKHZwLT52 X2NvdW50ID09IDAgfHwgdnAtPnZfaWZsYWcgJiBWSV9ET09NRUQpIHsKIAkJLyoKLQkJICogSWYg dGhpcyB3YXMgYSBzdGF0aWNhbGx5IGNhY2hlZCBlbnRyeSwgc2ltcGx5IHNldCB0aGUKLQkJICog Y2FjaGVkIHZub2RlIHRvIE5VTEwuCisJCSAqIFJlYWxseSByZW1vdmUgdGhpcyB2bm9kZQogCQkg Ki8KLQkJZ2UtPmdmc2Vfdm5vZGUgPSBOVUxMOwotCX0KLQlWSV9VTkxPQ0sodnApOworCQlkYXRh ID0gdnAtPnZfZGF0YTsKKwkJaWYgKGdlICE9IE5VTEwpIHsKKwkJCS8qCisJCQkgKiBJZiB0aGlz IHdhcyBhIHN0YXRpY2FsbHkgY2FjaGVkIGVudHJ5LCBzaW1wbHkgc2V0IHRoZQorCQkJICogY2Fj aGVkIHZub2RlIHRvIE5VTEwuCisJCQkgKi8KKwkJCWdlLT5nZnNlX3Zub2RlID0gTlVMTDsKKwkJ fQorI2lmZGVmIFRPRE8KKwkJaWYgKHZwLT52X2ZsYWcgJiBWX1hBVFRSRElSKQorCQkJVklfVU5M T0NLKGZwLT5nZnNfcGFyZW50KTsKKyNlbmRpZgogCi0JLyoKLQkgKiBGcmVlIHZub2RlIGFuZCBy ZWxlYXNlIHBhcmVudAotCSAqLwotCWlmIChmcC0+Z2ZzX3BhcmVudCkgewotCQlpZiAoZHApCi0J CQlnZnNfZGlyX3VubG9jayhkcCk7Ci0JCVZPUF9VTkxPQ0sodnAsIDApOwotCQlWTl9SRUxFKGZw LT5nZnNfcGFyZW50KTsKLQkJdm5fbG9jayh2cCwgTEtfRVhDTFVTSVZFIHwgTEtfUkVUUlkpOwor CQkvKgorCQkgKiBGcmVlIHZub2RlIGFuZCByZWxlYXNlIHBhcmVudAorCQkgKi8KKwkJaWYgKGZw LT5nZnNfcGFyZW50KSB7CisJCQlpZiAoZHApCisJCQkJZ2ZzX2Rpcl91bmxvY2soZHApOworCQkJ Vk5fUkVMRShmcC0+Z2ZzX3BhcmVudCk7CisJCX0gZWxzZSB7CisJCQlBU1NFUlQodnAtPnZfdmZz cCAhPSBOVUxMKTsKKwkJCVZGU19SRUxFKHZwLT52X3Zmc3ApOworCQl9CiAJfSBlbHNlIHsKLQkJ QVNTRVJUKHZwLT52X3Zmc3AgIT0gTlVMTCk7Ci0JCVZGU19SRUxFKHZwLT52X3Zmc3ApOwotCX0K KwkJZGF0YSA9IE5VTEw7CiAjaWZkZWYgVE9ETwotCWlmICh2cC0+dl9mbGFnICYgVl9YQVRUUkRJ UikKLQkJVklfVU5MT0NLKGZwLT5nZnNfcGFyZW50KTsKKwkJaWYgKHZwLT52X2ZsYWcgJiBWX1hB VFRSRElSKQorCQkJVklfVU5MT0NLKGZwLT5nZnNfcGFyZW50KTsKICNlbmRpZgorCQlpZiAoZHAp CisJCQlnZnNfZGlyX3VubG9jayhkcCk7CisJfQorCiAJcmV0dXJuIChkYXRhKTsKIH0KIApAQCAt MTIzMCwxNiArMTIzMiwxNSBAQCBnZnNfdm9wX2luYWN0aXZlKGFwKQogewogCXZub2RlX3QgKnZw ID0gYXAtPmFfdnA7CiAJZ2ZzX2ZpbGVfdCAqZnAgPSB2cC0+dl9kYXRhOworCXZvaWQgKmRhdGE7 CiAKIAlpZiAoZnAtPmdmc190eXBlID09IEdGU19ESVIpCi0JCWdmc19kaXJfaW5hY3RpdmUodnAp OworCQlkYXRhID0gZ2ZzX2Rpcl9pbmFjdGl2ZSh2cCk7CiAJZWxzZQotCQlnZnNfZmlsZV9pbmFj dGl2ZSh2cCk7CisJCWRhdGEgPSBnZnNfZmlsZV9pbmFjdGl2ZSh2cCk7CiAKLQlWSV9MT0NLKHZw KTsKLQl2cC0+dl9kYXRhID0gTlVMTDsKLQlWSV9VTkxPQ0sodnApOwotCWttZW1fZnJlZShmcCwg ZnAtPmdmc19zaXplKTsKKwlpZiAoZGF0YSAhPSBOVUxMKQorCQlrbWVtX2ZyZWUoZGF0YSwgZnAt Pmdmc19zaXplKTsKIAogCXJldHVybiAoMCk7CiB9CmRpZmYgLS1naXQgYS9zeXMvY2RkbC9jb250 cmliL29wZW5zb2xhcmlzL3V0cy9jb21tb24vZnMvemZzL3pmc19jdGxkaXIuYyBiL3N5cy9jZGRs L2NvbnRyaWIvb3BlbnNvbGFyaXMvdXRzL2NvbW1vbi9mcy96ZnMvemZzX2N0bGRpci5jCmluZGV4 IDI4YWIxZmEuLmIxZjAwNzUgMTAwNjQ0Ci0tLSBhL3N5cy9jZGRsL2NvbnRyaWIvb3BlbnNvbGFy aXMvdXRzL2NvbW1vbi9mcy96ZnMvemZzX2N0bGRpci5jCisrKyBiL3N5cy9jZGRsL2NvbnRyaWIv b3BlbnNvbGFyaXMvdXRzL2NvbW1vbi9mcy96ZnMvemZzX2N0bGRpci5jCkBAIC02MTIsNyArNjEy LDcgQEAgemZzY3RsX2ZyZWVic2Rfcm9vdF9sb29rdXAoYXApCiAKIAllcnIgPSB6ZnNjdGxfcm9v dF9sb29rdXAoZHZwLCBubSwgdnBwLCBOVUxMLCAwLCBOVUxMLCBjciwgTlVMTCwgTlVMTCwgTlVM TCk7CiAJaWYgKGVyciA9PSAwICYmIChubVswXSAhPSAnLicgfHwgbm1bMV0gIT0gJ1wwJykpCi0J CXZuX2xvY2soKnZwcCwgTEtfRVhDTFVTSVZFIHwgTEtfUkVUUlkpOworCQllcnIgPSB2bl9sb2Nr KCp2cHAsIGFwLT5hX2NucC0+Y25fbGtmbGFncyk7CiAJcmV0dXJuIChlcnIpOwogfQogCkBAIC05 NzUsOCArOTc1LDExIEBAIHpmc2N0bF9zbmFwZGlyX2xvb2t1cChhcCkKIAlaRlNfRU5URVIoemZz dmZzKTsKIAogCWlmIChnZnNfbG9va3VwX2RvdCh2cHAsIGR2cCwgemZzdmZzLT56X2N0bGRpciwg bm0pID09IDApIHsKKwkJZXJyID0gMDsKKwkJaWYgKG5tWzBdICE9ICcuJyB8fCBubVsxXSAhPSAn XDAnKQorCQkJZXJyID0gdm5fbG9jaygqdnBwLCBhcC0+YV9jbnAtPmNuX2xrZmxhZ3MpOwogCQla RlNfRVhJVCh6ZnN2ZnMpOwotCQlyZXR1cm4gKDApOworCQlyZXR1cm4gKGVycik7CiAJfQogCiAJ aWYgKGZsYWdzICYgRklHTk9SRUNBU0UpIHsKQEAgLTEwMDQsNyArMTAwNyw3IEBAIHpmc2N0bF9z bmFwZGlyX2xvb2t1cChhcCkKIAlpZiAoKHNlcCA9IGF2bF9maW5kKCZzZHAtPnNkX3NuYXBzLCAm c2VhcmNoLCAmd2hlcmUpKSAhPSBOVUxMKSB7CiAJCSp2cHAgPSBzZXAtPnNlX3Jvb3Q7CiAJCVZO X0hPTEQoKnZwcCk7Ci0JCWVyciA9IHRyYXZlcnNlKHZwcCwgTEtfRVhDTFVTSVZFIHwgTEtfUkVU UlkpOworCQllcnIgPSB0cmF2ZXJzZSh2cHAsIGFwLT5hX2NucC0+Y25fbGtmbGFncyk7CiAJCWlm IChlcnIgIT0gMCkgewogCQkJVk5fUkVMRSgqdnBwKTsKIAkJCSp2cHAgPSBOVUxMOwpAQCAtMTAx Myw2ICsxMDE2LDcgQEAgemZzY3RsX3NuYXBkaXJfbG9va3VwKGFwKQogCQkJICogVGhlIHNuYXBz aG90IHdhcyB1bm1vdW50ZWQgYmVoaW5kIG91ciBiYWNrcywKIAkJCSAqIHRyeSB0byByZW1vdW50 IGl0LgogCQkJICovCisJCQlWT1BfVU5MT0NLKCp2cHAsIDApOwogCQkJVkVSSUZZKHpmc2N0bF9z bmFwc2hvdF96bmFtZShkdnAsIG5tLCBNQVhOQU1FTEVOLCBzbmFwbmFtZSkgPT0gMCk7CiAJCQln b3RvIGRvbW91bnQ7CiAJCX0gZWxzZSB7CkBAIC0xMDY0LDcgKzEwNjgsNiBAQCB6ZnNjdGxfc25h cGRpcl9sb29rdXAoYXApCiAJc2VwLT5zZV9uYW1lID0ga21lbV9hbGxvYyhzdHJsZW4obm0pICsg MSwgS01fU0xFRVApOwogCSh2b2lkKSBzdHJjcHkoc2VwLT5zZV9uYW1lLCBubSk7CiAJKnZwcCA9 IHNlcC0+c2Vfcm9vdCA9IHpmc2N0bF9zbmFwc2hvdF9ta25vZGUoZHZwLCBkbXVfb2Jqc2V0X2lk KHNuYXApKTsKLQlWTl9IT0xEKCp2cHApOwogCWF2bF9pbnNlcnQoJnNkcC0+c2Rfc25hcHMsIHNl cCwgd2hlcmUpOwogCiAJZG11X29ianNldF9yZWxlKHNuYXAsIEZUQUcpOwpAQCAtMTA5MSw3ICsx MDk0LDYgQEAgZG9tb3VudDoKIAltdXRleF9leGl0KCZzZHAtPnNkX2xvY2spOwogCVpGU19FWElU KHpmc3Zmcyk7CiAKLSNpZmRlZiBpbGx1bW9zCiAJLyoKIAkgKiBJZiB3ZSBoYWQgYW4gZXJyb3Is IGRyb3Agb3VyIGhvbGQgb24gdGhlIHZub2RlIGFuZAogCSAqIHpmc2N0bF9zbmFwc2hvdF9pbmFj dGl2ZSgpIHdpbGwgY2xlYW4gdXAuCkBAIC0xMTAwLDEwICsxMTAyLDYgQEAgZG9tb3VudDoKIAkJ Vk5fUkVMRSgqdnBwKTsKIAkJKnZwcCA9IE5VTEw7CiAJfQotI2Vsc2UKLQlpZiAoZXJyICE9IDAp Ci0JCSp2cHAgPSBOVUxMOwotI2VuZGlmCiAJcmV0dXJuIChlcnIpOwogfQogCkBAIC0xMTMwLDgg KzExMjgsMTEgQEAgemZzY3RsX3NoYXJlc19sb29rdXAoYXApCiAJc3RybGNweShubSwgY25wLT5j bl9uYW1lcHRyLCBjbnAtPmNuX25hbWVsZW4gKyAxKTsKIAogCWlmIChnZnNfbG9va3VwX2RvdCh2 cHAsIGR2cCwgemZzdmZzLT56X2N0bGRpciwgbm0pID09IDApIHsKKwkJZXJyb3IgPSAwOworCQlp ZiAobm1bMF0gIT0gJy4nIHx8IG5tWzFdICE9ICdcMCcpCisJCQllcnJvciA9IHZuX2xvY2soKnZw cCwgYXAtPmFfY25wLT5jbl9sa2ZsYWdzKTsKIAkJWkZTX0VYSVQoemZzdmZzKTsKLQkJcmV0dXJu ICgwKTsKKwkJcmV0dXJuIChlcnJvcik7CiAJfQogCiAJaWYgKHpmc3Zmcy0+el9zaGFyZXNfZGly ID09IDApIHsKQEAgLTEzNDQsMjIgKzEzNDUsMTUgQEAgemZzY3RsX3NuYXBkaXJfaW5hY3RpdmUo YXApCiAJdm5vZGVfdCAqdnAgPSBhcC0+YV92cDsKIAl6ZnNjdGxfc25hcGRpcl90ICpzZHAgPSB2 cC0+dl9kYXRhOwogCXpmc19zbmFwZW50cnlfdCAqc2VwOwotCi0JLyoKLQkgKiBPbiBmb3JjZWQg dW5tb3VudCB3ZSBoYXZlIHRvIGZyZWUgc25hcHNob3RzIGZyb20gaGVyZS4KLQkgKi8KLQltdXRl eF9lbnRlcigmc2RwLT5zZF9sb2NrKTsKLQl3aGlsZSAoKHNlcCA9IGF2bF9maXJzdCgmc2RwLT5z ZF9zbmFwcykpICE9IE5VTEwpIHsKLQkJYXZsX3JlbW92ZSgmc2RwLT5zZF9zbmFwcywgc2VwKTsK LQkJa21lbV9mcmVlKHNlcC0+c2VfbmFtZSwgc3RybGVuKHNlcC0+c2VfbmFtZSkgKyAxKTsKLQkJ a21lbV9mcmVlKHNlcCwgc2l6ZW9mICh6ZnNfc25hcGVudHJ5X3QpKTsKKwl2b2lkICpwcml2YXRl OworCisJcHJpdmF0ZSA9IGdmc19kaXJfaW5hY3RpdmUodnApOworCWlmIChwcml2YXRlICE9IE5V TEwpIHsKKwkJQVNTRVJUKGF2bF9udW1ub2Rlcygmc2RwLT5zZF9zbmFwcykgPT0gMCk7CisJCW11 dGV4X2Rlc3Ryb3koJnNkcC0+c2RfbG9jayk7CisJCWF2bF9kZXN0cm95KCZzZHAtPnNkX3NuYXBz KTsKKwkJa21lbV9mcmVlKHByaXZhdGUsIHNpemVvZiAoemZzY3RsX3NuYXBkaXJfdCkpOwogCX0K LQltdXRleF9leGl0KCZzZHAtPnNkX2xvY2spOwotCWdmc19kaXJfaW5hY3RpdmUodnApOwotCUFT U0VSVChhdmxfbnVtbm9kZXMoJnNkcC0+c2Rfc25hcHMpID09IDApOwotCW11dGV4X2Rlc3Ryb3ko JnNkcC0+c2RfbG9jayk7Ci0JYXZsX2Rlc3Ryb3koJnNkcC0+c2Rfc25hcHMpOwotCWttZW1fZnJl ZShzZHAsIHNpemVvZiAoemZzY3RsX3NuYXBkaXJfdCkpOwogCiAJcmV0dXJuICgwKTsKIH0KQEAg LTE0NDEsNyArMTQzNSw2IEBAIHpmc2N0bF9zbmFwc2hvdF9ta25vZGUodm5vZGVfdCAqcHZwLCB1 aW50NjRfdCBvYmpzZXQpCiAKIAl2cCA9IGdmc19kaXJfY3JlYXRlKHNpemVvZiAoemZzY3RsX25v ZGVfdCksIHB2cCwgcHZwLT52X3Zmc3AsCiAJICAgICZ6ZnNjdGxfb3BzX3NuYXBzaG90LCBOVUxM LCBOVUxMLCBNQVhOQU1FTEVOLCBOVUxMLCBOVUxMKTsKLQlWTl9IT0xEKHZwKTsKIAl6Y3AgPSB2 cC0+dl9kYXRhOwogCXpjcC0+emNfaWQgPSBvYmpzZXQ7CiAJVk9QX1VOTE9DSyh2cCwgMCk7CkBA IC0xNDYyLDE4ICsxNDU1LDI1IEBAIHpmc2N0bF9zbmFwc2hvdF9pbmFjdGl2ZShhcCkKIAl6ZnNj dGxfc25hcGRpcl90ICpzZHA7CiAJemZzX3NuYXBlbnRyeV90ICpzZXAsICpuZXh0OwogCWludCBs b2NrZWQ7Ci0Jdm5vZGVfdCAqZHZwOworCWdmc19kaXJfdCAqZHAgPSB2cC0+dl9kYXRhOworCXZu b2RlX3QgKmR2cCA9IGRwLT5nZnNkX2ZpbGUuZ2ZzX3BhcmVudDsKIAotCWlmICh2cC0+dl9jb3Vu dCA+IDApCi0JCWdvdG8gZW5kOwotCi0JVkVSSUZZKGdmc19kaXJfbG9va3VwKHZwLCAiLi4iLCAm ZHZwLCBjciwgMCwgTlVMTCwgTlVMTCkgPT0gMCk7CisJVk5fSE9MRChkdnApOworCVZPUF9VTkxP Q0sodnAsIDApOwogCXNkcCA9IGR2cC0+dl9kYXRhOwotCVZPUF9VTkxPQ0soZHZwLCAwKTsKIAog CWlmICghKGxvY2tlZCA9IE1VVEVYX0hFTEQoJnNkcC0+c2RfbG9jaykpKQogCQltdXRleF9lbnRl cigmc2RwLT5zZF9sb2NrKTsKIAorCXZuX2xvY2sodnAsIExLX0VYQ0xVU0lWRSB8IExLX1JFVFJZ KTsKKworCWlmICh2cC0+dl9jb3VudCA+IDAgJiYgISh2cC0+dl9pZmxhZyAmIFZJX0RPT01FRCkp IHsKKwkJaWYgKCFsb2NrZWQpCisJCQltdXRleF9leGl0KCZzZHAtPnNkX2xvY2spOworCQlWTl9S RUxFKGR2cCk7CisJCXJldHVybigwKTsKKwl9CisKIAlBU1NFUlQoIXZuX2lzbW50cHQodnApKTsK IAogCXNlcCA9IGF2bF9maXJzdCgmc2RwLT5zZF9zbmFwcyk7CkBAIC0xNDk0LDcgKzE0OTQsNiBA QCB6ZnNjdGxfc25hcHNob3RfaW5hY3RpdmUoYXApCiAJCW11dGV4X2V4aXQoJnNkcC0+c2RfbG9j ayk7CiAJVk5fUkVMRShkdnApOwogCi1lbmQ6CiAJLyoKIAkgKiBEaXNwb3NlIG9mIHRoZSB2bm9k ZSBmb3IgdGhlIHNuYXBzaG90IG1vdW50IHBvaW50LgogCSAqIFRoaXMgaXMgc2FmZSB0byBkbyBi ZWNhdXNlIG9uY2UgdGhpcyBlbnRyeSBoYXMgYmVlbiByZW1vdmVkCkBAIC0xNTg4LDcgKzE1ODcs NyBAQCB6ZnNjdGxfc25hcHNob3RfbG9va3VwKGFwKQogCWVycm9yID0gemZzY3RsX3Jvb3RfbG9v a3VwKHpmc3Zmcy0+el9jdGxkaXIsICJzbmFwc2hvdCIsIHZwcCwKIAkgICAgTlVMTCwgMCwgTlVM TCwgY3IsIE5VTEwsIE5VTEwsIE5VTEwpOwogCWlmIChlcnJvciA9PSAwKQotCQl2bl9sb2NrKCp2 cHAsIExLX0VYQ0xVU0lWRSB8IExLX1JFVFJZKTsKKwkJZXJyb3IgPSB2bl9sb2NrKCp2cHAsIGFw LT5hX2NucC0+Y25fbGtmbGFncyk7CiAJcmV0dXJuIChlcnJvcik7CiB9CiAKZGlmZiAtLWdpdCBh L3N5cy9jZGRsL2NvbnRyaWIvb3BlbnNvbGFyaXMvdXRzL2NvbW1vbi9mcy96ZnMvemZzX3Zmc29w cy5jIGIvc3lzL2NkZGwvY29udHJpYi9vcGVuc29sYXJpcy91dHMvY29tbW9uL2ZzL3pmcy96ZnNf dmZzb3BzLmMKaW5kZXggOGViODk1My4uOGVhNzY2MSAxMDA2NDQKLS0tIGEvc3lzL2NkZGwvY29u dHJpYi9vcGVuc29sYXJpcy91dHMvY29tbW9uL2ZzL3pmcy96ZnNfdmZzb3BzLmMKKysrIGIvc3lz L2NkZGwvY29udHJpYi9vcGVuc29sYXJpcy91dHMvY29tbW9uL2ZzL3pmcy96ZnNfdmZzb3BzLmMK QEAgLTIwMzYsMTIgKzIwMzYsNyBAQCB6ZnNfdW1vdW50KHZmc190ICp2ZnNwLCBpbnQgZmZsYWcp CiAJICovCiAJaWYgKHpmc3Zmcy0+el9jdGxkaXIgIT0gTlVMTCkKIAkJemZzY3RsX2Rlc3Ryb3ko emZzdmZzKTsKLQlpZiAoemZzdmZzLT56X2lzc25hcCkgewotCQl2bm9kZV90ICpzdnAgPSB2ZnNw LT5tbnRfdm5vZGVjb3ZlcmVkOwogCi0JCWlmIChzdnAtPnZfY291bnQgPj0gMikKLQkJCVZOX1JF TEUoc3ZwKTsKLQl9CiAJemZzX2ZyZWV2ZnModmZzcCk7CiAKIAlyZXR1cm4gKDApOwpkaWZmIC0t Z2l0IGEvc3lzL2tlcm4vdmZzX3N1YnIuYyBiL3N5cy9rZXJuL3Zmc19zdWJyLmMKaW5kZXggOTFj NjRhMy4uNjRiZWM2YiAxMDA2NDQKLS0tIGEvc3lzL2tlcm4vdmZzX3N1YnIuYworKysgYi9zeXMv a2Vybi92ZnNfc3Vici5jCkBAIC0xMTk4LDcgKzExOTgsNyBAQCBpbnNtbnRxdWUxKHN0cnVjdCB2 bm9kZSAqdnAsIHN0cnVjdCBtb3VudCAqbXAsCiAJfQogCXZwLT52X21vdW50ID0gbXA7CiAJTU5U X1JFRihtcCk7Ci0JVEFJTFFfSU5TRVJUX1RBSUwoJm1wLT5tbnRfbnZub2RlbGlzdCwgdnAsIHZf bm1udHZub2Rlcyk7CisJVEFJTFFfSU5TRVJUX0hFQUQoJm1wLT5tbnRfbnZub2RlbGlzdCwgdnAs IHZfbm1udHZub2Rlcyk7CiAJVk5BU1NFUlQobXAtPm1udF9udm5vZGVsaXN0c2l6ZSA+PSAwLCB2 cCwKIAkJKCJuZWcgbW91bnQgcG9pbnQgdm5vZGUgbGlzdCBzaXplIikpOwogCW1wLT5tbnRfbnZu b2RlbGlzdHNpemUrKzsK --=_7b2dab942a25523d4e82e8dd34ed90f6-- From owner-zfs-devel@FreeBSD.ORG Sun Feb 2 23:26:31 2014 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 5F3CFEC0 for ; Sun, 2 Feb 2014 23:26:31 +0000 (UTC) Received: from pi.nmdps.net (pi.nmdps.net [IPv6:2a01:be00:10:201:0:80:0:1]) by mx1.freebsd.org (Postfix) with ESMTP id 8508510EA for ; Sun, 2 Feb 2014 23:26:29 +0000 (UTC) Received: from pi.nmdps.net (localhost [127.0.0.1]) (Authenticated sender: krichy@cflinux.hu) by pi.nmdps.net (Postfix) with ESMTPSA id A2D211178 for ; Mon, 3 Feb 2014 00:26:19 +0100 (CET) MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=_158609ab4d8429e04dff41c787e8eb7a" Date: Mon, 03 Feb 2014 00:26:17 +0100 From: krichy@cflinux.hu To: zfs-devel@freebsd.org Subject: Re: Fwd: Re: ZFS .zfs DoS In-Reply-To: References: Message-ID: <91bec6ab8248941c43490eb93c71fbdd@cflinux.hu> X-Sender: krichy@cflinux.hu User-Agent: Roundcube Webmail/0.9.5 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Feb 2014 23:26:31 -0000 --=_158609ab4d8429e04dff41c787e8eb7a Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset=UTF-8; format=flowed Again a new version, passing INVARIANTS checks. 2014-01-21 16:57 időpontban krichy@cflinux.hu ezt írta: > A new version of the ZFS/GFS patch, fixing possible leaks. > > Please someone comment. > > 2014-01-20 20:21 időpontban krichy@cflinux.hu ezt írta: >> -------- Eredeti üzenet -------- >> Tárgy: Re: ZFS .zfs DoS >> Dátum: 2014-01-20 16:30 >> Feladó: krichy@cflinux.hu >> Címzett: Richard Kojedzinszky >> Másolat: freebsd-fs@freebsd.org, freebsd-security@freebsd.org >> >> Dear users, >> >> I've worked out a patch for my known issues, please somebody test >> them, and give recommendations, fixes. >> >> Regards, >> >> 2014-01-17 03:11 időpontban Richard Kojedzinszky ezt írta: >>> Dear users, >>> >>> For a long time now I've been investigating problems relating FreeBSD >>> ZFS .zfs handling, and found that I am not enough to fix issues. >>> Until >>> fixes arrive, unfortunately a regular user can DoS a FreeBSD system >>> which has ZFS filesystems with the attached script. While the script >>> expects a snapshot argument to be given, actually the first test case >>> does not need that, only a mounted zfs filesystem is enough. For more >>> of the tests a snapshot may be needed, and later ones need root >>> account also. >>> >>> I would recommend that until this gets rewritten or fixed at all, one >>> should disable access to .zfs at all with someting like I've >>> attached. >>> >>> Regards, >>> Kojedzinszky Richard --=_158609ab4d8429e04dff41c787e8eb7a Content-Transfer-Encoding: base64 Content-Type: text/x-diff; name=gfs.patch Content-Disposition: attachment; filename=gfs.patch; size=10935 Y29tbWl0IGY2NzFhZGNkYjRiNDhhOTBmMjY1Mzk1YWI4ODhmNjlmNzhhZDY2NTAKQXV0aG9yOiBS aWNoYXJkIEtvamVkemluc3preSA8a3JpY2h5QHR2bmV0d29yay5odT4KRGF0ZTogICBUaHUgSmFu IDIzIDE3OjIxOjQwIDIwMTQgKzAxMDAKCiAgICBaRlMvR0ZTIGhhbmRsaW5nIGZpeGVzCgpkaWZm IC0tZ2l0IGEvc3lzL2NkZGwvY29tcGF0L29wZW5zb2xhcmlzL2tlcm4vb3BlbnNvbGFyaXNfbG9v a3VwLmMgYi9zeXMvY2RkbC9jb21wYXQvb3BlbnNvbGFyaXMva2Vybi9vcGVuc29sYXJpc19sb29r dXAuYwppbmRleCA2NTY1ODEwLi43NGVkMTczIDEwMDY0NAotLS0gYS9zeXMvY2RkbC9jb21wYXQv b3BlbnNvbGFyaXMva2Vybi9vcGVuc29sYXJpc19sb29rdXAuYworKysgYi9zeXMvY2RkbC9jb21w YXQvb3BlbnNvbGFyaXMva2Vybi9vcGVuc29sYXJpc19sb29rdXAuYwpAQCAtODEsNiArODEsOCBA QCB0cmF2ZXJzZSh2bm9kZV90ICoqY3ZwcCwgaW50IGxrdHlwZSkKIAkgKiBwcm9ncmVzcyBvbiB0 aGlzIHZub2RlLgogCSAqLwogCisJdm5fbG9jayhjdnAsIGxrdHlwZSk7CisKIAlmb3IgKDs7KSB7 CiAJCS8qCiAJCSAqIFJlYWNoZWQgdGhlIGVuZCBvZiB0aGUgbW91bnQgY2hhaW4/CkBAIC04OSwx MyArOTEsNyBAQCB0cmF2ZXJzZSh2bm9kZV90ICoqY3ZwcCwgaW50IGxrdHlwZSkKIAkJaWYgKHZm c3AgPT0gTlVMTCkKIAkJCWJyZWFrOwogCQllcnJvciA9IHZmc19idXN5KHZmc3AsIDApOwotCQkv KgotCQkgKiB0dnAgaXMgTlVMTCBmb3IgKmN2cHAgdm5vZGUsIHdoaWNoIHdlIGNhbid0IHVubG9j ay4KLQkJICovCi0JCWlmICh0dnAgIT0gTlVMTCkKLQkJCXZwdXQoY3ZwKTsKLQkJZWxzZQotCQkJ dnJlbGUoY3ZwKTsKKwkJVk9QX1VOTE9DSyhjdnAsIDApOwogCQlpZiAoZXJyb3IpCiAJCQlyZXR1 cm4gKGVycm9yKTsKIApAQCAtMTA3LDYgKzEwMyw5IEBAIHRyYXZlcnNlKHZub2RlX3QgKipjdnBw LCBpbnQgbGt0eXBlKQogCQl2ZnNfdW5idXN5KHZmc3ApOwogCQlpZiAoZXJyb3IgIT0gMCkKIAkJ CXJldHVybiAoZXJyb3IpOworCisJCVZOX1JFTEUoY3ZwKTsKKwogCQljdnAgPSB0dnA7CiAJfQog CmRpZmYgLS1naXQgYS9zeXMvY2RkbC9jb21wYXQvb3BlbnNvbGFyaXMva2Vybi9vcGVuc29sYXJp c192ZnMuYyBiL3N5cy9jZGRsL2NvbXBhdC9vcGVuc29sYXJpcy9rZXJuL29wZW5zb2xhcmlzX3Zm cy5jCmluZGV4IGEyNTMyZjguLmMzMDJhNTQgMTAwNjQ0Ci0tLSBhL3N5cy9jZGRsL2NvbXBhdC9v cGVuc29sYXJpcy9rZXJuL29wZW5zb2xhcmlzX3Zmcy5jCisrKyBiL3N5cy9jZGRsL2NvbXBhdC9v cGVuc29sYXJpcy9rZXJuL29wZW5zb2xhcmlzX3Zmcy5jCkBAIC0xOTQsMTAgKzE5NCw4IEBAIG1v dW50X3NuYXBzaG90KGt0aHJlYWRfdCAqdGQsIHZub2RlX3QgKip2cHAsIGNvbnN0IGNoYXIgKmZz dHlwZSwgY2hhciAqZnNwYXRoLAogCQlWSV9MT0NLKHZwKTsKIAkJdnAtPnZfaWZsYWcgJj0gflZJ X01PVU5UOwogCQlWSV9VTkxPQ0sodnApOwotCQl2cmVsZSh2cCk7CiAJCXZmc191bmJ1c3kobXAp OwogCQl2ZnNfbW91bnRfZGVzdHJveShtcCk7Ci0JCSp2cHAgPSBOVUxMOwogCQlyZXR1cm4gKGVy cm9yKTsKIAl9CiAKQEAgLTIyOCw3ICsyMjYsNyBAQCBtb3VudF9zbmFwc2hvdChrdGhyZWFkX3Qg KnRkLCB2bm9kZV90ICoqdnBwLCBjb25zdCBjaGFyICpmc3R5cGUsIGNoYXIgKmZzcGF0aCwKIAl2 ZnNfZXZlbnRfc2lnbmFsKE5VTEwsIFZRX01PVU5ULCAwKTsKIAlpZiAoVkZTX1JPT1QobXAsIExL X0VYQ0xVU0lWRSwgJm12cCkpCiAJCXBhbmljKCJtb3VudDogbG9zdCBtb3VudCIpOwotCXZwdXQo dnApOworCVZPUF9VTkxPQ0sodnAsIDApOwogCXZmc191bmJ1c3kobXApOwogCSp2cHAgPSBtdnA7 CiAJcmV0dXJuICgwKTsKZGlmZiAtLWdpdCBhL3N5cy9jZGRsL2NvbnRyaWIvb3BlbnNvbGFyaXMv dXRzL2NvbW1vbi9mcy9nZnMuYyBiL3N5cy9jZGRsL2NvbnRyaWIvb3BlbnNvbGFyaXMvdXRzL2Nv bW1vbi9mcy9nZnMuYwppbmRleCA1OTk0NGExLi5mYTljZjgzIDEwMDY0NAotLS0gYS9zeXMvY2Rk bC9jb250cmliL29wZW5zb2xhcmlzL3V0cy9jb21tb24vZnMvZ2ZzLmMKKysrIGIvc3lzL2NkZGwv Y29udHJpYi9vcGVuc29sYXJpcy91dHMvY29tbW9uL2ZzL2dmcy5jCkBAIC00NDgsNyArNDQ4LDYg QEAgZ2ZzX2xvb2t1cF9kb3Qodm5vZGVfdCAqKnZwcCwgdm5vZGVfdCAqZHZwLCB2bm9kZV90ICpw dnAsIGNvbnN0IGNoYXIgKm5tKQogCQkJVk5fSE9MRChwdnApOwogCQkJKnZwcCA9IHB2cDsKIAkJ fQotCQl2bl9sb2NrKCp2cHAsIExLX0VYQ0xVU0lWRSB8IExLX1JFVFJZKTsKIAkJcmV0dXJuICgw KTsKIAl9CiAKQEAgLTQ4NSw3ICs0ODQsOSBAQCBnZnNfZmlsZV9jcmVhdGUoc2l6ZV90IHNpemUs IHZub2RlX3QgKnB2cCwgdmZzX3QgKnZmc3AsIHZub2Rlb3BzX3QgKm9wcykKIAlmcCA9IGttZW1f emFsbG9jKHNpemUsIEtNX1NMRUVQKTsKIAllcnJvciA9IGdldG5ld3Zub2RlKCJ6ZnMiLCB2ZnNw LCBvcHMsICZ2cCk7CiAJQVNTRVJUKGVycm9yID09IDApOwotCXZuX2xvY2sodnAsIExLX0VYQ0xV U0lWRSB8IExLX1JFVFJZKTsKKwllcnJvciA9IHZuX2xvY2sodnAsIExLX0VYQ0xVU0lWRSk7CisJ S0FTU0VSVChlcnJvciA9PSAwLCAoInZuX2xvY2soKSBmYWlsZWQ6IGVycm9yICVkIiwgZXJyb3Ip KTsKKwlWTl9MT0NLX0FTSEFSRSh2cCk7CiAJdnAtPnZfZGF0YSA9IChjYWRkcl90KWZwOwogCiAJ LyoKQEAgLTYzNywxMiArNjM4LDcgQEAgZ2ZzX2ZpbGVfaW5hY3RpdmUodm5vZGVfdCAqdnApCiAJ aWYgKGZwLT5nZnNfcGFyZW50ID09IE5VTEwgfHwgKHZwLT52X2ZsYWcgJiBWX1hBVFRSRElSKSkK IAkJZ290byBmb3VuZDsKIAotCS8qCi0JICogWFhYIGNvcGUgd2l0aCBhIEZyZWVCU0Qtc3BlY2lm aWMgcmFjZSB3aGVyZWluIHRoZSBwYXJlbnQncwotCSAqIHNuYXBzaG90IGRhdGEgY2FuIGJlIGZy ZWVkIGJlZm9yZSB0aGUgcGFyZW50IGlzCi0JICovCi0JaWYgKChkcCA9IGZwLT5nZnNfcGFyZW50 LT52X2RhdGEpID09IE5VTEwpCi0JCXJldHVybiAoTlVMTCk7CisJZHAgPSBmcC0+Z2ZzX3BhcmVu dC0+dl9kYXRhOwogCiAJLyoKIAkgKiBGaXJzdCwgc2VlIGlmIHRoaXMgdm5vZGUgaXMgY2FjaGVk IGluIHRoZSBwYXJlbnQuCkBAIC02NjksMzcgKzY2NSw0NCBAQCBmb3VuZDoKIAlpZiAodnAtPnZf ZmxhZyAmIFZfWEFUVFJESVIpCiAJCVZJX0xPQ0soZnAtPmdmc19wYXJlbnQpOwogI2VuZGlmCi0J VklfTE9DSyh2cCk7Ci0JLyoKLQkgKiBSZWFsbHkgcmVtb3ZlIHRoaXMgdm5vZGUKLQkgKi8KLQlk YXRhID0gdnAtPnZfZGF0YTsKLQlpZiAoZ2UgIT0gTlVMTCkgeworCWlmICh2cC0+dl9jb3VudCA9 PSAwIHx8IHZwLT52X2lmbGFnICYgVklfRE9PTUVEKSB7CiAJCS8qCi0JCSAqIElmIHRoaXMgd2Fz IGEgc3RhdGljYWxseSBjYWNoZWQgZW50cnksIHNpbXBseSBzZXQgdGhlCi0JCSAqIGNhY2hlZCB2 bm9kZSB0byBOVUxMLgorCQkgKiBSZWFsbHkgcmVtb3ZlIHRoaXMgdm5vZGUKIAkJICovCi0JCWdl LT5nZnNlX3Zub2RlID0gTlVMTDsKLQl9Ci0JVklfVU5MT0NLKHZwKTsKKwkJZGF0YSA9IHZwLT52 X2RhdGE7CisJCWlmIChnZSAhPSBOVUxMKSB7CisJCQkvKgorCQkJICogSWYgdGhpcyB3YXMgYSBz dGF0aWNhbGx5IGNhY2hlZCBlbnRyeSwgc2ltcGx5IHNldCB0aGUKKwkJCSAqIGNhY2hlZCB2bm9k ZSB0byBOVUxMLgorCQkJICovCisJCQlnZS0+Z2ZzZV92bm9kZSA9IE5VTEw7CisJCX0KKyNpZmRl ZiBUT0RPCisJCWlmICh2cC0+dl9mbGFnICYgVl9YQVRUUkRJUikKKwkJCVZJX1VOTE9DSyhmcC0+ Z2ZzX3BhcmVudCk7CisjZW5kaWYKIAotCS8qCi0JICogRnJlZSB2bm9kZSBhbmQgcmVsZWFzZSBw YXJlbnQKLQkgKi8KLQlpZiAoZnAtPmdmc19wYXJlbnQpIHsKLQkJaWYgKGRwKQotCQkJZ2ZzX2Rp cl91bmxvY2soZHApOwotCQlWT1BfVU5MT0NLKHZwLCAwKTsKLQkJVk5fUkVMRShmcC0+Z2ZzX3Bh cmVudCk7Ci0JCXZuX2xvY2sodnAsIExLX0VYQ0xVU0lWRSB8IExLX1JFVFJZKTsKKwkJLyoKKwkJ ICogRnJlZSB2bm9kZSBhbmQgcmVsZWFzZSBwYXJlbnQKKwkJICovCisJCWlmIChmcC0+Z2ZzX3Bh cmVudCkgeworCQkJaWYgKGRwKQorCQkJCWdmc19kaXJfdW5sb2NrKGRwKTsKKwkJCVZOX1JFTEUo ZnAtPmdmc19wYXJlbnQpOworCQl9IGVsc2UgeworCQkJQVNTRVJUKHZwLT52X3Zmc3AgIT0gTlVM TCk7CisJCQlWRlNfUkVMRSh2cC0+dl92ZnNwKTsKKwkJfQogCX0gZWxzZSB7Ci0JCUFTU0VSVCh2 cC0+dl92ZnNwICE9IE5VTEwpOwotCQlWRlNfUkVMRSh2cC0+dl92ZnNwKTsKLQl9CisJCWRhdGEg PSBOVUxMOwogI2lmZGVmIFRPRE8KLQlpZiAodnAtPnZfZmxhZyAmIFZfWEFUVFJESVIpCi0JCVZJ X1VOTE9DSyhmcC0+Z2ZzX3BhcmVudCk7CisJCWlmICh2cC0+dl9mbGFnICYgVl9YQVRUUkRJUikK KwkJCVZJX1VOTE9DSyhmcC0+Z2ZzX3BhcmVudCk7CiAjZW5kaWYKKwkJaWYgKGRwKQorCQkJZ2Zz X2Rpcl91bmxvY2soZHApOworCX0KKwogCXJldHVybiAoZGF0YSk7CiB9CiAKQEAgLTEyMzAsMTYg KzEyMzMsMTUgQEAgZ2ZzX3ZvcF9pbmFjdGl2ZShhcCkKIHsKIAl2bm9kZV90ICp2cCA9IGFwLT5h X3ZwOwogCWdmc19maWxlX3QgKmZwID0gdnAtPnZfZGF0YTsKKwl2b2lkICpkYXRhOwogCiAJaWYg KGZwLT5nZnNfdHlwZSA9PSBHRlNfRElSKQotCQlnZnNfZGlyX2luYWN0aXZlKHZwKTsKKwkJZGF0 YSA9IGdmc19kaXJfaW5hY3RpdmUodnApOwogCWVsc2UKLQkJZ2ZzX2ZpbGVfaW5hY3RpdmUodnAp OworCQlkYXRhID0gZ2ZzX2ZpbGVfaW5hY3RpdmUodnApOwogCi0JVklfTE9DSyh2cCk7Ci0JdnAt PnZfZGF0YSA9IE5VTEw7Ci0JVklfVU5MT0NLKHZwKTsKLQlrbWVtX2ZyZWUoZnAsIGZwLT5nZnNf c2l6ZSk7CisJaWYgKGRhdGEgIT0gTlVMTCkKKwkJa21lbV9mcmVlKGRhdGEsIGZwLT5nZnNfc2l6 ZSk7CiAKIAlyZXR1cm4gKDApOwogfQpkaWZmIC0tZ2l0IGEvc3lzL2NkZGwvY29udHJpYi9vcGVu c29sYXJpcy91dHMvY29tbW9uL2ZzL3pmcy96ZnNfY3RsZGlyLmMgYi9zeXMvY2RkbC9jb250cmli L29wZW5zb2xhcmlzL3V0cy9jb21tb24vZnMvemZzL3pmc19jdGxkaXIuYwppbmRleCAzY2M3MGIy Li4yZDNiZTgzIDEwMDY0NAotLS0gYS9zeXMvY2RkbC9jb250cmliL29wZW5zb2xhcmlzL3V0cy9j b21tb24vZnMvemZzL3pmc19jdGxkaXIuYworKysgYi9zeXMvY2RkbC9jb250cmliL29wZW5zb2xh cmlzL3V0cy9jb21tb24vZnMvemZzL3pmc19jdGxkaXIuYwpAQCAtNjEyLDcgKzYxMiw3IEBAIHpm c2N0bF9mcmVlYnNkX3Jvb3RfbG9va3VwKGFwKQogCiAJZXJyID0gemZzY3RsX3Jvb3RfbG9va3Vw KGR2cCwgbm0sIHZwcCwgTlVMTCwgMCwgTlVMTCwgY3IsIE5VTEwsIE5VTEwsIE5VTEwpOwogCWlm IChlcnIgPT0gMCAmJiAobm1bMF0gIT0gJy4nIHx8IG5tWzFdICE9ICdcMCcpKQotCQl2bl9sb2Nr KCp2cHAsIExLX0VYQ0xVU0lWRSB8IExLX1JFVFJZKTsKKwkJZXJyID0gdm5fbG9jaygqdnBwLCBh cC0+YV9jbnAtPmNuX2xrZmxhZ3MpOwogCXJldHVybiAoZXJyKTsKIH0KIApAQCAtOTc1LDggKzk3 NSwxMSBAQCB6ZnNjdGxfc25hcGRpcl9sb29rdXAoYXApCiAJWkZTX0VOVEVSKHpmc3Zmcyk7CiAK IAlpZiAoZ2ZzX2xvb2t1cF9kb3QodnBwLCBkdnAsIHpmc3Zmcy0+el9jdGxkaXIsIG5tKSA9PSAw KSB7CisJCWVyciA9IDA7CisJCWlmIChubVswXSAhPSAnLicgfHwgbm1bMV0gIT0gJ1wwJykKKwkJ CWVyciA9IHZuX2xvY2soKnZwcCwgYXAtPmFfY25wLT5jbl9sa2ZsYWdzKTsKIAkJWkZTX0VYSVQo emZzdmZzKTsKLQkJcmV0dXJuICgwKTsKKwkJcmV0dXJuIChlcnIpOwogCX0KIAogCWlmIChmbGFn cyAmIEZJR05PUkVDQVNFKSB7CkBAIC0xMDA0LDcgKzEwMDcsNyBAQCB6ZnNjdGxfc25hcGRpcl9s b29rdXAoYXApCiAJaWYgKChzZXAgPSBhdmxfZmluZCgmc2RwLT5zZF9zbmFwcywgJnNlYXJjaCwg JndoZXJlKSkgIT0gTlVMTCkgewogCQkqdnBwID0gc2VwLT5zZV9yb290OwogCQlWTl9IT0xEKCp2 cHApOwotCQllcnIgPSB0cmF2ZXJzZSh2cHAsIExLX0VYQ0xVU0lWRSB8IExLX1JFVFJZKTsKKwkJ ZXJyID0gdHJhdmVyc2UodnBwLCBhcC0+YV9jbnAtPmNuX2xrZmxhZ3MpOwogCQlpZiAoZXJyICE9 IDApIHsKIAkJCVZOX1JFTEUoKnZwcCk7CiAJCQkqdnBwID0gTlVMTDsKQEAgLTEwMTMsNiArMTAx Niw3IEBAIHpmc2N0bF9zbmFwZGlyX2xvb2t1cChhcCkKIAkJCSAqIFRoZSBzbmFwc2hvdCB3YXMg dW5tb3VudGVkIGJlaGluZCBvdXIgYmFja3MsCiAJCQkgKiB0cnkgdG8gcmVtb3VudCBpdC4KIAkJ CSAqLworCQkJVk9QX1VOTE9DSygqdnBwLCAwKTsKIAkJCVZFUklGWSh6ZnNjdGxfc25hcHNob3Rf em5hbWUoZHZwLCBubSwgTUFYTkFNRUxFTiwgc25hcG5hbWUpID09IDApOwogCQkJZ290byBkb21v dW50OwogCQl9IGVsc2UgewpAQCAtMTA2NCw3ICsxMDY4LDYgQEAgemZzY3RsX3NuYXBkaXJfbG9v a3VwKGFwKQogCXNlcC0+c2VfbmFtZSA9IGttZW1fYWxsb2Moc3RybGVuKG5tKSArIDEsIEtNX1NM RUVQKTsKIAkodm9pZCkgc3RyY3B5KHNlcC0+c2VfbmFtZSwgbm0pOwogCSp2cHAgPSBzZXAtPnNl X3Jvb3QgPSB6ZnNjdGxfc25hcHNob3RfbWtub2RlKGR2cCwgZG11X29ianNldF9pZChzbmFwKSk7 Ci0JVk5fSE9MRCgqdnBwKTsKIAlhdmxfaW5zZXJ0KCZzZHAtPnNkX3NuYXBzLCBzZXAsIHdoZXJl KTsKIAogCWRtdV9vYmpzZXRfcmVsZShzbmFwLCBGVEFHKTsKQEAgLTEwOTEsNyArMTA5NCw2IEBA IGRvbW91bnQ6CiAJbXV0ZXhfZXhpdCgmc2RwLT5zZF9sb2NrKTsKIAlaRlNfRVhJVCh6ZnN2ZnMp OwogCi0jaWZkZWYgaWxsdW1vcwogCS8qCiAJICogSWYgd2UgaGFkIGFuIGVycm9yLCBkcm9wIG91 ciBob2xkIG9uIHRoZSB2bm9kZSBhbmQKIAkgKiB6ZnNjdGxfc25hcHNob3RfaW5hY3RpdmUoKSB3 aWxsIGNsZWFuIHVwLgpAQCAtMTEwMCwxMCArMTEwMiw2IEBAIGRvbW91bnQ6CiAJCVZOX1JFTEUo KnZwcCk7CiAJCSp2cHAgPSBOVUxMOwogCX0KLSNlbHNlCi0JaWYgKGVyciAhPSAwKQotCQkqdnBw ID0gTlVMTDsKLSNlbmRpZgogCXJldHVybiAoZXJyKTsKIH0KIApAQCAtMTEzMCw4ICsxMTI4LDEx IEBAIHpmc2N0bF9zaGFyZXNfbG9va3VwKGFwKQogCXN0cmxjcHkobm0sIGNucC0+Y25fbmFtZXB0 ciwgY25wLT5jbl9uYW1lbGVuICsgMSk7CiAKIAlpZiAoZ2ZzX2xvb2t1cF9kb3QodnBwLCBkdnAs IHpmc3Zmcy0+el9jdGxkaXIsIG5tKSA9PSAwKSB7CisJCWVycm9yID0gMDsKKwkJaWYgKG5tWzBd ICE9ICcuJyB8fCBubVsxXSAhPSAnXDAnKQorCQkJZXJyb3IgPSB2bl9sb2NrKCp2cHAsIGFwLT5h X2NucC0+Y25fbGtmbGFncyk7CiAJCVpGU19FWElUKHpmc3Zmcyk7Ci0JCXJldHVybiAoMCk7CisJ CXJldHVybiAoZXJyb3IpOwogCX0KIAogCWlmICh6ZnN2ZnMtPnpfc2hhcmVzX2RpciA9PSAwKSB7 CkBAIC0xMzQ0LDIyICsxMzQ1LDE1IEBAIHpmc2N0bF9zbmFwZGlyX2luYWN0aXZlKGFwKQogCXZu b2RlX3QgKnZwID0gYXAtPmFfdnA7CiAJemZzY3RsX3NuYXBkaXJfdCAqc2RwID0gdnAtPnZfZGF0 YTsKIAl6ZnNfc25hcGVudHJ5X3QgKnNlcDsKLQotCS8qCi0JICogT24gZm9yY2VkIHVubW91bnQg d2UgaGF2ZSB0byBmcmVlIHNuYXBzaG90cyBmcm9tIGhlcmUuCi0JICovCi0JbXV0ZXhfZW50ZXIo JnNkcC0+c2RfbG9jayk7Ci0Jd2hpbGUgKChzZXAgPSBhdmxfZmlyc3QoJnNkcC0+c2Rfc25hcHMp KSAhPSBOVUxMKSB7Ci0JCWF2bF9yZW1vdmUoJnNkcC0+c2Rfc25hcHMsIHNlcCk7Ci0JCWttZW1f ZnJlZShzZXAtPnNlX25hbWUsIHN0cmxlbihzZXAtPnNlX25hbWUpICsgMSk7Ci0JCWttZW1fZnJl ZShzZXAsIHNpemVvZiAoemZzX3NuYXBlbnRyeV90KSk7CisJdm9pZCAqcHJpdmF0ZTsKKworCXBy aXZhdGUgPSBnZnNfZGlyX2luYWN0aXZlKHZwKTsKKwlpZiAocHJpdmF0ZSAhPSBOVUxMKSB7CisJ CUFTU0VSVChhdmxfbnVtbm9kZXMoJnNkcC0+c2Rfc25hcHMpID09IDApOworCQltdXRleF9kZXN0 cm95KCZzZHAtPnNkX2xvY2spOworCQlhdmxfZGVzdHJveSgmc2RwLT5zZF9zbmFwcyk7CisJCWtt ZW1fZnJlZShwcml2YXRlLCBzaXplb2YgKHpmc2N0bF9zbmFwZGlyX3QpKTsKIAl9Ci0JbXV0ZXhf ZXhpdCgmc2RwLT5zZF9sb2NrKTsKLQlnZnNfZGlyX2luYWN0aXZlKHZwKTsKLQlBU1NFUlQoYXZs X251bW5vZGVzKCZzZHAtPnNkX3NuYXBzKSA9PSAwKTsKLQltdXRleF9kZXN0cm95KCZzZHAtPnNk X2xvY2spOwotCWF2bF9kZXN0cm95KCZzZHAtPnNkX3NuYXBzKTsKLQlrbWVtX2ZyZWUoc2RwLCBz aXplb2YgKHpmc2N0bF9zbmFwZGlyX3QpKTsKIAogCXJldHVybiAoMCk7CiB9CkBAIC0xNDQxLDcg KzE0MzUsNiBAQCB6ZnNjdGxfc25hcHNob3RfbWtub2RlKHZub2RlX3QgKnB2cCwgdWludDY0X3Qg b2Jqc2V0KQogCiAJdnAgPSBnZnNfZGlyX2NyZWF0ZShzaXplb2YgKHpmc2N0bF9ub2RlX3QpLCBw dnAsIHB2cC0+dl92ZnNwLAogCSAgICAmemZzY3RsX29wc19zbmFwc2hvdCwgTlVMTCwgTlVMTCwg TUFYTkFNRUxFTiwgTlVMTCwgTlVMTCk7Ci0JVk5fSE9MRCh2cCk7CiAJemNwID0gdnAtPnZfZGF0 YTsKIAl6Y3AtPnpjX2lkID0gb2Jqc2V0OwogCVZPUF9VTkxPQ0sodnAsIDApOwpAQCAtMTQ2Miwx OCArMTQ1NSwyMyBAQCB6ZnNjdGxfc25hcHNob3RfaW5hY3RpdmUoYXApCiAJemZzY3RsX3NuYXBk aXJfdCAqc2RwOwogCXpmc19zbmFwZW50cnlfdCAqc2VwLCAqbmV4dDsKIAlpbnQgbG9ja2VkOwot CXZub2RlX3QgKmR2cDsKKwlnZnNfZGlyX3QgKmRwID0gdnAtPnZfZGF0YTsKKwl2bm9kZV90ICpk dnAgPSBkcC0+Z2ZzZF9maWxlLmdmc19wYXJlbnQ7CiAKLQlpZiAodnAtPnZfY291bnQgPiAwKQot CQlnb3RvIGVuZDsKLQotCVZFUklGWShnZnNfZGlyX2xvb2t1cCh2cCwgIi4uIiwgJmR2cCwgY3Is IDAsIE5VTEwsIE5VTEwpID09IDApOworCVZPUF9VTkxPQ0sodnAsIDApOwogCXNkcCA9IGR2cC0+ dl9kYXRhOwotCVZPUF9VTkxPQ0soZHZwLCAwKTsKIAogCWlmICghKGxvY2tlZCA9IE1VVEVYX0hF TEQoJnNkcC0+c2RfbG9jaykpKQogCQltdXRleF9lbnRlcigmc2RwLT5zZF9sb2NrKTsKIAorCXZu X2xvY2sodnAsIExLX0VYQ0xVU0lWRSB8IExLX1JFVFJZKTsKKworCWlmICh2cC0+dl9jb3VudCA+ IDAgJiYgISh2cC0+dl9pZmxhZyAmIFZJX0RPT01FRCkpIHsKKwkJaWYgKCFsb2NrZWQpCisJCQlt dXRleF9leGl0KCZzZHAtPnNkX2xvY2spOworCQlyZXR1cm4oMCk7CisJfQorCiAJQVNTRVJUKCF2 bl9pc21udHB0KHZwKSk7CiAKIAlzZXAgPSBhdmxfZmlyc3QoJnNkcC0+c2Rfc25hcHMpOwpAQCAt MTQ5Miw5ICsxNDkwLDcgQEAgemZzY3RsX3NuYXBzaG90X2luYWN0aXZlKGFwKQogCiAJaWYgKCFs b2NrZWQpCiAJCW11dGV4X2V4aXQoJnNkcC0+c2RfbG9jayk7Ci0JVk5fUkVMRShkdnApOwogCi1l bmQ6CiAJLyoKIAkgKiBEaXNwb3NlIG9mIHRoZSB2bm9kZSBmb3IgdGhlIHNuYXBzaG90IG1vdW50 IHBvaW50LgogCSAqIFRoaXMgaXMgc2FmZSB0byBkbyBiZWNhdXNlIG9uY2UgdGhpcyBlbnRyeSBo YXMgYmVlbiByZW1vdmVkCkBAIC0xNTg4LDcgKzE1ODQsNyBAQCB6ZnNjdGxfc25hcHNob3RfbG9v a3VwKGFwKQogCWVycm9yID0gemZzY3RsX3Jvb3RfbG9va3VwKHpmc3Zmcy0+el9jdGxkaXIsICJz bmFwc2hvdCIsIHZwcCwKIAkgICAgTlVMTCwgMCwgTlVMTCwgY3IsIE5VTEwsIE5VTEwsIE5VTEwp OwogCWlmIChlcnJvciA9PSAwKQotCQl2bl9sb2NrKCp2cHAsIExLX0VYQ0xVU0lWRSB8IExLX1JF VFJZKTsKKwkJZXJyb3IgPSB2bl9sb2NrKCp2cHAsIGFwLT5hX2NucC0+Y25fbGtmbGFncyk7CiAJ cmV0dXJuIChlcnJvcik7CiB9CiAKZGlmZiAtLWdpdCBhL3N5cy9jZGRsL2NvbnRyaWIvb3BlbnNv bGFyaXMvdXRzL2NvbW1vbi9mcy96ZnMvemZzX3Zmc29wcy5jIGIvc3lzL2NkZGwvY29udHJpYi9v cGVuc29sYXJpcy91dHMvY29tbW9uL2ZzL3pmcy96ZnNfdmZzb3BzLmMKaW5kZXggNzEwNmQwOS4u OTY5NDRkMiAxMDA2NDQKLS0tIGEvc3lzL2NkZGwvY29udHJpYi9vcGVuc29sYXJpcy91dHMvY29t bW9uL2ZzL3pmcy96ZnNfdmZzb3BzLmMKKysrIGIvc3lzL2NkZGwvY29udHJpYi9vcGVuc29sYXJp cy91dHMvY29tbW9uL2ZzL3pmcy96ZnNfdmZzb3BzLmMKQEAgLTIwMzcsMTIgKzIwMzcsNyBAQCB6 ZnNfdW1vdW50KHZmc190ICp2ZnNwLCBpbnQgZmZsYWcpCiAJICovCiAJaWYgKHpmc3Zmcy0+el9j dGxkaXIgIT0gTlVMTCkKIAkJemZzY3RsX2Rlc3Ryb3koemZzdmZzKTsKLQlpZiAoemZzdmZzLT56 X2lzc25hcCkgewotCQl2bm9kZV90ICpzdnAgPSB2ZnNwLT5tbnRfdm5vZGVjb3ZlcmVkOwogCi0J CWlmIChzdnAtPnZfY291bnQgPj0gMikKLQkJCVZOX1JFTEUoc3ZwKTsKLQl9CiAJemZzX2ZyZWV2 ZnModmZzcCk7CiAKIAlyZXR1cm4gKDApOwpkaWZmIC0tZ2l0IGEvc3lzL2tlcm4vdmZzX3N1YnIu YyBiL3N5cy9rZXJuL3Zmc19zdWJyLmMKaW5kZXggZjc2MTMzNC4uZGNlNjAyNCAxMDA2NDQKLS0t IGEvc3lzL2tlcm4vdmZzX3N1YnIuYworKysgYi9zeXMva2Vybi92ZnNfc3Vici5jCkBAIC0xMTg3 LDcgKzExODcsNyBAQCBpbnNtbnRxdWUxKHN0cnVjdCB2bm9kZSAqdnAsIHN0cnVjdCBtb3VudCAq bXAsCiAJfQogCXZwLT52X21vdW50ID0gbXA7CiAJTU5UX1JFRihtcCk7Ci0JVEFJTFFfSU5TRVJU X1RBSUwoJm1wLT5tbnRfbnZub2RlbGlzdCwgdnAsIHZfbm1udHZub2Rlcyk7CisJVEFJTFFfSU5T RVJUX0hFQUQoJm1wLT5tbnRfbnZub2RlbGlzdCwgdnAsIHZfbm1udHZub2Rlcyk7CiAJVk5BU1NF UlQobXAtPm1udF9udm5vZGVsaXN0c2l6ZSA+PSAwLCB2cCwKIAkJKCJuZWcgbW91bnQgcG9pbnQg dm5vZGUgbGlzdCBzaXplIikpOwogCW1wLT5tbnRfbnZub2RlbGlzdHNpemUrKzsK --=_158609ab4d8429e04dff41c787e8eb7a-- From owner-zfs-devel@FreeBSD.ORG Tue Feb 4 01:34:30 2014 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 927525F7 for ; Tue, 4 Feb 2014 01:34:30 +0000 (UTC) Received: from mail-ig0-x22b.google.com (mail-ig0-x22b.google.com [IPv6:2607:f8b0:4001:c05::22b]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 544241D61 for ; Tue, 4 Feb 2014 01:34:30 +0000 (UTC) Received: by mail-ig0-f171.google.com with SMTP id uy17so5882689igb.4 for ; Mon, 03 Feb 2014 17:34:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=dataix.net; s=rsa; h=date:from:to:subject:message-id:reply-to:mime-version:content-type :content-disposition; bh=wlnl3n2ZvMi60NEbbzjOohkx8HinYub8EkXCWtfh1bo=; b=UKNuPsFB2HRWj7MJS0RRUaC4WjWtzAvTEv5cvQn6PwrOCV0B1xZLIKZdqwhQhhPOLY sVVg/DblyeINxQbMgITIOLlwXo5oNjDH3qgQnxVBZMhMX19vPYTtfHRbM39MrBWL2/y6 NPoSkA1NxBicwFJAeGiJBKpz4lH83FRqagLpo= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:date:from:to:subject:message-id:reply-to :mime-version:content-type:content-disposition; bh=wlnl3n2ZvMi60NEbbzjOohkx8HinYub8EkXCWtfh1bo=; b=GBbQE4HxdtPVMuBKMT3/6iucTKVUhSe+VexlwL2f2Zv1D6mfpEdUeA5LhmBgzPykp6 GC+WpeQvkStXbGrKMdyC8JIx1zxNAf6HCp7szIQNlCFUGsN6Z21/Ph/MmIx0da240QmN AZ5PLU+PD6TQhPF8QlfCAzKKCK99GQRG4SM6S6bPdnp9StN7yowtwRmuGYY98fC6Hfd1 uuagRd0nrcYSyu+X2784ombsG1rA57eARlEW6XYj6azhMHnTtfdm6PDvbiCRPyUVkQWA 5iMfy2CdTDqSqJuk1faaekaFsvz77NcIBnUHXej4A332aSyOR3sWFNV5U4xdpHq/Npdt 91QA== X-Gm-Message-State: ALoCoQkeO1ZpRSu73hDqQELiKoA0BTLoyRKEVg9FagqrIGxvDylV1XHsPDiX57YeKal6biBM7/7A X-Received: by 10.50.61.101 with SMTP id o5mr15084851igr.31.1391477669667; Mon, 03 Feb 2014 17:34:29 -0800 (PST) Received: from DataIX.local (75-128-101-59.dhcp.sgnw.mi.charter.com. [75.128.101.59]) by mx.google.com with ESMTPSA id f15sm35930175igd.3.2014.02.03.17.34.28 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 03 Feb 2014 17:34:28 -0800 (PST) Received: from DataIX.local (localhost [127.0.0.1]) by DataIX.local (8.14.7/8.14.7) with ESMTP id s141YQCE019288 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Mon, 3 Feb 2014 20:34:27 -0500 (EST) (envelope-from jhellenthal@DataIX.net) Received: (from jh@localhost) by DataIX.local (8.14.7/8.14.7/Submit) id s141YQD4019234; Mon, 3 Feb 2014 20:34:26 -0500 (EST) (envelope-from jhellenthal@DataIX.net) Date: Mon, 3 Feb 2014 20:34:25 -0500 From: jhellenthal@dataix.net To: zfs-devel@freebsd.org, stable@freebsd.org Subject: WITHOUT_CDDL leftovers (libzfs_core.so.2, zstreamdump) Message-ID: <20140204013425.GA98236@DataIX.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list Reply-To: zfs-devel@freebsd.org List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Feb 2014 01:34:30 -0000 It appears that a couple things were left out of the 'delete-old*' targets and left behind from being removed in 10-STABLE at least on powerpc but likely elsewhere. Could someone take a moment to go through this when they get a chance ? Thanks Unresolvable link(s) found in: /lib/libzfs_core.so.2 libnvpair.so.2 Unresolvable link(s) found in: /usr/bin/zstreamdump libnvpair.so.2 libumem.so.2 libzpool.so.2 libavl.so.2 -- - (2^(N-1)) JJH48-ARIN From owner-zfs-devel@FreeBSD.ORG Tue Feb 4 01:40:12 2014 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3F9CD736 for ; Tue, 4 Feb 2014 01:40:12 +0000 (UTC) Received: from mail-ig0-x232.google.com (mail-ig0-x232.google.com [IPv6:2607:f8b0:4001:c05::232]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 003E41DE7 for ; Tue, 4 Feb 2014 01:40:11 +0000 (UTC) Received: by mail-ig0-f178.google.com with SMTP id uq10so5776729igb.5 for ; Mon, 03 Feb 2014 17:40:11 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=dataix.net; s=rsa; h=date:from:to:subject:message-id:references:mime-version :content-type:content-disposition:in-reply-to; bh=VSyUfMhLgY6ePkHT4ZfJblO92cgKgfLIKpE7UKBgXh0=; b=cxQqD90P1sBusAHoecRmq/7y0h1sFflpBVyoVxYj04WLbYo5XgU1KbEX0h/82Jzu/N iFfJN8kvS3gxdcOMXR9jGdkivZ/t1OK/r22Tz2McrRcgK3M/3aUw9ydgj+iLQdjwzGGR mDoCz1gI5qS/ysXQq8bJYD4tbMGX3D4SDIuJQ= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:date:from:to:subject:message-id:references :mime-version:content-type:content-disposition:in-reply-to; bh=VSyUfMhLgY6ePkHT4ZfJblO92cgKgfLIKpE7UKBgXh0=; b=eNIHQpxRCLaxyA3fzp3pMBGk8mE0r5cliWyH3nQs6+vTtpicAdMH5Zz+G594c2OaNU fSrP+U2cmnld4D0MxoauQ+y9+opRdG5B/FgWB9+nWIW8XH4mBUsgTP3uVe9HgkMJ6HIl stLVdVaqwepei3NIS9zDo+lZWt7oO/CDnqskk85kKHZggA2hqo6HFyWIh36TmBJal2QH xAcxbImskqF4QVhXA0km6u0MQJZ50V8UhxvfhYb+Xix07Ktcc7iz5I9kWKSgem9CwKpA 3+bb41TtWtB8C6zD7R/jtePpwWwayrhQZ/IH2n4jqyYBBV7R/Vu60AUgIbON8sSS7Rwc uYXg== X-Gm-Message-State: ALoCoQmunY/9HZH4sCH01+qsiw/2VhLJMsiZdNvzt5CbEnpYYwcgq8CXfrdWmT2pO3NCX0DzQzGP X-Received: by 10.50.230.20 with SMTP id su20mr15225133igc.18.1391478011443; Mon, 03 Feb 2014 17:40:11 -0800 (PST) Received: from DataIX.local (75-128-101-59.dhcp.sgnw.mi.charter.com. [75.128.101.59]) by mx.google.com with ESMTPSA id ml2sm35923826igb.10.2014.02.03.17.40.10 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 03 Feb 2014 17:40:10 -0800 (PST) Received: from DataIX.local (localhost [127.0.0.1]) by DataIX.local (8.14.7/8.14.7) with ESMTP id s141e8Z5023068 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Mon, 3 Feb 2014 20:40:08 -0500 (EST) (envelope-from jhellenthal@DataIX.net) Received: (from jh@localhost) by DataIX.local (8.14.7/8.14.7/Submit) id s141e7ar023067; Mon, 3 Feb 2014 20:40:07 -0500 (EST) (envelope-from jhellenthal@DataIX.net) Date: Mon, 3 Feb 2014 20:40:07 -0500 From: jhellenthal@dataix.net To: zfs-devel@freebsd.org, stable@freebsd.org Subject: Re: WITHOUT_CDDL leftovers (libzfs_core.so.2, zstreamdump) Message-ID: <20140204014007.GA22901@DataIX.net> References: <20140204013425.GA98236@DataIX.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20140204013425.GA98236@DataIX.net> X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Feb 2014 01:40:12 -0000 On Mon, Feb 03, 2014 at 08:34:25PM -0500, jhellenthal@DataIX.net wrote: > > It appears that a couple things were left out of the 'delete-old*' targets and left behind from being removed in 10-STABLE at least on powerpc but likely elsewhere. > > Could someone take a moment to go through this when they get a chance ? > > Thanks > > Unresolvable link(s) found in: /lib/libzfs_core.so.2 > libnvpair.so.2 > Unresolvable link(s) found in: /usr/bin/zstreamdump > libnvpair.so.2 > libumem.so.2 > libzpool.so.2 > libavl.so.2 One more Unresolvable link(s) found in: /usr/sbin/zhack^M libnvpair.so.2^M libumem.so.2^M libuutil.so.2^M libzfs.so.2^M libzpool.so.2 > > -- > > - (2^(N-1)) JJH48-ARIN > -- - (2^(N-1)) JJH48-ARIN From owner-zfs-devel@FreeBSD.ORG Sat Feb 8 20:32:48 2014 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id EAFF932A for ; Sat, 8 Feb 2014 20:32:48 +0000 (UTC) Received: from mailA.getsnappy.com (mailA.getsnappy.com [72.29.186.40]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id CB34D1C77 for ; Sat, 8 Feb 2014 20:32:48 +0000 (UTC) Received: from [192.168.0.2] (24-205-237-111.dhcp.snlo.ca.charter.com [24.205.237.111]) (authenticated bits=0) by mailA.getsnappy.com (8.14.7/8.14.3) with ESMTP id s18K27ge007928 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NO) for ; Sat, 8 Feb 2014 12:02:08 -0800 (PST) (envelope-from brian@timestudybuddy.com) X-Authentication-Warning: mailA.getsnappy.com: Host 24-205-237-111.dhcp.snlo.ca.charter.com [24.205.237.111] claimed to be [192.168.0.2] From: Brian Gardner Subject: Outage related to hard drive failure Message-Id: <1BBDE50F-DB02-4E67-83C3-B98946CABA2B@timestudybuddy.com> Date: Sat, 8 Feb 2014 12:02:07 -0800 To: zfs-devel@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 7.1 \(1827\)) X-Mailer: Apple Mail (2.1827) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.17 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 08 Feb 2014 20:32:49 -0000 Last year upgraded my production servers from ufs on adaptec raid = controllers to zfs raidz on lsi controllers. Last week I had an outage, = the culprit being a failed hard in my raidz. My log was littered with = kernel messages such as the ones below. About 20 minutes after the = first message some aspects of my host where hung, and I was notified = that my site was down. I noticed these messages in my log but running a = zfs status however showed only a few checksum errors and the failed = drive didn't get degraded. Manually degrading the drive solved my = problems. This seems very odd to me. It's almost as if zfs wasn't = getting messages from the lsi driver regarding these read/write = failures. Do I need to tune something in the mps/lsi or zfs drivers to = help it deal with failures? I'm running FreeBSD 8.3-Release-p3 with the = generic kernel, nothing unusual about my setup other than I use jails = extensively. There are four drives in the raidz configuration in = question: zpool status pool: storage state: ONLINE scan: resilvered 67.6G in 2h21m with 0 errors on Tue Aug 6 00:44:14 = 2013 config: NAME STATE READ WRITE CKSUM storage ONLINE 0 0 0 raidz1-0 ONLINE 0 0 0 da0 ONLINE 0 0 0 da1 ONLINE 0 0 0 da2 ONLINE 0 0 0 da3 ONLINE 0 0 0 Jan 29 19:03:18 host2 kernel: (da0:mpslsi0:0:0:0): WRITE(10). CDB: 2a 0 = 9 72 d8 13 0 0 1d 0=20 Jan 29 19:03:18 host2 kernel: (da0:mpslsi0:0:0:0): CAM status: SCSI = Status Error Jan 29 19:03:18 host2 kernel: (da0:mpslsi0:0:0:0): SCSI status: Check = Condition Jan 29 19:03:18 host2 kernel: (da0:mpslsi0:0:0:0): SCSI sense: Deferred = error: MEDIUM ERROR info:97ef13b asc:15,1 (Mechanical positioning error) = actual retry count: 15 Jan 29 19:04:00 host2 kernel: (da0:mpslsi0:0:0:0): READ(10). CDB: 28 0 9 = 75 b9 96 0 0 b 0=20 Jan 29 19:04:00 host2 kernel: (da0:mpslsi0:0:0:0): CAM status: SCSI = Status Error Jan 29 19:04:00 host2 kernel: (da0:mpslsi0:0:0:0): SCSI status: Check = Condition Jan 29 19:04:00 host2 kernel: (da0:mpslsi0:0:0:0): SCSI sense: MEDIUM = ERROR info:975b99b asc:15,1 (Mechanical positioning error) actual retry = count: 15 Jan 29 19:04:54 host2 kernel: mpslsi0: mpssas_scsiio_timeout checking sc = 0xffffff80005ee000 cm 0xffffff800060d408 Jan 29 19:04:54 host2 kernel: (da0:mpslsi0:0:0:0): READ(10). CDB: 28 0 2 = e3 6a 1a 0 0 1 0 length 512 SMID 153 command timeout cm = 0xffffff800060d408 ccb 0xffffff000a1ce800 Jan 29 19:04:54 host2 kernel: mpslsi0: mpssas_alloc_tm freezing simq Jan 29 19:04:54 host2 kernel: mpslsi0: timedout cm 0xffffff800060d408 = allocated tm 0xffffff8000604718 Jan 29 19:04:54 host2 kernel: (da0:mpslsi0:0:0:0): READ(10). CDB: 28 0 2 = e3 6a 1a 0 0 1 0 length 512 SMID 153 completed timedout cm = 0xffffff800060d408 ccb 0xffffff000a1ce800 during recovery ioc 8048 scsi = 0 state c xfer 0 Jan 29 19:04:54 host2 kernel: (noperiph:mpslsi0:0:0:0): SMID 43 abort = TaskMID 153 status 0x0 code 0x0 count 1 Jan 29 19:04:54 host2 kernel: (noperiph:mpslsi0:0:0:0): SMID 43 finished = recovery after aborting TaskMID 153 Jan 29 19:04:54 host2 kernel: mpslsi0: mpssas_free_tm releasing simq Jan 29 19:04:54 host2 kernel: mpslsi0: mpssas_scsiio_timeout checking sc = 0xffffff80005ee000 cm 0xffffff800060d928 Jan 29 19:04:54 host2 kernel: (da0:mpslsi0:0:0:0): READ(10). CDB: 28 0 9 = 41 c7 53 0 0 b 0 length 5632 SMID 157 command timeout cm = 0xffffff800060d928 ccb 0xffffff0176ccb000 Jan 29 19:04:54 host2 kernel: mpslsi0: mpssas_alloc_tm freezing simq Jan 29 19:04:54 host2 kernel: mpslsi0: timedout cm 0xffffff800060d928 = allocated tm 0xffffff8000604860 Jan 29 19:04:54 host2 kernel: (da0:mpslsi0:0:0:0): READ(10). CDB: 28 0 9 = 41 c7 53 0 0 b 0 length 5632 SMID 157 completed timedout cm = 0xffffff800060d928 ccb 0xffffff0176ccb000 during recovery ioc 8048 scsi = 0 state c xfer 0(noperiph:mpslsi0:0:0:0): SMID 44 abort TaskMID 157 = status 0x0 code 0x0 count 1 Jan 29 19:04:54 host2 kernel: (noperiph:mpslsi0:0:0:0): SMID 44 finished = recovery after aborting TaskMID 157 Jan 29 19:04:54 host2 kernel: mpslsi0: mpssas_free_tm releasing simq Jan 29 19:04:54 host2 kernel: mpslsi0: mpssas_scsiio_timeout checking sc = 0xffffff80005ee000 cm 0xffffff80006235a8 Jan 29 19:04:54 host2 kernel: (da0:mpslsi0:0:0:0): WRITE(10). CDB: 2a 0 = 9 72 e0 d3 0 1 0 0 length 131072 SMID 429 command timeout cm = 0xffffff80006235a8 ccb 0xffffff000c600000 Jan 29 19:04:54 host2 kernel: mpslsi0: mpssas_alloc_tm freezing simq Jan 29 19:04:54 host2 kernel: mpslsi0: timedout cm 0xffffff80006235a8 = allocated tm 0xffffff80006049a8 Jan 29 19:04:54 host2 kernel: mpslsi0: mpssas_scsiio_timeout checking sc = 0xffffff80005ee000 cm 0xffffff800063e2e0 Jan 29 19:04:54 host2 kernel: (da0:mpslsi0:0:0:0): WRITE(10). CDB: 2a 0 = 9 72 e1 d3 0 0 3 0 length 1536 SMID 764 command timeout cm = 0xffffff800063e2e0 ccb 0xffffff01dfa19800 Jan 29 19:04:54 host2 kernel: mpslsi0: queued timedout cm = 0xffffff800063e2e0 for processing by tm 0xffffff80006049a8 Jan 29 19:04:54 host2 kernel: (da0:mpslsi0:0:0:0): WRITE(10). CDB: 2a 0 = 9 72 e0 d3 0 1 0 0 length 131072 SMID 429 completed timedout cm = 0xffffff80006235a8 ccb 0xffffff000c600000 during recovery ioc 8048 scsi = 0 state c xfe(noperiph:mpslsi0:0:0:0): SMID 45 abort TaskMID 429 status = 0x0 code 0x0 count 1 Jan 29 19:04:54 host2 kernel: (noperiph:mpslsi0:0:0:0): SMID 45 = continuing recovery after aborting TaskMID 429 Jan 29 19:04:54 host2 kernel: mpslsi0: mpssas_scsiio_timeout checking sc = 0xffffff80005ee000 cm 0xffffff8000610890 Jan 29 19:04:54 host2 kernel: (da0:mpslsi0:0:0:0): WRITE(10). CDB: 2a 0 = 9 72 e3 31 0 0 be 0 length 97280 SMID 194 command timeout cm = 0xffffff8000610890 ccb 0xffffff013d0f3800 Jan 29 19:04:54 host2 kernel: mpslsi0: queued timedout cm = 0xffffff8000610890 for processing by tm 0xffffff80006049a8 Jan 29 19:04:54 host2 kernel: (da0:mpslsi0:0:0:0): WRITE(10). CDB: 2a 0 = 9 72 e1 d3 0 0 3 0 length 1536 SMID 764 completed timedout cm = 0xffffff800063e2e0 ccb 0xffffff01dfa19800 during recovery ioc 8048 scsi = 0 state c xfer (noperiph:mpslsi0:0:0:0): SMID 45 abort TaskMID 764 = status 0x0 code 0x0 count 1= From owner-zfs-devel@FreeBSD.ORG Sat Feb 8 21:28:24 2014 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 41968205; Sat, 8 Feb 2014 21:28:24 +0000 (UTC) Received: from pi.nmdps.net (pi.nmdps.net [109.61.102.5]) by mx1.freebsd.org (Postfix) with ESMTP id CB24E10D6; Sat, 8 Feb 2014 21:28:23 +0000 (UTC) Received: from pi.nmdps.net (localhost [127.0.0.1]) (Authenticated sender: krichy@cflinux.hu) by pi.nmdps.net (Postfix) with ESMTPSA id 03B7115E6; Sat, 8 Feb 2014 22:28:15 +0100 (CET) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Date: Sat, 08 Feb 2014 22:28:13 +0100 From: krichy@cflinux.hu To: zfs-devel@freebsd.org, freebsd-scsi@freebsd.org Subject: Re: Outage related to hard drive failure Message-ID: X-Sender: krichy@cflinux.hu User-Agent: Roundcube Webmail/0.9.5 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 08 Feb 2014 21:28:24 -0000 Dear Brian, Unfortunately I just can report about same issue I ran into a few weeks ago. We run a FreeNAS server which hosts the VM images, and serves them over NFS. One time I began receiving notifications that the virtual hosts served by this NAS went down. I checked the NAS, found that one drive attached to a mps/lsi HBA stopped responding to the HBA at all, and thus blocked the whole pool. That was also strange to me that neither a timeout event happened, so actually zfs thought that all drives are fine, just one blocked the whole pool IOs. And unfortunately I even could not offline that drive, only a hard reset helped. The drive was so unresponsive that the bootup for FreeNAS also took long, but at least that time zfs somehow noticed the drive is missing, and removed it from the pool. And after that the pool worked fine, of course in a degraded state, but healthy, we could initiate a replacement. I only have the logs for the bootup: mps0: mpssas_scsiio_timeout checking sc 0xffffff8000fd1000 cm 0xffffff800100c0a0 (probe14:mps0:0:14:0): INQUIRY. CDB: 12 00 00 00 24 00 length 36 SMID 500 command timeout cm 0xffffff800100c0a0 ccb 0xfffffe0010f03000 mps0: mpssas_alloc_tm freezing simq mps0: timedout cm 0xffffff800100c0a0 allocated tm 0xffffff8000fe47b0 (probe14:mps0:0:14:0): INQUIRY. CDB: 12 00 00 00 24 00 length 36 SMID 500 completed timedout cm 0xffffff800100c0a0 ccb 0xfffffe0010f03000 during recovery ioc 8048 scsi 0 state c xfer 0 (noperiph:mps0:0:14:0): SMID 6 abort TaskMID 500 status 0x0 code 0x0 count 1 (noperiph:mps0:0:14:0): SMID 6 finished recovery after aborting TaskMID 500 mps0: mpssas_free_tm releasing simq (probe14:mps0:0:14:0): INQUIRY. CDB: 12 00 00 00 24 00 (probe14:mps0:0:14:0): CAM status: Command timeout (probe14:mps0:0:14:0): Retrying command mps0: mpssas_scsiio_timeout checking sc 0xffffff8000fd1000 cm 0xffffff8001032f50 (probe14:mps0:0:14:0): INQUIRY. CDB: 12 00 00 00 24 00 length 36 SMID 986 command timeout cm 0xffffff8001032f50 ccb 0xfffffe0010f03000 mps0: mpssas_alloc_tm freezing simq mps0: timedout cm 0xffffff8001032f50 allocated tm 0xffffff8000fe48f8 (probe14:mps0:0:14:0): INQUIRY. CDB: 12 00 00 00 24 00 length 36 SMID 986 completed timedout cm 0xffffff8001032f50 ccb 0xfffffe0010f03000 during recovery ioc 8048 scsi 0 state c xfer 0 (noperiph:mps0:0:14:0): SMID 7 abort TaskMID 986 status 0x0 code 0x0 count 1 (noperiph:mps0:0:14:0): SMID 7 finished recovery after aborting TaskMID 986 mps0: mpssas_free_tm releasing simq (probe14:mps0:0:14:0): INQUIRY. CDB: 12 00 00 00 24 00 (probe14:mps0:0:14:0): CAM status: Command timeout (probe14:mps0:0:14:0): Retrying command mps0: mpssas_scsiio_timeout checking sc 0xffffff8000fd1000 cm 0xffffff8001010c38 (probe14:mps0:0:14:0): INQUIRY. CDB: 12 00 00 00 24 00 length 36 SMID 559 command timeout cm 0xffffff8001010c38 ccb 0xfffffe0010f03000 mps0: mpssas_alloc_tm freezing simq mps0: timedout cm 0xffffff8001010c38 allocated tm 0xffffff8000fe4a40 (probe14:mps0:0:14:0): INQUIRY. CDB: 12 00 00 00 24 00 length 36 SMID 559 completed timedout cm 0xffffff8001010c38 ccb 0xfffffe0010f03000 during recovery ioc 8048 scsi 0 state c xfer 0 (noperiph:mps0:0:14:0): SMID 8 abort TaskMID 559 status 0x0 code 0x0 count 1 (noperiph:mps0:0:14:0): SMID 8 finished recovery after aborting TaskMID 559 mps0: mpssas_free_tm releasing simq (probe14:mps0:0:14:0): INQUIRY. CDB: 12 00 00 00 24 00 (probe14:mps0:0:14:0): CAM status: Command timeout (probe14:mps0:0:14:0): Retrying command mps0: mpssas_scsiio_timeout checking sc 0xffffff8000fd1000 cm 0xffffff8001007278 (probe14:mps0:0:14:0): INQUIRY. CDB: 12 00 00 00 24 00 length 36 SMID 439 command timeout cm 0xffffff8001007278 ccb 0xfffffe0010f03000 mps0: mpssas_alloc_tm freezing simq mps0: timedout cm 0xffffff8001007278 allocated tm 0xffffff8000fe4b88 (probe14:mps0:0:14:0): INQUIRY. CDB: 12 00 00 00 24 00 length 36 SMID 439 completed timedout cm 0xffffff8001007278 ccb 0xfffffe0010f03000 during recovery ioc 8048 scsi 0 state c xfer 0 (noperiph:mps0:0:14:0): SMID 9 abort TaskMID 439 status 0x0 code 0x0 count 1 (noperiph:mps0:0:14:0): SMID 9 finished recovery after aborting TaskMID 439 mps0: mpssas_free_tm releasing simq (probe14:mps0:0:14:0): INQUIRY. CDB: 12 00 00 00 24 00 (probe14:mps0:0:14:0): CAM status: Command timeout (probe14:mps0:0:14:0): Retrying command A side note is that the drive was removed from its hot-swap bay, then reinserted, and since then it is working fine. That is a seagate ST32000645SS. And no errors reported. Regards, 2014-02-08 21:02 időpontban Brian Gardner ezt írta: > Last year upgraded my production servers from ufs on adaptec raid > controllers to zfs raidz on lsi controllers. Last week I had an > outage, the culprit being a failed hard in my raidz. My log was > littered with kernel messages such as the ones below. About 20 > minutes after the first message some aspects of my host where hung, > and I was notified that my site was down. I noticed these messages in > my log but running a zfs status however showed only a few checksum > errors and the failed drive didn't get degraded. Manually degrading > the drive solved my problems. This seems very odd to me. It's almost > as if zfs wasn't getting messages from the lsi driver regarding these > read/write failures. Do I need to tune something in the mps/lsi or > zfs drivers to help it deal with failures? I'm running FreeBSD > 8.3-Release-p3 with the generic kernel, nothing unusual about my setup > other than I use jails extensively. There are four drives in the > raidz configuration in question: > > zpool status > pool: storage > state: ONLINE > scan: resilvered 67.6G in 2h21m with 0 errors on Tue Aug 6 00:44:14 > 2013 > config: > > NAME STATE READ WRITE CKSUM > storage ONLINE 0 0 0 > raidz1-0 ONLINE 0 0 0 > da0 ONLINE 0 0 0 > da1 ONLINE 0 0 0 > da2 ONLINE 0 0 0 > da3 ONLINE 0 0 0 > > Jan 29 19:03:18 host2 kernel: (da0:mpslsi0:0:0:0): WRITE(10). CDB: 2a > 0 9 72 d8 13 0 0 1d 0 > Jan 29 19:03:18 host2 kernel: (da0:mpslsi0:0:0:0): CAM status: SCSI > Status Error > Jan 29 19:03:18 host2 kernel: (da0:mpslsi0:0:0:0): SCSI status: Check > Condition > Jan 29 19:03:18 host2 kernel: (da0:mpslsi0:0:0:0): SCSI sense: > Deferred error: MEDIUM ERROR info:97ef13b asc:15,1 (Mechanical > positioning error) actual retry count: 15 > Jan 29 19:04:00 host2 kernel: (da0:mpslsi0:0:0:0): READ(10). CDB: 28 0 > 9 75 b9 96 0 0 b 0 > Jan 29 19:04:00 host2 kernel: (da0:mpslsi0:0:0:0): CAM status: SCSI > Status Error > Jan 29 19:04:00 host2 kernel: (da0:mpslsi0:0:0:0): SCSI status: Check > Condition > Jan 29 19:04:00 host2 kernel: (da0:mpslsi0:0:0:0): SCSI sense: MEDIUM > ERROR info:975b99b asc:15,1 (Mechanical positioning error) actual > retry count: 15 > Jan 29 19:04:54 host2 kernel: mpslsi0: mpssas_scsiio_timeout checking > sc 0xffffff80005ee000 cm 0xffffff800060d408 > Jan 29 19:04:54 host2 kernel: (da0:mpslsi0:0:0:0): READ(10). CDB: 28 0 > 2 e3 6a 1a 0 0 1 0 length 512 SMID 153 command timeout cm > 0xffffff800060d408 ccb 0xffffff000a1ce800 > Jan 29 19:04:54 host2 kernel: mpslsi0: mpssas_alloc_tm freezing simq > Jan 29 19:04:54 host2 kernel: mpslsi0: timedout cm 0xffffff800060d408 > allocated tm 0xffffff8000604718 > Jan 29 19:04:54 host2 kernel: (da0:mpslsi0:0:0:0): READ(10). CDB: 28 0 > 2 e3 6a 1a 0 0 1 0 length 512 SMID 153 completed timedout cm > 0xffffff800060d408 ccb 0xffffff000a1ce800 during recovery ioc 8048 > scsi 0 state c xfer 0 > Jan 29 19:04:54 host2 kernel: (noperiph:mpslsi0:0:0:0): SMID 43 abort > TaskMID 153 status 0x0 code 0x0 count 1 > Jan 29 19:04:54 host2 kernel: (noperiph:mpslsi0:0:0:0): SMID 43 > finished recovery after aborting TaskMID 153 > Jan 29 19:04:54 host2 kernel: mpslsi0: mpssas_free_tm releasing simq > Jan 29 19:04:54 host2 kernel: mpslsi0: mpssas_scsiio_timeout checking > sc 0xffffff80005ee000 cm 0xffffff800060d928 > Jan 29 19:04:54 host2 kernel: (da0:mpslsi0:0:0:0): READ(10). CDB: 28 0 > 9 41 c7 53 0 0 b 0 length 5632 SMID 157 command timeout cm > 0xffffff800060d928 ccb 0xffffff0176ccb000 > Jan 29 19:04:54 host2 kernel: mpslsi0: mpssas_alloc_tm freezing simq > Jan 29 19:04:54 host2 kernel: mpslsi0: timedout cm 0xffffff800060d928 > allocated tm 0xffffff8000604860 > Jan 29 19:04:54 host2 kernel: (da0:mpslsi0:0:0:0): READ(10). CDB: 28 0 > 9 41 c7 53 0 0 b 0 length 5632 SMID 157 completed timedout cm > 0xffffff800060d928 ccb 0xffffff0176ccb000 during recovery ioc 8048 > scsi 0 state c xfer 0(noperiph:mpslsi0:0:0:0): SMID 44 abort TaskMID > 157 status 0x0 code 0x0 count 1 > Jan 29 19:04:54 host2 kernel: (noperiph:mpslsi0:0:0:0): SMID 44 > finished recovery after aborting TaskMID 157 > Jan 29 19:04:54 host2 kernel: mpslsi0: mpssas_free_tm releasing simq > Jan 29 19:04:54 host2 kernel: mpslsi0: mpssas_scsiio_timeout checking > sc 0xffffff80005ee000 cm 0xffffff80006235a8 > Jan 29 19:04:54 host2 kernel: (da0:mpslsi0:0:0:0): WRITE(10). CDB: 2a > 0 9 72 e0 d3 0 1 0 0 length 131072 SMID 429 command timeout cm > 0xffffff80006235a8 ccb 0xffffff000c600000 > Jan 29 19:04:54 host2 kernel: mpslsi0: mpssas_alloc_tm freezing simq > Jan 29 19:04:54 host2 kernel: mpslsi0: timedout cm 0xffffff80006235a8 > allocated tm 0xffffff80006049a8 > Jan 29 19:04:54 host2 kernel: mpslsi0: mpssas_scsiio_timeout checking > sc 0xffffff80005ee000 cm 0xffffff800063e2e0 > Jan 29 19:04:54 host2 kernel: (da0:mpslsi0:0:0:0): WRITE(10). CDB: 2a > 0 9 72 e1 d3 0 0 3 0 length 1536 SMID 764 command timeout cm > 0xffffff800063e2e0 ccb 0xffffff01dfa19800 > Jan 29 19:04:54 host2 kernel: mpslsi0: queued timedout cm > 0xffffff800063e2e0 for processing by tm 0xffffff80006049a8 > Jan 29 19:04:54 host2 kernel: (da0:mpslsi0:0:0:0): WRITE(10). CDB: 2a > 0 9 72 e0 d3 0 1 0 0 length 131072 SMID 429 completed timedout cm > 0xffffff80006235a8 ccb 0xffffff000c600000 during recovery ioc 8048 > scsi 0 state c xfe(noperiph:mpslsi0:0:0:0): SMID 45 abort TaskMID 429 > status 0x0 code 0x0 count 1 > Jan 29 19:04:54 host2 kernel: (noperiph:mpslsi0:0:0:0): SMID 45 > continuing recovery after aborting TaskMID 429 > Jan 29 19:04:54 host2 kernel: mpslsi0: mpssas_scsiio_timeout checking > sc 0xffffff80005ee000 cm 0xffffff8000610890 > Jan 29 19:04:54 host2 kernel: (da0:mpslsi0:0:0:0): WRITE(10). CDB: 2a > 0 9 72 e3 31 0 0 be 0 length 97280 SMID 194 command timeout cm > 0xffffff8000610890 ccb 0xffffff013d0f3800 > Jan 29 19:04:54 host2 kernel: mpslsi0: queued timedout cm > 0xffffff8000610890 for processing by tm 0xffffff80006049a8 > Jan 29 19:04:54 host2 kernel: (da0:mpslsi0:0:0:0): WRITE(10). CDB: 2a > 0 9 72 e1 d3 0 0 3 0 length 1536 SMID 764 completed timedout cm > 0xffffff800063e2e0 ccb 0xffffff01dfa19800 during recovery ioc 8048 > scsi 0 state c xfer (noperiph:mpslsi0:0:0:0): SMID 45 abort TaskMID > 764 status 0x0 code 0x0 count 1 > _______________________________________________ > zfs-devel@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/zfs-devel > To unsubscribe, send any mail to "zfs-devel-unsubscribe@freebsd.org" From owner-zfs-devel@FreeBSD.ORG Fri Feb 14 11:55:06 2014 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A450E5AF for ; Fri, 14 Feb 2014 11:55:06 +0000 (UTC) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id D465C1E59 for ; Fri, 14 Feb 2014 11:55:05 +0000 (UTC) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id NAA13151 for ; Fri, 14 Feb 2014 13:55:04 +0200 (EET) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1WEHMF-000Nec-VC for zfs-devel@FreeBSD.org; Fri, 14 Feb 2014 13:55:03 +0200 Message-ID: <52FE03E0.90004@FreeBSD.org> Date: Fri, 14 Feb 2014 13:54:08 +0200 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.2.0 MIME-Version: 1.0 To: zfs-devel@FreeBSD.org Subject: making zdb -A option work X-Enigmail-Version: 1.6 Content-Type: text/plain; charset=X-VIET-VPS Content-Transfer-Encoding: 7bit X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 14 Feb 2014 11:55:06 -0000 https://github.com/avg-I/freebsd/compare/master...wip;zdb-A zdb sets aok variable based on -A option, but aok is not honored anywhere in userland. -- Andriy Gapon From owner-zfs-devel@FreeBSD.ORG Fri Feb 14 11:53:51 2014 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 9CBAF522; Fri, 14 Feb 2014 11:53:51 +0000 (UTC) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id BB9A01E41; Fri, 14 Feb 2014 11:53:47 +0000 (UTC) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id NAA13111; Fri, 14 Feb 2014 13:53:46 +0200 (EET) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1WEHKz-000NeT-V3; Fri, 14 Feb 2014 13:53:45 +0200 Message-ID: <52FE0378.7070608@FreeBSD.org> Date: Fri, 14 Feb 2014 13:52:24 +0200 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.2.0 MIME-Version: 1.0 To: freebsd-fs Subject: Re: l2arc_feed_thread cpu utlization References: <52B2D8D6.8090306@FreeBSD.org> In-Reply-To: <52B2D8D6.8090306@FreeBSD.org> X-Enigmail-Version: 1.6 Content-Type: text/plain; charset=x-viet-vps Content-Transfer-Encoding: 7bit X-Mailman-Approved-At: Fri, 14 Feb 2014 12:40:35 +0000 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 14 Feb 2014 11:53:51 -0000 on 19/12/2013 13:30 Andriy Gapon said the following: > > This is just a heads up, no patch yet. > > l2arc_feed_thread periodically wakes up and scans certain amount of ARC buffers > and writes eligible buffers to a cache device. > Number of scanned buffers is limited by a threshold on the amount of data in the > buffers seen. The threshold is applied on a per buffer list basis. In upstream > there are 4 relevant lists: (data, metadata) X (MFU, MRU). In FreeBSD each of > the lists was subdivided into 16 lists. This was done to reduce contention on > the locks that protect the lists. But as a side effect l2arc_feed_thread can > scan 16 times more data (~ buffers). > > So, if you have a rather large ARC and L2ARC and your buffers tend to be > sufficiently small, then you could observe l2arc_feed_thread burning a > noticeable amount of CPU. On some of our systems I observed it using up to 40% > of a single core. Scaling back the threshold by factor of 16 makes CPU > utilization go down by the same factor. > > I plan to commit this change to FreeBSD ZFS code. > Any comments are welcome. Here is what I have in mind: https://github.com/avg-I/freebsd/compare/wip;hc;l2arc_feed_thread_scan_rate The calculations in the macro look somewhat ugly, but they should be correct :-) -- Andriy Gapon From owner-zfs-devel@FreeBSD.ORG Fri Feb 14 14:35:23 2014 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3766CE34 for ; Fri, 14 Feb 2014 14:35:23 +0000 (UTC) Received: from mail-qa0-f44.google.com (mail-qa0-f44.google.com [209.85.216.44]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id EA3821D71 for ; Fri, 14 Feb 2014 14:35:22 +0000 (UTC) Received: by mail-qa0-f44.google.com with SMTP id w5so18410850qac.3 for ; Fri, 14 Feb 2014 06:35:21 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=h1MmWcgv8gf07MDYCokXlt0Wdxyy10YH3Ci470pd09s=; b=Il1Msqi0yU4Zh9Qf8OFABVYieFiKKHJ1wnB561PNhGCj1BjYofQbs+P/nJ5CFWfXkU /eIrQ1ctujpGe4hKDL+luP5FIYIdZtbJ2KvNmdms3NK+FJKvKdXruR9tBsijlZaTY1mF 34poiuiAyxkblqSE+u9j8KZEm12oez/7IzyzOuXwClV5rj2AXnveiUU77c3zp1ohSakm wTEUVkK0tWUKVCGrHHvBsThCZj+NywL04CqPSrGEWLLT553W+jxHOUbfIm0ARfAgxO2F 9eK7M4oJMuNtPgaZKIKyxUAewxC/97i9DOdtWqnwBXPkk9YEo2CpDJLL2srW2aD23LsE 7lhw== X-Gm-Message-State: ALoCoQlWUNJ5c46Ss5fbdI06jV6wfc/ngsiRiH8WPeVCPkQc/fXBE2dvb+gKEHA8E5xnzf10R7vn MIME-Version: 1.0 X-Received: by 10.140.36.107 with SMTP id o98mr12710914qgo.25.1392388521711; Fri, 14 Feb 2014 06:35:21 -0800 (PST) Received: by 10.140.36.229 with HTTP; Fri, 14 Feb 2014 06:35:21 -0800 (PST) In-Reply-To: <52FE03E0.90004@FreeBSD.org> References: <52FE03E0.90004@FreeBSD.org> Date: Fri, 14 Feb 2014 07:35:21 -0700 Message-ID: Subject: Re: making zdb -A option work From: Will Andrews To: Andriy Gapon Content-Type: text/plain; charset=UTF-8 Cc: "zfs-devel@freebsd.org" X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 14 Feb 2014 14:35:23 -0000 LGTM. --Will. On Fri, Feb 14, 2014 at 4:54 AM, Andriy Gapon wrote: > > https://github.com/avg-I/freebsd/compare/master...wip;zdb-A > > zdb sets aok variable based on -A option, but aok is not honored anywhere in > userland. > > -- > Andriy Gapon > > _______________________________________________ > zfs-devel@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/zfs-devel > To unsubscribe, send any mail to "zfs-devel-unsubscribe@freebsd.org" From owner-zfs-devel@FreeBSD.ORG Fri Feb 14 17:02:08 2014 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E98BAC1A; Fri, 14 Feb 2014 17:02:08 +0000 (UTC) Received: from aslan.scsiguy.com (www.scsiguy.com [70.89.174.89]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id A5AD31E1F; Fri, 14 Feb 2014 17:02:08 +0000 (UTC) Received: from jt-mbp.home.scsiguy.org (jt-mbp.home.scsiguy.org [192.168.0.61]) (authenticated bits=0) by aslan.scsiguy.com (8.14.7/8.14.7) with ESMTP id s1EH1v90011050 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NO); Fri, 14 Feb 2014 10:02:01 -0700 (MST) (envelope-from gibbs@scsiguy.com) Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.1 \(1827\)) Subject: Re: making zdb -A option work From: "Justin T. Gibbs" In-Reply-To: <52FE03E0.90004@FreeBSD.org> Date: Fri, 14 Feb 2014 10:02:07 -0700 Content-Transfer-Encoding: quoted-printable Message-Id: <171D5D66-FEF7-4878-AC99-FB7A30BC66E3@scsiguy.com> References: <52FE03E0.90004@FreeBSD.org> To: Andriy Gapon X-Mailer: Apple Mail (2.1827) Cc: zfs-devel@FreeBSD.org X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 14 Feb 2014 17:02:09 -0000 On Feb 14, 2014, at 4:54 AM, Andriy Gapon wrote: >=20 > https://github.com/avg-I/freebsd/compare/master...wip;zdb-A >=20 > zdb sets aok variable based on -A option, but aok is not honored = anywhere in > userland. >=20 > --=20 > Andriy Gapon It=92s not very DRY. Perhaps use a macro: ABORT_UNLESS_OK()? =97 Justin From owner-zfs-devel@FreeBSD.ORG Wed Feb 19 19:05:58 2014 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id F18438B3; Wed, 19 Feb 2014 19:05:58 +0000 (UTC) Received: from anubis.delphij.net (anubis.delphij.net [IPv6:2001:470:1:117::25]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id CFB40184F; Wed, 19 Feb 2014 19:05:58 +0000 (UTC) Received: from zeta.ixsystems.com (unknown [69.198.165.132]) (using TLSv1 with cipher ECDHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by anubis.delphij.net (Postfix) with ESMTPSA id 79FC41390A; Wed, 19 Feb 2014 11:05:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=delphij.net; s=anubis; t=1392836758; bh=Wt/pM53rtZmD05WBePftDu383k9XUBwoKzLhmz2Tf1Y=; h=Date:From:Reply-To:To:Subject; b=1ZGy6nzDmCrZqzoU0eyaNPJLuehizs1iCcavskCCFxI5AAD+h85+3cjk5FMkaunOR FSxDXMjEvh8Z2h+XKhsuS5yduWh+LGf3d4ukhFxR6J+QcdMGVhkJEwcweBN0ebx+fi GVUuSAC1se6fdll18a4+Pk0/ChnVTJ1WrWXha+WM= Message-ID: <53050096.6050301@delphij.net> Date: Wed, 19 Feb 2014 11:05:58 -0800 From: Xin Li Organization: The FreeBSD Project MIME-Version: 1.0 To: "zfs-devel@freebsd.org" , Martin Matuska Subject: casesensitivity property X-Enigmail-Version: 1.6 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list Reply-To: d@delphij.net List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Feb 2014 19:05:59 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 Hi, In manual page, it says: casesensitivity=sensitive | insensitive | mixed The casesensitivity property is currently not supported on FreeBSD. Is this still true? (When I created casesensitivity=insensitive, it does what I am expecting). Cheers, - -- Xin LI https://www.delphij.net/ FreeBSD - The Power to Serve! Live free or die -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (FreeBSD) iQIcBAEBCgAGBQJTBQCWAAoJEJW2GBstM+nsu4gP/1kr2FQVXMKozCuqEv+abwSe ebdpfWQ3eeBw+ZWzmrCLaUDdnoVHsyf/aI3i9oBDWn/nNw0/o4qrujs35JiKi/Vz 6r8gv0KVkmq1zrAgFT2IxDkkyS30347MvuWxNVUR20t4Yf/Z7MWGxJFtys9deQ8C e6hp5Ag9mWRJiYZ3+TNascGx9F+hWJCHeMPzinJQoN+5w5gwZt7293TveuJlzk1N LjlRk0e8s2qe+8VHgBGxIBK9KR5INkbIWqVvEko3+Lkwf2h9/w7yDuMwsEZd6z2t ZhGgPuO4FIcXME0wrC8SUWSMft0g5t4lQm84KkmdAjwlgyV5dC42MpaL/0cocRzh V1CbXoqHqDzgGeSo7RauFk0oHE4ZPLddE4w8DqCzXGcW/0wWsLTqZkq2ysoSPJLl OsUt7mt8R6Yf2apTciJP1guXfwaFV8CT7y+6ln3StWgKM2C6mLSE+/1n/elsXr3A 1BFU5oH/s3sPJgRy6+hDuVeKvep6lhj13DwVdCcGZitR5HRZv9mkqHhj+0QCV+D9 V+UR/L58bT4zf0i4Mpk+Nonn+9RNY0o4IWMXkRGiiyuS98SbHJNnlxzcPAATySfu 0tBcWs/czHHcbzzyWEOI0+1dvJ9MXm7+XX1K5CnqwRVQYn1D7W0V86V/zqddjpgI yb5C+fTZstnbijtEgDWA =j+Nh -----END PGP SIGNATURE----- From owner-zfs-devel@FreeBSD.ORG Mon Mar 3 09:32:48 2014 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id CC07EEAD; Mon, 3 Mar 2014 09:32:48 +0000 (UTC) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id EAA8CEE8; Mon, 3 Mar 2014 09:32:47 +0000 (UTC) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id LAA03512; Mon, 03 Mar 2014 11:32:46 +0200 (EET) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1WKPEs-000NaS-1f; Mon, 03 Mar 2014 11:32:46 +0200 Message-ID: <53144C06.40207@FreeBSD.org> Date: Mon, 03 Mar 2014 11:31:50 +0200 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.2.0 MIME-Version: 1.0 To: freebsd-fs Subject: Re: l2arc_feed_thread cpu utlization References: <52B2D8D6.8090306@FreeBSD.org> <52FE0378.7070608@FreeBSD.org> In-Reply-To: <52FE0378.7070608@FreeBSD.org> X-Enigmail-Version: 1.6 Content-Type: text/plain; charset=x-viet-vps Content-Transfer-Encoding: 7bit X-Mailman-Approved-At: Mon, 03 Mar 2014 12:30:38 +0000 Cc: Brendan Gregg X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Mar 2014 09:32:48 -0000 on 14/02/2014 13:52 Andriy Gapon said the following: > on 19/12/2013 13:30 Andriy Gapon said the following: >> >> This is just a heads up, no patch yet. >> >> l2arc_feed_thread periodically wakes up and scans certain amount of ARC buffers >> and writes eligible buffers to a cache device. >> Number of scanned buffers is limited by a threshold on the amount of data in the >> buffers seen. The threshold is applied on a per buffer list basis. In upstream >> there are 4 relevant lists: (data, metadata) X (MFU, MRU). In FreeBSD each of >> the lists was subdivided into 16 lists. This was done to reduce contention on >> the locks that protect the lists. But as a side effect l2arc_feed_thread can >> scan 16 times more data (~ buffers). >> >> So, if you have a rather large ARC and L2ARC and your buffers tend to be >> sufficiently small, then you could observe l2arc_feed_thread burning a >> noticeable amount of CPU. On some of our systems I observed it using up to 40% >> of a single core. Scaling back the threshold by factor of 16 makes CPU >> utilization go down by the same factor. >> >> I plan to commit this change to FreeBSD ZFS code. >> Any comments are welcome. > > Here is what I have in mind: > https://github.com/avg-I/freebsd/compare/wip;hc;l2arc_feed_thread_scan_rate > > The calculations in the macro look somewhat ugly, but they should be correct :-) > Looks like the patch did more than I wanted. Not only it limited how much data is scanned per list, but it also limited how much data was written in total. So, this new patch should be better: https://github.com/avg-I/freebsd/compare/master...review;l2arc-feed-thread-scan-size.diff -- Andriy Gapon From owner-zfs-devel@FreeBSD.ORG Fri Mar 14 13:00:19 2014 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 266B8CC0 for ; Fri, 14 Mar 2014 13:00:19 +0000 (UTC) Received: from smtp1.multiplay.co.uk (smtp1.multiplay.co.uk [85.236.96.35]) by mx1.freebsd.org (Postfix) with ESMTP id CAC9576A for ; Fri, 14 Mar 2014 13:00:18 +0000 (UTC) Received: by smtp1.multiplay.co.uk (Postfix, from userid 65534) id 7D70E20E7088D; Fri, 14 Mar 2014 12:54:11 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.multiplay.co.uk X-Spam-Level: X-Spam-Status: No, score=-2.5 required=8.0 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham version=3.3.1 Received: from r2d2 (82-69-141-170.dsl.in-addr.zen.co.uk [82.69.141.170]) by smtp1.multiplay.co.uk (Postfix) with ESMTPA id 5820B20E7088B for ; Fri, 14 Mar 2014 12:54:09 +0000 (UTC) Message-ID: From: "Steven Hartland" To: Subject: Fw: Reoccurring ZFS performance problems [RESOLVED] Date: Fri, 14 Mar 2014 12:54:08 -0000 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="Windows-1252"; reply-type=response Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.5931 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.6157 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 14 Mar 2014 13:00:19 -0000 Just fowarding to zfs-devel in case not everyone is on -fs. ----- Original Message ----- From: "Karl Denninger" To: Sent: Friday, March 14, 2014 11:21 AM Subject: Re: Reoccurring ZFS performance problems [RESOLVED] On 3/12/2014 1:01 PM, Karl Denninger wrote: > > On 3/10/2014 2:38 PM, Adrian Gschwend wrote: >> On 10.03.14 18:40, Adrian Gschwend wrote: >> >>> It looks like finally my MySQL process finished and now the system is >>> back to completely fine: >> ok it doesn't look it's only MySQL, stopped the process a while ago and >> while it got calmer, I still have the issue. > ZFS can be convinced to engage in what I can only surmise is > pathological behavior, and I've seen no fix for it when it happens -- > but there are things you can do to mitigate it. > > What IMHO _*should*_ happen is that the ARC cache should shrink as > necessary to prevent paging, subject to vfs.zfs.arc_min. To prevent > pathological problems with segments that have been paged off hours (or > more!) ago and never get paged back in because that particular piece > of code never executes again (but the process is also still alive so > the system cannot reclaim it and thus it shows "committed" in pstat -s > but unless it is paged back in has no impact on system performance) > the policing on this would have to apply a "reasonableness" filter to > those pages (e.g. if it has been out on the page file for longer than > "X", ignore that particular allocation unit for this purpose.) > > This would cause the ARC cache to flush itself down automatically as > executable and data segment RAM commitments increase. > > The documentation says that this is the case and how it should work > but it doesn't appear to actually be this way in practice for many > workloads. I have seen "wired" RAM pinned at 20GB on one of my > servers here with a fairly large DBMS running -- with pieces of its > working set and even the a user's shell (!) getting paged off, yet the > ARC cache is not pared down to release memory. Indeed you can let the > system run for hours under these conditions and the ARC wired memory > will not decrease. Cutting back the DBMS's internal buffering does > not help. > > What I've done here is restrict the ARC cache size in an attempt to > prevent this particular bit of bogosity from biting me, and it appears > to (sort of) work. Unfortunately you cannot tune this while the > system is running (otherwise a user daemon could conceivably slash > away at the arc_max sysctl and force the deallocation of wired memory > if it detected paging -- or near-paging, such as free memory below > some user-configured threshold), only at boot time in /boot/loader.conf. > > This is something that, should I get myself a nice hunk of free time, > I may dive into and attempt to fix. It would likely take me quite a > while to get up to speed on this as I've not gotten into the zfs code > at all -- and mistakes in there could easily corrupt files.... (in > other words definitely NOT something to play with on a production > system!) > > I have to assume there's a pretty-good reason why you can't change > arc_max while the system is running; it _*can*_ be changed on a > running system on some other implementations (e.g. Solaris.) It is > marked with CTLFLAG_RDTUN in the arc management file which prohibits > run-time changes and the only place I see it referenced with a quick > look is in the arc_init code. > > Note that the test in arc.c for "arc_reclaim_needed" appears to be > pretty basic -- essentially the system will not aggressively try to > reclaim memory unless used kmem > 3/4 of its size. > > (snippet from arc.c around line 2494 of arc.c in 10-STABLE; path > /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs) > > #else /* !sun */ > if (kmem_used() > (kmem_size() * 3) / 4) > return (1); > #endif /* sun */ > > Up above that there's a test for "vm_paging_needed()" that would > (theoretically) appear to trigger first in these situations, but it > doesn't in many cases. > > IMHO this is too-basic of a test and leads to pathological situations > in that the system may wind up paging things off as opposed to paring > back the ARC cache. As soon as the working set of something that's > actually getting cycles gets paged out in most cases system > performance goes straight in the trash. > > On sun machines (from reading the code) it will allegedly try to pare > any time the "lotsfree" (plus "needfree" + "extra") amount of free > memory is invaded. > > As an example this is what a server I own that is exhibiting this > behavior now shows: > 20202500 wire > 1414052 act > 2323280 inact > 110340 cache > 414484 free > 1694896 buf > > Of that "wired" mem 15.7G of it is ARC cache (with a target of 15.81, > so it's essentially right up against it.) > > That "free" number would be ok if it didn't result in the system > having trashy performance -- but it does on occasion. Incidentally the > allocated swap is about 195k blocks (~200 Megabytes) which isn't much > all-in, but it's enough to force actual fetches of recently-used > programs (e.g. your shell!) from paged-off space. The thing is that if > the test in the code (75% of kmem available consumed) was looking only > at "free" the system should be aggressively trying to free up ARC > cache. It clearly is not; the included code calls this: > > uint64_t > kmem_used(void) > { > > return (vmem_size(kmem_arena, VMEM_ALLOC)); > } > > I need to dig around and see exactly what that's measuring, because > what's quite clear is that the system _*thinks*_ it has plenty of free > memory when it very-clearly is essentially out! In fact free memory > at the moment (~400MB) is 1.7% of the total, _*not*_ 25%. From this I > surmise that the "vmem_size" call is not returning the sum of all the > above "in use" sizes (except perhaps "inact"); were it to do so that > would be essentially 100% of installed RAM and the ARC cache should be > actively under shrinkage, but it clearly is not. > > I'll keep this one on my "to-do" list somewhere and if I get the > chance see if I can come up with a better test. What might be > interesting is to change the test to be "pare if free space less > (pagefile space in use plus some modest margin) < 0" > > Fixing this tidbit of code could potentially be pretty significant in > terms of resolving the occasional but very annoying "freeze" problems > that people sometimes run into, along with some mildly-pathological > but very-significant behavior in terms of how the ARC cache > auto-scales and its impact on performance. I'm nowhere near > up-to-speed enough on the internals of the kernel when it comes to > figuring out what it has committed (e.g. how much swap is out, etc) > and thus there's going to be a lot of code-reading involved before I > can attempt something useful. > In the context of the above, here's a fix. Enjoy. http://www.freebsd.org/cgi/query-pr.cgi?pr=187572 > Category: kern > Responsible: freebsd-bugs > Synopsis: ZFS ARC cache code does not properly handle low memory > Arrival-Date: Fri Mar 14 11:20:00 UTC 2014 -- -- Karl karl@denninger.net From owner-zfs-devel@FreeBSD.ORG Tue Apr 1 13:09:57 2014 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D87BEC60 for ; Tue, 1 Apr 2014 13:09:57 +0000 (UTC) Received: from smtp1.multiplay.co.uk (smtp1.multiplay.co.uk [85.236.96.35]) by mx1.freebsd.org (Postfix) with ESMTP id 89897F58 for ; Tue, 1 Apr 2014 13:09:57 +0000 (UTC) Received: by smtp1.multiplay.co.uk (Postfix, from userid 65534) id 8909120E7088B; Tue, 1 Apr 2014 13:09:55 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.multiplay.co.uk X-Spam-Level: X-Spam-Status: No, score=-0.0 required=8.0 tests=AWL,BAYES_00,DOS_OE_TO_MX, FSL_HELO_NON_FQDN_1,HELO_NO_DOMAIN,MIME_QP_LONG_LINE,RDNS_DYNAMIC autolearn=no version=3.3.1 Received: from r2d2 (82-69-141-170.dsl.in-addr.zen.co.uk [82.69.141.170]) by smtp1.multiplay.co.uk (Postfix) with ESMTP id 5964D20E70886 for ; Tue, 1 Apr 2014 13:09:50 +0000 (UTC) Message-ID: <87EFFB8AB4B34F47AD78A2946B9D577E@multiplay.co.uk> From: "Steven Hartland" To: Subject: Adding zfs_min_auto_ashift any objections? Date: Tue, 1 Apr 2014 14:09:47 +0100 MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="----=_NextPart_000_0943_01CF4DB4.09CB1570" X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.5931 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.6157 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 01 Apr 2014 13:09:57 -0000 This is a multi-part message in MIME format. ------=_NextPart_000_0943_01CF4DB4.09CB1570 Content-Type: text/plain; format=flowed; charset="iso-8859-1"; reply-type=original Content-Transfer-Encoding: 7bit I'm looking to add the ability to configure a system wide minimum ashift value which is useful for future compatibility e.g. when you want to force the use of 4k sectors even though current disks are 512b. Now we have logical / physical sector size calculations in the tree this makes doing this trivial but wanted to check no one has any objections to the attached patch? Regards Steve ------=_NextPart_000_0943_01CF4DB4.09CB1570 Content-Type: application/octet-stream; name="zfs-min-ashift.patch" Content-Transfer-Encoding: quoted-printable Content-Disposition: attachment; filename="zfs-min-ashift.patch" Add the ability to set a minimum ashift size for pool creation or root = level vdev=0A= addition.=0A= =0A= Change max_auto_ashift sysctl to error when an invalid value is = requested instead=0A= of silently limiting it.=0A= --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/spa.h.orig = 2014-03-26 17:52:33.000000000 +0000=0A= +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/spa.h 2014-04-01 = 13:05:59.782311846 +0000=0A= @@ -93,7 +93,7 @@=0A= #define SPA_BLOCKSIZES (SPA_MAXBLOCKSHIFT - SPA_MINBLOCKSHIFT + 1)=0A= =0A= /*=0A= - * Maximum supported logical ashift.=0A= + * Default maximum supported logical ashift.=0A= *=0A= * The current 8k allocation block size limit is due to the 8k=0A= * aligned/sized operations performed by vdev_probe() on=0A= @@ -104,6 +104,11 @@=0A= #define SPA_MAXASHIFT 13=0A= =0A= /*=0A= + * Default minimum supported logical ashift.=0A= + */=0A= +#define SPA_MINASHIFT SPA_MINBLOCKSHIFT=0A= +=0A= +/*=0A= * Size of block to hold the configuration data (a packed nvlist)=0A= */=0A= #define SPA_CONFIG_BLOCKSIZE (1ULL << 14)=0A= --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev.c.orig = 2014-04-01 13:05:20.557113825 +0000=0A= +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev.c 2014-04-01 = 13:05:37.538112255 +0000=0A= @@ -52,7 +52,7 @@=0A= * Virtual device management.=0A= */=0A= =0A= -/**=0A= +/*=0A= * The limit for ZFS to automatically increase a top-level vdev's ashift=0A= * from logical ashift to physical ashift.=0A= *=0A= @@ -60,19 +60,34 @@=0A= * child->vdev_ashift =3D 9 (512 bytes)=0A= * child->vdev_physical_ashift =3D 12 (4096 bytes)=0A= * zfs_max_auto_ashift =3D 11 (2048 bytes)=0A= + * zfs_min_auto_ashift =3D 9 (512 bytes)=0A= *=0A= - * On pool creation or the addition of a new top-leve vdev, ZFS will=0A= - * bump the ashift of the top-level vdev to 2048.=0A= + * On pool creation or the addition of a new top-level vdev, ZFS will=0A= + * increase the ashift of the top-level vdev to 2048 as limited by=0A= + * zfs_max_auto_ashift.=0A= *=0A= * Example: one or more 512B emulation child vdevs=0A= * child->vdev_ashift =3D 9 (512 bytes)=0A= * child->vdev_physical_ashift =3D 12 (4096 bytes)=0A= * zfs_max_auto_ashift =3D 13 (8192 bytes)=0A= + * zfs_min_auto_ashift =3D 9 (512 bytes)=0A= + *=0A= + * On pool creation or the addition of a new top-level vdev, ZFS will=0A= + * increase the ashift of the top-level vdev to 4096 to match the=0A= + * max vdev_physical_ashift.=0A= *=0A= - * On pool creation or the addition of a new top-leve vdev, ZFS will=0A= - * bump the ashift of the top-level vdev to 4096.=0A= + * Example: one or more 512B emulation child vdevs=0A= + * child->vdev_ashift =3D 9 (512 bytes)=0A= + * child->vdev_physical_ashift =3D 9 (512 bytes)=0A= + * zfs_max_auto_ashift =3D 13 (8192 bytes)=0A= + * zfs_min_auto_ashift =3D 12 (4096 bytes)=0A= + *=0A= + * On pool creation or the addition of a new top-level vdev, ZFS will=0A= + * increase the ashift of the top-level vdev to 4096 to match the=0A= + * zfs_min_auto_ashift.=0A= */=0A= static uint64_t zfs_max_auto_ashift =3D SPA_MAXASHIFT;=0A= +static uint64_t zfs_min_auto_ashift =3D SPA_MINASHIFT;=0A= =0A= static int=0A= sysctl_vfs_zfs_max_auto_ashift(SYSCTL_HANDLER_ARGS)=0A= @@ -85,8 +100,8 @@=0A= if (err !=3D 0 || req->newptr =3D=3D NULL)=0A= return (err);=0A= =0A= - if (val > SPA_MAXASHIFT)=0A= - val =3D SPA_MAXASHIFT;=0A= + if (val > SPA_MAXASHIFT || val < zfs_min_auto_ashift)=0A= + return (EINVAL);=0A= =0A= zfs_max_auto_ashift =3D val;=0A= =0A= @@ -95,7 +110,31 @@=0A= SYSCTL_PROC(_vfs_zfs, OID_AUTO, max_auto_ashift,=0A= CTLTYPE_U64 | CTLFLAG_MPSAFE | CTLFLAG_RW, 0, sizeof(uint64_t),=0A= sysctl_vfs_zfs_max_auto_ashift, "QU",=0A= - "Cap on logical -> physical ashift adjustment on new top-level = vdevs.");=0A= + "Max ashift used when optimising for logical -> physical sectors = size on "=0A= + "new top-level vdevs.");=0A= +=0A= +static int=0A= +sysctl_vfs_zfs_min_auto_ashift(SYSCTL_HANDLER_ARGS)=0A= +{=0A= + uint64_t val;=0A= + int err;=0A= +=0A= + val =3D zfs_min_auto_ashift;=0A= + err =3D sysctl_handle_64(oidp, &val, 0, req);=0A= + if (err !=3D 0 || req->newptr =3D=3D NULL)=0A= + return (err);=0A= +=0A= + if (val < SPA_MINASHIFT || val > zfs_max_auto_ashift)=0A= + return (EINVAL);=0A= +=0A= + zfs_min_auto_ashift =3D val;=0A= +=0A= + return (0);=0A= +}=0A= +SYSCTL_PROC(_vfs_zfs, OID_AUTO, min_auto_ashift,=0A= + CTLTYPE_U64 | CTLFLAG_MPSAFE | CTLFLAG_RW, 0, sizeof(uint64_t),=0A= + sysctl_vfs_zfs_min_auto_ashift, "QU",=0A= + "Min ashift used when creating new top-level vdevs.");=0A= =0A= static vdev_ops_t *vdev_ops_table[] =3D {=0A= &vdev_root_ops,=0A= @@ -1636,19 +1675,28 @@=0A= }=0A= =0A= /*=0A= - * Maximize performance by inflating the configured ashift for=0A= - * top level vdevs to be as close to the physical ashift as=0A= - * possible without exceeding the administrator specified=0A= - * limit.=0A= + * Maximize performance by inflating the configured ashift for top level=0A= + * vdevs to be as close to the physical ashift as possible while = maintaining=0A= + * administrator defined limits and ensuring it doesn't go below the=0A= + * logical ashift.=0A= */=0A= void=0A= vdev_ashift_optimize(vdev_t *vd)=0A= {=0A= - if (vd =3D=3D vd->vdev_top &&=0A= - (vd->vdev_ashift < vd->vdev_physical_ashift) &&=0A= - (vd->vdev_ashift < zfs_max_auto_ashift)) {=0A= - vd->vdev_ashift =3D MIN(zfs_max_auto_ashift,=0A= - vd->vdev_physical_ashift);=0A= + if (vd =3D=3D vd->vdev_top) {=0A= + if (vd->vdev_ashift < vd->vdev_physical_ashift) {=0A= + vd->vdev_ashift =3D MIN(=0A= + MAX(zfs_max_auto_ashift, vd->vdev_ashift),=0A= + MAX(zfs_min_auto_ashift, vd->vdev_physical_ashift));=0A= + } else {=0A= + /*=0A= + * Unusual case where logical ashift > physical ashift so we can't=0A= + * cap the calculated ashift based on max ashift as that would cause=0A= + * failures.=0A= + * We still check if we need to increase it to match the min ashift.=0A= + */=0A= + vd->vdev_ashift =3D MAX(zfs_min_auto_ashift, vd->vdev_ashift);=0A= + }=0A= }=0A= }=0A= =0A= ------=_NextPart_000_0943_01CF4DB4.09CB1570-- From owner-zfs-devel@FreeBSD.ORG Tue Apr 1 16:12:22 2014 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 92CF8C70; Tue, 1 Apr 2014 16:12:22 +0000 (UTC) Received: from mail.dawidek.net (garage.dawidek.net [91.121.88.72]) by mx1.freebsd.org (Postfix) with ESMTP id 5A381A37; Tue, 1 Apr 2014 16:12:21 +0000 (UTC) Received: from localhost (58.wheelsystems.com [83.12.187.58]) by mail.dawidek.net (Postfix) with ESMTPSA id 581697C8; Tue, 1 Apr 2014 18:06:40 +0200 (CEST) Date: Tue, 1 Apr 2014 18:08:57 +0200 From: Pawel Jakub Dawidek To: Steven Hartland Subject: Re: Adding zfs_min_auto_ashift any objections? Message-ID: <20140401160856.GA3939@garage.freebsd.pl> References: <87EFFB8AB4B34F47AD78A2946B9D577E@multiplay.co.uk> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="wRRV7LY7NUeQGEoC" Content-Disposition: inline In-Reply-To: <87EFFB8AB4B34F47AD78A2946B9D577E@multiplay.co.uk> X-OS: FreeBSD 11.0-CURRENT amd64 User-Agent: Mutt/1.5.22 (2013-10-16) Cc: zfs-devel@FreeBSD.org X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 01 Apr 2014 16:12:22 -0000 --wRRV7LY7NUeQGEoC Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Tue, Apr 01, 2014 at 02:09:47PM +0100, Steven Hartland wrote: > I'm looking to add the ability to configure a system wide > minimum ashift value which is useful for future compatibility > e.g. when you want to force the use of 4k sectors even though > current disks are 512b. >=20 > Now we have logical / physical sector size calculations in the > tree this makes doing this trivial but wanted to check no one > has any objections to the attached patch? LGTM. --=20 Pawel Jakub Dawidek http://www.wheelsystems.com FreeBSD committer http://www.FreeBSD.org Am I Evil? Yes, I Am! http://mobter.com --wRRV7LY7NUeQGEoC Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (FreeBSD) iEYEARECAAYFAlM65JgACgkQForvXbEpPzTqQQCg33N8ueuWDO9tkvxis+dXqqur zFAAoKwJtz7OFyYUBpfgRYGAhqL0edJX =YlOK -----END PGP SIGNATURE----- --wRRV7LY7NUeQGEoC-- From owner-zfs-devel@FreeBSD.ORG Tue Apr 1 16:25:38 2014 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 49529164 for ; Tue, 1 Apr 2014 16:25:38 +0000 (UTC) Received: from mail-ie0-f179.google.com (mail-ie0-f179.google.com [209.85.223.179]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 1276FB33 for ; Tue, 1 Apr 2014 16:25:37 +0000 (UTC) Received: by mail-ie0-f179.google.com with SMTP id lx4so9158175iec.24 for ; Tue, 01 Apr 2014 09:25:37 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:date:from:to:message-id:in-reply-to:references :subject:mime-version:content-type; bh=+pYks/V3XzPuON0FuhSe1EsCvCP1dVwb1tqM+Lg9cVM=; b=Spe78g6VfGsXwfkaMXJHCyFjbJLpTbOSRBnk+L5Kbp9C2dzAL1bnQUrjYvmPw2j+ta s6ZQL1gS6FFBYgnNRaa7gisTaz3wNLgc5q+tGO9DIvrMa6ZogZ8AHaMKN2ZiiAKdrxg9 vP+2bl10jS9LyQ+Y32kbf44fSH7AqmWdfbIgk7PmUdlHK+GiSTxgqW/zwb2MTdKKQF9z WuDGXP1TQyHITRSx7iLAIZjE4r1xVFM6NRgESgzteplXrjCtSifTHNl1Rr28/OrZq7j3 06s6DWMuxpwydBVR2fNnaqkkLAbHvqDEeP9wXY24eNaEJwVCCXE4N+hdRiGxDMdQPtjc D/zg== X-Gm-Message-State: ALoCoQnXCdceYw5F9IHYtL72dddyg1ofXXAS0sd3gaFDlSh9QbJmMjcQ0O9qOAK3KV2qgV0U8/gd X-Received: by 10.50.111.138 with SMTP id ii10mr3226594igb.34.1396369117364; Tue, 01 Apr 2014 09:18:37 -0700 (PDT) Received: from scorpius.firepipe.net (207-225-98-3.dia.static.qwest.net. [207.225.98.3]) by mx.google.com with ESMTPSA id z10sm4311457igm.8.2014.04.01.09.18.36 for (version=TLSv1.2 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 01 Apr 2014 09:18:36 -0700 (PDT) Date: Tue, 1 Apr 2014 10:18:38 -0600 From: Will Andrews To: Steven Hartland , zfs-devel@freebsd.org Message-ID: In-Reply-To: <87EFFB8AB4B34F47AD78A2946B9D577E@multiplay.co.uk> References: <87EFFB8AB4B34F47AD78A2946B9D577E@multiplay.co.uk> Subject: Re: Adding zfs_min_auto_ashift any objections? X-Mailer: Airmail (231) MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Content-Disposition: inline X-Content-Filtered-By: Mailman/MimeDel 2.1.17 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 01 Apr 2014 16:25:38 -0000 =46rom:=C2=A0Steven Hartland smh=40freebsd.org I'm looking to add the ability to configure a system wide =20 minimum ashift value which is useful for future compatibility =20 e.g. when you want to force the use of 4k sectors even though =20 current disks are 512b. =20 Now we have logical / physical sector size calculations in the =20 tree this makes doing this trivial but wanted to check no one =20 has any objections to the attached patch=3F =20 A few nits: - Please reduce the line length to < 80 chars. - style(9): indent followup lines by 4 spaces, not tabs. =E2=80=94Will. From owner-zfs-devel@FreeBSD.ORG Tue May 20 09:52:44 2014 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 40A6C1B7 for ; Tue, 20 May 2014 09:52:44 +0000 (UTC) Received: from mail-ve0-f177.google.com (mail-ve0-f177.google.com [209.85.128.177]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 023902292 for ; Tue, 20 May 2014 09:52:40 +0000 (UTC) Received: by mail-ve0-f177.google.com with SMTP id db11so267637veb.22 for ; Tue, 20 May 2014 02:52:33 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:from:date:message-id:subject:to :content-type:content-transfer-encoding; bh=CtSmVly6ZTqCfQHMCJWnRm/43bQMyMWJJD9Rw8ReqN8=; b=Mpu+K4xVw3SW1h5PMCkvOBUef5Bv/w87hW4PrRwedNAZ4gReUVpI2Ijq1XiNPVVs14 qnz5+KJfgHmS2exfRo7pMbDsB0f0hVHxkTDVHSc2mctXRsPG0l9OOSjT3k1fAePBtwJQ ImnsLoIzjFy9eVteb3AI/OCl7xoC3OJ7I+XNdTfV0FmCN5RSC2hw2Re+fWSt1T1vqCyN RUGAWGhrlx8T4jyoi3ptT5QFOY+DBfGbbLjFn7o73c/xKGwtPyHtg1LnVtmEZzx/n/Rs IgmB3wUXDgVe/AicEoDQarMlyoQaZwiXxGmBTsDdFQFdR8OOb8VF7lHBeGiBJC9fW+RS bPIg== X-Gm-Message-State: ALoCoQlSSfJx458YF21Br2Us8zm7zXDEyYT4o7fmXJecZcJDxi2KqCRMDm+Dm4mV+DLakG31e/z0 X-Received: by 10.58.188.115 with SMTP id fz19mr684443vec.40.1400579211799; Tue, 20 May 2014 02:46:51 -0700 (PDT) MIME-Version: 1.0 Received: by 10.58.32.227 with HTTP; Tue, 20 May 2014 02:46:21 -0700 (PDT) X-Originating-IP: [64.69.46.199] From: Anthony Ananich Date: Tue, 20 May 2014 12:46:21 +0300 Message-ID: Subject: Removing log device from ZFS pool To: zfs-devel@freebsd.org Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 May 2014 09:52:44 -0000 Hi! Here is what I tried to do: 1) create zfs pool (two hard disks) 2) add log device to the pool 3) add cache device to the pool 4) reboot server In this scenario log device dies during the reboot. ----- # zpool list tank NAME SIZE ALLOC FREE CAP DEDUP HEALTH ALTROOT tank 928G 274G 654G 29% 1.00x ONLINE - # zpool status tank pool: tank state: ONLINE scan: none requested config: NAME STATE READ WRITE CKSUM tank ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 gpt/disk1 ONLINE 0 0 0 gpt/disk2 ONLINE 0 0 0 errors: No known data errors # mdconfig -a -t swap -s 128m -u 1 # zpool add tank log /dev/md1 # zpool status tank pool: tank state: ONLINE scan: none requested config: NAME STATE READ WRITE CKSUM raptor2 ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 gpt/disk1 ONLINE 0 0 0 gpt/disk2 ONLINE 0 0 0 logs md1 ONLINE 0 0 0 errors: No known data errors ----- As long as I'm using volatile device /dev/md1 in this example, it is destroyed during reboot. Due to documentation this is not critical, I can just ignore unsaved data and discard uncommitted log entries. However it does not work for me in reality: ----- # zpool status tank pool: tank state: FAULTED status: An intent log record could not be read. Waiting for adminstrator intervention to fix the faulted pool. action: Either restore the affected device(s) and run 'zpool online', or ignore the intent log records by running 'zpool clear'. see: http://illumos.org/msg/ZFS-8000-K4 scan: none requested config: NAME STATE READ WRITE CKSUM tank FAULTED 0 0 0 mirror-0 ONLINE 0 0 0 gpt/disk1 ONLINE 0 0 0 gpt/disk2 ONLINE 0 0 0 logs 6324139563861643487 UNAVAIL 0 0 0 was /dev/md1 # zpool clear tank cannot clear errors for tank: one or more devices is currently unavailable # zpool remove tank 6324139563861643487 cannot open 'tank': pool is unavailable # zpool online tank md1 cannot open =E2=80=98tank=E2=80=99: pool is unavailable ----- O wonder if I'm doing something wrong and this is expected behaviour? Or that's just a bug? I'm using zfs v5000 at FreeBSD 9.2-RELEASE. Regards, Anthony From owner-zfs-devel@FreeBSD.ORG Tue May 20 09:58:43 2014 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id F0668225 for ; Tue, 20 May 2014 09:58:42 +0000 (UTC) Received: from smtp1.multiplay.co.uk (smtp1.multiplay.co.uk [85.236.96.35]) by mx1.freebsd.org (Postfix) with ESMTP id 88A4F22C7 for ; Tue, 20 May 2014 09:58:42 +0000 (UTC) Received: by smtp1.multiplay.co.uk (Postfix, from userid 65534) id 21FBE20E7088C; Tue, 20 May 2014 09:58:40 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.multiplay.co.uk X-Spam-Level: ** X-Spam-Status: No, score=2.2 required=8.0 tests=AWL,BAYES_00,DOS_OE_TO_MX, FSL_HELO_NON_FQDN_1,HELO_NO_DOMAIN,RDNS_DYNAMIC,STOX_REPLY_TYPE autolearn=no version=3.3.1 Received: from r2d2 (82-69-141-170.dsl.in-addr.zen.co.uk [82.69.141.170]) by smtp1.multiplay.co.uk (Postfix) with ESMTPS id 5276820E7088A; Tue, 20 May 2014 09:58:36 +0000 (UTC) Message-ID: <18A917985D7C42C595ED8FF3C03810E6@multiplay.co.uk> From: "Steven Hartland" To: "Anthony Ananich" , References: Subject: Re: Removing log device from ZFS pool Date: Tue, 20 May 2014 10:58:37 +0100 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="UTF-8"; reply-type=original Content-Transfer-Encoding: 8bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.5931 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.6157 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 May 2014 09:58:43 -0000 Simply don't that will break the world, log devices must be persistent Regards Steve ----- Original Message ----- From: "Anthony Ananich" To: Sent: Tuesday, May 20, 2014 10:46 AM Subject: Removing log device from ZFS pool > Hi! > > Here is what I tried to do: > > 1) create zfs pool (two hard disks) > 2) add log device to the pool > 3) add cache device to the pool > 4) reboot server > > In this scenario log device dies during the reboot. > > ----- > # zpool list tank > NAME SIZE ALLOC FREE CAP DEDUP HEALTH ALTROOT > tank 928G 274G 654G 29% 1.00x ONLINE - > > # zpool status tank > pool: tank > state: ONLINE > scan: none requested > config: > NAME STATE READ WRITE CKSUM > tank ONLINE 0 0 0 > mirror-0 ONLINE 0 0 0 > gpt/disk1 ONLINE 0 0 0 > gpt/disk2 ONLINE 0 0 0 > errors: No known data errors > > # mdconfig -a -t swap -s 128m -u 1 > # zpool add tank log /dev/md1 > # zpool status tank > pool: tank > state: ONLINE > scan: none requested > config: > NAME STATE READ WRITE CKSUM > raptor2 ONLINE 0 0 0 > mirror-0 ONLINE 0 0 0 > gpt/disk1 ONLINE 0 0 0 > gpt/disk2 ONLINE 0 0 0 > logs > md1 ONLINE 0 0 0 > errors: No known data errors > ----- > > As long as I'm using volatile device /dev/md1 in this example, it is > destroyed during reboot. > > Due to documentation this is not critical, I can just ignore unsaved > data and discard uncommitted log entries. > > However it does not work for me in reality: > > ----- > # zpool status tank > pool: tank > state: FAULTED > status: An intent log record could not be read. > Waiting for adminstrator intervention to fix the faulted pool. > action: Either restore the affected device(s) and run 'zpool online', > or ignore the intent log records by running 'zpool clear'. > see: http://illumos.org/msg/ZFS-8000-K4 > scan: none requested > config: > > NAME STATE READ WRITE CKSUM > tank FAULTED 0 0 0 > mirror-0 ONLINE 0 0 0 > gpt/disk1 ONLINE 0 0 0 > gpt/disk2 ONLINE 0 0 0 > logs > 6324139563861643487 UNAVAIL 0 0 0 was /dev/md1 > > # zpool clear tank > cannot clear errors for tank: one or more devices is currently unavailable > > # zpool remove tank 6324139563861643487 > cannot open 'tank': pool is unavailable > > # zpool online tank md1 > cannot open ‘tank’: pool is unavailable > > ----- > > O wonder if I'm doing something wrong and this is expected behaviour? > Or that's just a bug? > > I'm using zfs v5000 at FreeBSD 9.2-RELEASE. > > Regards, > Anthony > _______________________________________________ > zfs-devel@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/zfs-devel > To unsubscribe, send any mail to "zfs-devel-unsubscribe@freebsd.org" From owner-zfs-devel@FreeBSD.ORG Tue May 20 10:21:44 2014 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E5FF477C for ; Tue, 20 May 2014 10:21:44 +0000 (UTC) Received: from mail-vc0-f175.google.com (mail-vc0-f175.google.com [209.85.220.175]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A3F2D2502 for ; Tue, 20 May 2014 10:21:44 +0000 (UTC) Received: by mail-vc0-f175.google.com with SMTP id hu19so304668vcb.34 for ; Tue, 20 May 2014 03:21:37 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc:content-type:content-transfer-encoding; bh=hL/njedMDPBocoz8YeoRCms2U9ZsV5+4YVq+4panCi0=; b=iFHdEHz3fLeURxoaTknKuTwG0WQ8U6biS+0wLcBCbiEWTkryZ2zkAUmmrPmBluxLJ5 IvdajxXq/zQvxctARJKqLIeGRAZD+i9amwEj5X802r2KFuBrjrS7jjnkatSEWJ0lnmwp Aq5morCgWhQasTxt3XdoLjuY0O/XEQuOwZNi3kNW4XIQYXNQzvr29IXiId5zN6CzpR8g yaLkNkfANnwNcVnizUklpkgI7vZ6gLoixGKNn2DF1UHvhMQqUQjQQ+6GRPZyCor9mBNg TyVTD8daXAVxPbiZZhtDNTeHuI6A5HyzsoeByV1zaVcf3+ZDwDFVHuPzGy3YuDGdjKGo vKHg== X-Gm-Message-State: ALoCoQlKCAmML22eoNNTLe61IDAruzpEe/qxg2BFz1OWg82lHM33oRr+UsLxRF9Evdc2vTpMf5Vd X-Received: by 10.58.13.104 with SMTP id g8mr36521629vec.16.1400581297154; Tue, 20 May 2014 03:21:37 -0700 (PDT) MIME-Version: 1.0 Received: by 10.58.32.227 with HTTP; Tue, 20 May 2014 03:21:07 -0700 (PDT) X-Originating-IP: [64.69.46.199] In-Reply-To: <18A917985D7C42C595ED8FF3C03810E6@multiplay.co.uk> References: <18A917985D7C42C595ED8FF3C03810E6@multiplay.co.uk> From: Anthony Ananich Date: Tue, 20 May 2014 13:21:07 +0300 Message-ID: Subject: Re: Removing log device from ZFS pool To: Steven Hartland Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: zfs-devel X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 May 2014 10:21:45 -0000 Hi, Steve, Thanks for the quick reply. So if I got you right, the data in the pool is lost, right? This doc assert the log device is not critical: http://docs.oracle.com/cd/E19253-01/819-5461/ghbxs/ Disks die from time to time. That's a reality. However in this particular case I'm nearly sure that all the transactions was committed to the persistent disks and ZIL was empty just due to specifics of the scenario. The only problem is that I'm using zfs v5000 and can not use Oracle Solaris to restore the pool. It does not support v5000. Do I have to patch sources to walk around this problem? Is there a more easy way? Thanks, Anthony On Tue, May 20, 2014 at 12:58 PM, Steven Hartland wrote: > Simply don't that will break the world, log devices must be persistent > > Regards > Steve > ----- Original Message ----- From: "Anthony Ananich" > > To: > Sent: Tuesday, May 20, 2014 10:46 AM > Subject: Removing log device from ZFS pool > > >> Hi! >> >> Here is what I tried to do: >> >> 1) create zfs pool (two hard disks) >> 2) add log device to the pool >> 3) add cache device to the pool >> 4) reboot server >> >> In this scenario log device dies during the reboot. >> >> ----- >> # zpool list tank >> NAME SIZE ALLOC FREE CAP DEDUP HEALTH ALTROOT >> tank 928G 274G 654G 29% 1.00x ONLINE - >> >> # zpool status tank >> pool: tank >> state: ONLINE >> scan: none requested >> config: >> NAME STATE READ WRITE CKSUM >> tank ONLINE 0 0 0 >> mirror-0 ONLINE 0 0 0 >> gpt/disk1 ONLINE 0 0 0 >> gpt/disk2 ONLINE 0 0 0 >> errors: No known data errors >> >> # mdconfig -a -t swap -s 128m -u 1 >> # zpool add tank log /dev/md1 >> # zpool status tank >> pool: tank >> state: ONLINE >> scan: none requested >> config: >> NAME STATE READ WRITE CKSUM >> raptor2 ONLINE 0 0 0 >> mirror-0 ONLINE 0 0 0 >> gpt/disk1 ONLINE 0 0 0 >> gpt/disk2 ONLINE 0 0 0 >> logs >> md1 ONLINE 0 0 0 >> errors: No known data errors >> ----- >> >> As long as I'm using volatile device /dev/md1 in this example, it is >> destroyed during reboot. >> >> Due to documentation this is not critical, I can just ignore unsaved >> data and discard uncommitted log entries. >> >> However it does not work for me in reality: >> >> ----- >> # zpool status tank >> pool: tank >> state: FAULTED >> status: An intent log record could not be read. >> Waiting for adminstrator intervention to fix the faulted pool. >> action: Either restore the affected device(s) and run 'zpool online', >> or ignore the intent log records by running 'zpool clear'. >> see: http://illumos.org/msg/ZFS-8000-K4 >> scan: none requested >> config: >> >> NAME STATE READ WRITE CKSUM >> tank FAULTED 0 0 0 >> mirror-0 ONLINE 0 0 0 >> gpt/disk1 ONLINE 0 0 0 >> gpt/disk2 ONLINE 0 0 0 >> logs >> 6324139563861643487 UNAVAIL 0 0 0 was /dev/md1 >> >> # zpool clear tank >> cannot clear errors for tank: one or more devices is currently unavailab= le >> >> # zpool remove tank 6324139563861643487 >> cannot open 'tank': pool is unavailable >> >> # zpool online tank md1 >> cannot open =E2=80=98tank=E2=80=99: pool is unavailable >> >> ----- >> >> O wonder if I'm doing something wrong and this is expected behaviour? >> Or that's just a bug? >> >> I'm using zfs v5000 at FreeBSD 9.2-RELEASE. >> >> Regards, >> Anthony >> _______________________________________________ >> zfs-devel@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/zfs-devel >> To unsubscribe, send any mail to "zfs-devel-unsubscribe@freebsd.org" > > From owner-zfs-devel@FreeBSD.ORG Tue May 20 10:31:46 2014 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 91BFFB1A for ; Tue, 20 May 2014 10:31:46 +0000 (UTC) Received: from smtp1.multiplay.co.uk (smtp1.multiplay.co.uk [85.236.96.35]) by mx1.freebsd.org (Postfix) with ESMTP id 4304825D9 for ; Tue, 20 May 2014 10:31:45 +0000 (UTC) Received: by smtp1.multiplay.co.uk (Postfix, from userid 65534) id 0A64420E70896; Tue, 20 May 2014 10:31:45 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.multiplay.co.uk X-Spam-Level: ** X-Spam-Status: No, score=2.2 required=8.0 tests=AWL,BAYES_00,DOS_OE_TO_MX, FSL_HELO_NON_FQDN_1,HELO_NO_DOMAIN,RDNS_DYNAMIC,STOX_REPLY_TYPE autolearn=no version=3.3.1 Received: from r2d2 (82-69-141-170.dsl.in-addr.zen.co.uk [82.69.141.170]) by smtp1.multiplay.co.uk (Postfix) with ESMTPS id 7D37E20E7088A; Tue, 20 May 2014 10:31:41 +0000 (UTC) Message-ID: <49503C725725478C92FA776FF65052D2@multiplay.co.uk> From: "Steven Hartland" To: "Anthony Ananich" References: <18A917985D7C42C595ED8FF3C03810E6@multiplay.co.uk> Subject: Re: Removing log device from ZFS pool Date: Tue, 20 May 2014 11:31:42 +0100 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="UTF-8"; reply-type=original Content-Transfer-Encoding: 8bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.5931 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.6157 Cc: zfs-devel X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 May 2014 10:31:46 -0000 Try importing with -m (Enables import with missing log devices.) Regard Steve ----- Original Message ----- From: "Anthony Ananich" To: "Steven Hartland" Cc: "zfs-devel" Sent: Tuesday, May 20, 2014 11:21 AM Subject: Re: Removing log device from ZFS pool Hi, Steve, Thanks for the quick reply. So if I got you right, the data in the pool is lost, right? This doc assert the log device is not critical: http://docs.oracle.com/cd/E19253-01/819-5461/ghbxs/ Disks die from time to time. That's a reality. However in this particular case I'm nearly sure that all the transactions was committed to the persistent disks and ZIL was empty just due to specifics of the scenario. The only problem is that I'm using zfs v5000 and can not use Oracle Solaris to restore the pool. It does not support v5000. Do I have to patch sources to walk around this problem? Is there a more easy way? Thanks, Anthony On Tue, May 20, 2014 at 12:58 PM, Steven Hartland wrote: > Simply don't that will break the world, log devices must be persistent > > Regards > Steve > ----- Original Message ----- From: "Anthony Ananich" > > To: > Sent: Tuesday, May 20, 2014 10:46 AM > Subject: Removing log device from ZFS pool > > >> Hi! >> >> Here is what I tried to do: >> >> 1) create zfs pool (two hard disks) >> 2) add log device to the pool >> 3) add cache device to the pool >> 4) reboot server >> >> In this scenario log device dies during the reboot. >> >> ----- >> # zpool list tank >> NAME SIZE ALLOC FREE CAP DEDUP HEALTH ALTROOT >> tank 928G 274G 654G 29% 1.00x ONLINE - >> >> # zpool status tank >> pool: tank >> state: ONLINE >> scan: none requested >> config: >> NAME STATE READ WRITE CKSUM >> tank ONLINE 0 0 0 >> mirror-0 ONLINE 0 0 0 >> gpt/disk1 ONLINE 0 0 0 >> gpt/disk2 ONLINE 0 0 0 >> errors: No known data errors >> >> # mdconfig -a -t swap -s 128m -u 1 >> # zpool add tank log /dev/md1 >> # zpool status tank >> pool: tank >> state: ONLINE >> scan: none requested >> config: >> NAME STATE READ WRITE CKSUM >> raptor2 ONLINE 0 0 0 >> mirror-0 ONLINE 0 0 0 >> gpt/disk1 ONLINE 0 0 0 >> gpt/disk2 ONLINE 0 0 0 >> logs >> md1 ONLINE 0 0 0 >> errors: No known data errors >> ----- >> >> As long as I'm using volatile device /dev/md1 in this example, it is >> destroyed during reboot. >> >> Due to documentation this is not critical, I can just ignore unsaved >> data and discard uncommitted log entries. >> >> However it does not work for me in reality: >> >> ----- >> # zpool status tank >> pool: tank >> state: FAULTED >> status: An intent log record could not be read. >> Waiting for adminstrator intervention to fix the faulted pool. >> action: Either restore the affected device(s) and run 'zpool online', >> or ignore the intent log records by running 'zpool clear'. >> see: http://illumos.org/msg/ZFS-8000-K4 >> scan: none requested >> config: >> >> NAME STATE READ WRITE CKSUM >> tank FAULTED 0 0 0 >> mirror-0 ONLINE 0 0 0 >> gpt/disk1 ONLINE 0 0 0 >> gpt/disk2 ONLINE 0 0 0 >> logs >> 6324139563861643487 UNAVAIL 0 0 0 was /dev/md1 >> >> # zpool clear tank >> cannot clear errors for tank: one or more devices is currently unavailable >> >> # zpool remove tank 6324139563861643487 >> cannot open 'tank': pool is unavailable >> >> # zpool online tank md1 >> cannot open ‘tank’: pool is unavailable >> >> ----- >> >> O wonder if I'm doing something wrong and this is expected behaviour? >> Or that's just a bug? >> >> I'm using zfs v5000 at FreeBSD 9.2-RELEASE. >> >> Regards, >> Anthony >> _______________________________________________ >> zfs-devel@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/zfs-devel >> To unsubscribe, send any mail to "zfs-devel-unsubscribe@freebsd.org" > > From owner-zfs-devel@FreeBSD.ORG Tue May 20 11:00:00 2014 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 6737A200 for ; Tue, 20 May 2014 11:00:00 +0000 (UTC) Received: from mail-vc0-f171.google.com (mail-vc0-f171.google.com [209.85.220.171]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 23CAD2803 for ; Tue, 20 May 2014 10:59:59 +0000 (UTC) Received: by mail-vc0-f171.google.com with SMTP id lc6so361188vcb.30 for ; Tue, 20 May 2014 03:59:53 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc:content-type:content-transfer-encoding; bh=vBpFdA7KfCNP+fW96xwvlkIdotN76w5RlEpfH5uG4Ro=; b=amanNXArXM/lyTNPVU0FDEwgM9QuJ6PeSl+bBjoZLBpt1dbFLzFmGqUI6i1hDzev7B ZqK1HAdu+n/LixzchabnPCOpI81+UCJ4KnxFkLsBNquL1lRQMa1+wdQkafuehZhi/8Rd soVbURR47xdCcid8dTcrhFuLWQwwWurBpomDEDb4yKLWKbY6OdVSyHpmqne70nt+KXKG BXWudSGKnQkZsmRvS2LpugezYazqqafixvlqQDb/MMNeDk/Lp7PZPlaTCnoaWtPRjSmh FmTZj0BeiL5y+SRtFhOP+ZRbivbds4lbZ7979I8rfPdFJC5h5KmJEyXTA7IKjzRh7ubV GvWg== X-Gm-Message-State: ALoCoQnWf4asZ8zWYTjQVAkMDCI5+kmYrhIUklnpkAKFdQxi+ct2n/pi57QSTVHhUPpUceGPV1c9 X-Received: by 10.220.161.8 with SMTP id p8mr3026269vcx.4.1400583257727; Tue, 20 May 2014 03:54:17 -0700 (PDT) MIME-Version: 1.0 Received: by 10.58.32.227 with HTTP; Tue, 20 May 2014 03:53:47 -0700 (PDT) X-Originating-IP: [64.69.46.199] In-Reply-To: <49503C725725478C92FA776FF65052D2@multiplay.co.uk> References: <18A917985D7C42C595ED8FF3C03810E6@multiplay.co.uk> <49503C725725478C92FA776FF65052D2@multiplay.co.uk> From: Anthony Ananich Date: Tue, 20 May 2014 13:53:47 +0300 Message-ID: Subject: Re: Removing log device from ZFS pool To: Steven Hartland Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: zfs-devel X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 May 2014 11:00:00 -0000 Hi, Steven! Thank you very much, it solved my problem! # zpool export tank # zpool import -m tank # zpool status tank pool: tank state: DEGRADED status: One or more devices could not be opened. Sufficient replicas exist= for the pool to continue functioning in a degraded state. action: Attach the missing device and online it using 'zpool online'. see: http://illumos.org/msg/ZFS-8000-2Q scan: none requested config: NAME STATE READ WRITE CKSUM tank DEGRADED 0 0 0 mirror-0 ONLINE 0 0 0 gpt/disk1 ONLINE 0 0 0 gpt/disk2 ONLINE 0 0 0 logs 6324139563861643487 UNAVAIL 0 0 0 was /dev/md1 cache gpt/disk3 ONLINE 0 0 0 errors: No known data errors # zpool clear tank # zpool remove tank 6324139563861643487 # zpool status tank pool: tank state: ONLINE scan: none requested config: NAME STATE READ WRITE CKSUM tank ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 gpt/disk1 ONLINE 0 0 0 gpt/disk2 ONLINE 0 0 0 cache gpt/disk3 ONLINE 0 0 0 errors: No known data errors Kind regards, Anthony On Tue, May 20, 2014 at 1:31 PM, Steven Hartland wrote: > Try importing with -m (Enables import with missing log devices.) > > Regard > > Steve > > ----- Original Message ----- From: "Anthony Ananich" > > To: "Steven Hartland" > Cc: "zfs-devel" > Sent: Tuesday, May 20, 2014 11:21 AM > Subject: Re: Removing log device from ZFS pool > > > > Hi, Steve, > > Thanks for the quick reply. So if I got you right, the data in the > pool is lost, right? > > This doc assert the log device is not critical: > http://docs.oracle.com/cd/E19253-01/819-5461/ghbxs/ > > Disks die from time to time. That's a reality. However in this > particular case I'm nearly sure that all the transactions was > committed to the persistent disks and ZIL was empty just due to > specifics of the scenario. > > The only problem is that I'm using zfs v5000 and can not use Oracle > Solaris to restore the pool. It does not support v5000. > > Do I have to patch sources to walk around this problem? Is there a > more easy way? > > Thanks, > Anthony > > > > > > On Tue, May 20, 2014 at 12:58 PM, Steven Hartland > wrote: >> >> Simply don't that will break the world, log devices must be persistent >> >> Regards >> Steve >> ----- Original Message ----- From: "Anthony Ananich" >> >> To: >> Sent: Tuesday, May 20, 2014 10:46 AM >> Subject: Removing log device from ZFS pool >> >> >>> Hi! >>> >>> Here is what I tried to do: >>> >>> 1) create zfs pool (two hard disks) >>> 2) add log device to the pool >>> 3) add cache device to the pool >>> 4) reboot server >>> >>> In this scenario log device dies during the reboot. >>> >>> ----- >>> # zpool list tank >>> NAME SIZE ALLOC FREE CAP DEDUP HEALTH ALTROOT >>> tank 928G 274G 654G 29% 1.00x ONLINE - >>> >>> # zpool status tank >>> pool: tank >>> state: ONLINE >>> scan: none requested >>> config: >>> NAME STATE READ WRITE CKSUM >>> tank ONLINE 0 0 0 >>> mirror-0 ONLINE 0 0 0 >>> gpt/disk1 ONLINE 0 0 0 >>> gpt/disk2 ONLINE 0 0 0 >>> errors: No known data errors >>> >>> # mdconfig -a -t swap -s 128m -u 1 >>> # zpool add tank log /dev/md1 >>> # zpool status tank >>> pool: tank >>> state: ONLINE >>> scan: none requested >>> config: >>> NAME STATE READ WRITE CKSUM >>> raptor2 ONLINE 0 0 0 >>> mirror-0 ONLINE 0 0 0 >>> gpt/disk1 ONLINE 0 0 0 >>> gpt/disk2 ONLINE 0 0 0 >>> logs >>> md1 ONLINE 0 0 0 >>> errors: No known data errors >>> ----- >>> >>> As long as I'm using volatile device /dev/md1 in this example, it is >>> destroyed during reboot. >>> >>> Due to documentation this is not critical, I can just ignore unsaved >>> data and discard uncommitted log entries. >>> >>> However it does not work for me in reality: >>> >>> ----- >>> # zpool status tank >>> pool: tank >>> state: FAULTED >>> status: An intent log record could not be read. >>> Waiting for adminstrator intervention to fix the faulted pool. >>> action: Either restore the affected device(s) and run 'zpool online', >>> or ignore the intent log records by running 'zpool clear'. >>> see: http://illumos.org/msg/ZFS-8000-K4 >>> scan: none requested >>> config: >>> >>> NAME STATE READ WRITE CKSUM >>> tank FAULTED 0 0 0 >>> mirror-0 ONLINE 0 0 0 >>> gpt/disk1 ONLINE 0 0 0 >>> gpt/disk2 ONLINE 0 0 0 >>> logs >>> 6324139563861643487 UNAVAIL 0 0 0 was /dev/md1 >>> >>> # zpool clear tank >>> cannot clear errors for tank: one or more devices is currently >>> unavailable >>> >>> # zpool remove tank 6324139563861643487 >>> cannot open 'tank': pool is unavailable >>> >>> # zpool online tank md1 >>> cannot open =E2=80=98tank=E2=80=99: pool is unavailable >>> >>> ----- >>> >>> O wonder if I'm doing something wrong and this is expected behaviour? >>> Or that's just a bug? >>> >>> I'm using zfs v5000 at FreeBSD 9.2-RELEASE. >>> >>> Regards, >>> Anthony >>> _______________________________________________ >>> zfs-devel@freebsd.org mailing list >>> http://lists.freebsd.org/mailman/listinfo/zfs-devel >>> To unsubscribe, send any mail to "zfs-devel-unsubscribe@freebsd.org" >> >> >> > From owner-zfs-devel@FreeBSD.ORG Wed Jun 11 18:25:59 2014 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 02039B28; Wed, 11 Jun 2014 18:25:59 +0000 (UTC) Received: from mail-ve0-x22b.google.com (mail-ve0-x22b.google.com [IPv6:2607:f8b0:400c:c01::22b]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A4E84256F; Wed, 11 Jun 2014 18:25:58 +0000 (UTC) Received: by mail-ve0-f171.google.com with SMTP id jz11so291152veb.16 for ; Wed, 11 Jun 2014 11:25:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=HgasuEKU6cFDC+gzGNNyiyUTZf2s4nCqq9yJNzcRgH0=; b=fk8+HSc7c6s7fAjsnTaUGUr/XHBHoA24wj7KBvwYlY/y8X8toUzc0JWs56NFfRDzlX kF/wNHk6Du4mZVBJFpeC9m7X7ydaqd7f4BxcK79AyvAazkKrzXXbmNxj3rDf65rDHlRB oganF3xajPdAQO6+vi0pFSYgcnPAIOc+r2j+d3hHe1ahfspS6C8cyBcaO+vLxsHf4VJA h2+wTiZlhV4g50zlshVHRK6tSGm83uXsdjjmkr8nbmAzMCWN6U3jZKfPqaTTeHpNX7RP 0smYInPUKd6DECbfkfTcl1V4jhd4ioUGtnkGgnftZp3OHiZB6V+2lQzCNJRyiRvETG0N 80Zg== MIME-Version: 1.0 X-Received: by 10.58.46.141 with SMTP id v13mr40611003vem.18.1402511157766; Wed, 11 Jun 2014 11:25:57 -0700 (PDT) Received: by 10.221.65.198 with HTTP; Wed, 11 Jun 2014 11:25:57 -0700 (PDT) In-Reply-To: <201406111252.02544.jhb@freebsd.org> References: <201406101158.08599.jhb@freebsd.org> <201406111252.02544.jhb@freebsd.org> Date: Wed, 11 Jun 2014 14:25:57 -0400 Message-ID: Subject: Re: ZFS import panic (kgdb backtrace attached) From: grarpamp To: freebsd-fs@freebsd.org Content-Type: text/plain; charset=UTF-8 Cc: zfs-devel@freebsd.org X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 Jun 2014 18:25:59 -0000 On Wed, Jun 11, 2014 at 12:52 PM, John Baldwin wrote: > On Tuesday, June 10, 2014 2:00:37 pm grarpamp wrote: >> On Tue, Jun 10, 2014 at 11:58 AM, John Baldwin wrote: >> > Can you do 'frame 7' and 'p *rt' and 'p *rt->rt_ops'? >> >> (kgdb) frame 7 >> #7 0xc13cb9f4 in range_tree_vacate (rt=0xc83dc000, func=0, arg=0x0) >> at > /.../src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/range_tree.c:364 >> 364 rt->rt_ops->rtop_vacate(rt, rt->rt_arg); >> (kgdb) p *rt >> $1 = {rt_root = {avl_root = 0xd6900780, avl_compar = 0xc13cb890 >> , avl_offset = 0, avl_numnodes = 1, avl_size = >> 48}, >> rt_space = 4294967296, rt_ops = 0x1, rt_arg = 0x0, rt_histogram = {0 >> }, rt_lock = 0xc8309000} >> (kgdb) p *rt->rt_ops >> Cannot access memory at address 0x1 > > Humm, that is the source of the actual fault. I've no idea why that would > be set to 1 however. Unfortunately you need someone more familiar with ZFS to > look at this further. Ok, copying thread to zfs-devel, not sure if it's the right place but probably has open-zfs.org people. I'll see about posting status on RELENG_10 x64 when I can move it there. http://docs.freebsd.org/cgi/mid.cgi?CAD2Ti29gKmED34S5z6NEUnaGOsx8m2uPEJiPWPZLcebJ6PD-mw http://docs.freebsd.org/cgi/mid.cgi?CAD2Ti2_DZqDbOnbwap-YrOEjavyRZ4H7JZ1r8mkk4_OPrYQEUg From owner-zfs-devel@FreeBSD.ORG Thu Jun 26 14:24:53 2014 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id DE5C577F for ; Thu, 26 Jun 2014 14:24:53 +0000 (UTC) Received: from wetteronline.de (mail.wetteronline.de [89.1.23.61]) by mx1.freebsd.org (Postfix) with ESMTP id E49622EA0 for ; Thu, 26 Jun 2014 14:24:51 +0000 (UTC) Received: from [192.168.30.235] (account tschlich@wetteronline.de HELO morpheus-2.local) by wetteronline.de (CommuniGate Pro SMTP 5.4.4) with ESMTPSA id 22451186 for zfs-devel@freebsd.org; Thu, 26 Jun 2014 08:14:33 +0000 Message-ID: <53ABD65F.7090101@wetteronline.de> Date: Thu, 26 Jun 2014 10:14:23 +0200 From: Thorsten Schlich Reply-To: thorsten.schlich@wetteronline.de Organization: WetterOnline GmbH User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: zfs-devel@freebsd.org Subject: Binary file corruption in raidz pool X-Enigmail-Version: 1.6 Content-Type: text/plain; charset=ISO-8859-15 Content-Transfer-Encoding: 7bit X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Jun 2014 14:24:53 -0000 Good morning, we are discovering some file corruption in two of our zpools in our archive system. Our archive system which stores our historical files contains two servers with a raidz zpool where the data is stored. The file-sending servers are based on zfs too but they have only 1 disk in the pool. The archive servers have FreeBSD 9.2 installed the other servers are currently on FreeBSD 8.3. In one case we store 15682 files every day (meteorological data) in our own binary format. But when the files are copied to the archive an amount between 9000 and 10000 files differ every day. When i compare the values of the original data with the copied data i can see various little differences. The data is only copied if the access time is older than 2 days so no one has changed the data at transfer. We've tested at this point with various situations (copy to ufs, copy to several other servers) but we couldn't reproduce the behaviour. Only this two servers with a raidz pool are having this behaviour. Disabling compression had changed nothing. I ask for a helping hand in investigating the problem. Is there a way to get more information out of zfs or a debug logging? Do you have any other ideas? Below this you can find the config of the raidz pools. If i can provide more info please let me know how. Thanks in advance. Regards, Thorsten NAME PROPERTY VALUE SOURCE tank size 21,8T - tank capacity 69% - tank altroot - default tank health ONLINE - tank guid 4365850585010436054 default tank version - default tank bootfs - default tank delegation on default tank autoreplace off default tank cachefile - default tank failmode wait default tank listsnapshots off default tank autoexpand on local tank dedupditto 0 default tank dedupratio 1.00x - tank free 6,66T - tank allocated 15,1T - tank readonly off - tank comment - default tank expandsize 0 - tank freeing 0 default tank feature@async_destroy enabled local tank feature@empty_bpobj enabled local tank feature@lz4_compress enabled local pool: tank state: ONLINE scan: none requested config: NAME STATE READ WRITE CKSUM tank ONLINE 0 0 0 raidz1-0 ONLINE 0 0 0 aacd0 ONLINE 0 0 0 aacd1 ONLINE 0 0 0 aacd2 ONLINE 0 0 0 aacd3 ONLINE 0 0 0 aacd4 ONLINE 0 0 0 aacd5 ONLINE 0 0 0 NAME PROPERTY VALUE SOURCE tank type filesystem - tank creation Fr Feb 14 8:13 2014 - tank used 12,5T - tank available 5,25T - tank referenced 683M - tank compressratio 1.74x - tank mounted yes - tank quota none default tank reservation none default tank recordsize 128K default tank mountpoint /space local tank sharenfs off default tank checksum on default tank compression gzip-9 local tank atime on default tank devices on default tank exec on default tank setuid on default tank readonly off default tank jailed off default tank snapdir hidden default tank aclmode discard default tank aclinherit restricted default tank canmount on default tank xattr off temporary tank copies 1 default tank version 5 - tank utf8only off - tank normalization none - tank casesensitivity sensitive - tank vscan off default tank nbmand off default tank sharesmb off default tank refquota none default tank refreservation none default tank primarycache all default tank secondarycache all default tank usedbysnapshots 0 - tank usedbydataset 683M - tank usedbychildren 12,5T - tank usedbyrefreservation 0 - tank logbias latency default tank dedup off default tank mlslabel - tank sync standard default tank refcompressratio 1.94x - tank written 683M - tank logicalused 21,7T - tank logicalreferenced 1,28G - From owner-zfs-devel@FreeBSD.ORG Mon Aug 25 18:38:32 2014 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B3DE2BD4; Mon, 25 Aug 2014 18:38:32 +0000 (UTC) Received: from mail-wg0-x22d.google.com (mail-wg0-x22d.google.com [IPv6:2a00:1450:400c:c00::22d]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 2BF383CA8; Mon, 25 Aug 2014 18:38:32 +0000 (UTC) Received: by mail-wg0-f45.google.com with SMTP id x12so13301843wgg.4 for ; Mon, 25 Aug 2014 11:38:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=lD0XNGy5D/HvK/IdJ3VmbJ3AqymQWUim71EXttANTgg=; b=s0g+ur85HNhydSOGc28+RNwBOsjT7aW26o7qNr7eido11tNF/BwMOTjlxl4wF7col5 v4ZZhxaWTL5vXXqo3yIzg/4howDrdA17kXUlPy9UUul9HAE45NHuF0IJYoshQMpaT0Dj icmsXZg5kCDpL+sYqLSjfcQ72GB2YFEbLgiYrt0iD4PcuNpt/l8MXYK/kNmcoAQpOkMy qm/HS2Yx0HNiU7uJB0kOLTySFGgLKl/0puc2OnLxQJCgsBcNY0xkCuvbvoMEQP02cAki REZm6UG07+yjfITCWOC3aGtUMJw/N8BPISk/fjNiQCdHOBmvPDi7GntigBAZahMUygUg long== MIME-Version: 1.0 X-Received: by 10.194.58.148 with SMTP id r20mr25200244wjq.66.1408991910315; Mon, 25 Aug 2014 11:38:30 -0700 (PDT) Received: by 10.194.9.134 with HTTP; Mon, 25 Aug 2014 11:38:30 -0700 (PDT) Date: Mon, 25 Aug 2014 12:38:30 -0600 Message-ID: Subject: Introducing the ZFS test suite From: To: "freebsd-testing@freebsd.org" , "zfs-devel@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 25 Aug 2014 18:38:32 -0000 I just merged the ZFS test suite to projects/zfsd/head in change 270604. The ZFS test suite was originally written by Sun as part of the STF (Solaris test framework). They open sourced it in OpenSolaris, then HighCloud partially ported it to FreeBSD, and Spectra Logic finished the port. We also added 37 testcases, fixed many broken ones, and converted them all to the ATF framework. gibbs, will, ken, and myself have all worked on it. We've also taken a few contributions from araujo and Steven Hartland. The structure of the tests reflects their heritage as part of STF. Each ATF test case has its own file, for example tests/sys/cddl/zfs/tests/exec/exec_001_pos.ksh. Each directory that contains at least one test case has a separate atf-ksh93 test program, for example tests/sys/cddl/zfs/tests/exec/exec_tests.sh. The test programs are basically just wrappers around the test case files, but they still need to source some kshlib files, which is why they must use atf-ksh93 instead of atf-sh. atf-ksh93 is a hack; it simply sets the ATF_SHELL environment variable and execs atf-sh. Technically, this isn't guaranteed to work as the atf-sh shell functions aren't guaranteed to work in ksh93. In practice, however, they all do. The directories tests/sys/cddl/zfs/{include,bin} contain support files used by many different tests. The ZFS test suite still has some problems. Some of these are blocking it from being merged to head: * The atf-ksh93 hack. This is not a good thing to officially add to ATF, but it would be a lot of work to convert the test programs to run under sh. Especially because of the use of ksh arrays. * Many test cases reference Spectra Logic internal bug numbers. We need to create FreeBSD bugs for each of them that also affects FreeBSD. * It uses some ATF config variables that are not yet described in tests(7). Remember, don't run these tests on a production system. They WILL cause panics and deadlocks, and they may cause data loss too. Happy Hacking, -Alan From owner-zfs-devel@FreeBSD.ORG Mon Sep 8 14:23:49 2014 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id AF898C7E for ; Mon, 8 Sep 2014 14:23:49 +0000 (UTC) Received: from mail-qg0-f48.google.com (mail-qg0-f48.google.com [209.85.192.48]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6F1C213BA for ; Mon, 8 Sep 2014 14:23:48 +0000 (UTC) Received: by mail-qg0-f48.google.com with SMTP id z107so15203078qgd.21 for ; Mon, 08 Sep 2014 07:23:48 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc:content-type; bh=hUsccFMpjCxXOH4ZcP+an6ovTiZPOaF3hL6p+q4DSWI=; b=U6UeTxwQUri8YTx+LQQnFGG2o7nhM2wt2AfuFyaAdTWg0rw+lLIFOGdy/h57VfPOrg ZFiGUQrKE9oQCe4J+LF9BGywrU/BkDYHdOaO3kVjd1vt4nQTvVZbk9mt4Z3loYJBIpps 76BIkgOxmHA+9k5mNWK8MFufHql0m1Ds7ewGdMgrWMOAJA8HKrzZlRmv/HYlR6eZsRfS a18ZGZIlaVyLZGa/P4zp2VJBOH3RhddRtv8ulC3kRjWeyiNyGbNjQ4fGzXTsZJTvcwB6 TamBgn5Z+PZBjFiHpv6XkL5ogonl2bXOsPDrPm8VUaPuOWTi/Wt7BWQJ6oaKWAviEluf VjBA== X-Gm-Message-State: ALoCoQmXIZ1UmAfc/nnSOyQ8nzEuIpGAOwM7uSOvhaZA0hu6ZwRWzGMISYqWKW85W3uQv2DZiznc X-Received: by 10.224.88.3 with SMTP id y3mr42091202qal.65.1410186228036; Mon, 08 Sep 2014 07:23:48 -0700 (PDT) MIME-Version: 1.0 Sender: jmmv@meroh.net Received: by 10.96.83.99 with HTTP; Mon, 8 Sep 2014 07:23:27 -0700 (PDT) X-Originating-IP: [2620:0:1003:1007:6091:363d:d526:1ccd] In-Reply-To: References: From: Julio Merino Date: Mon, 8 Sep 2014 10:23:27 -0400 X-Google-Sender-Auth: bP0mtxfwFgV4lDSIbLx6h8TixyE Message-ID: Subject: Re: Introducing the ZFS test suite To: Alan Somers Content-Type: text/plain; charset=UTF-8 Cc: "freebsd-testing@freebsd.org" , "zfs-devel@freebsd.org" X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Sep 2014 14:23:49 -0000 On Mon, Aug 25, 2014 at 2:38 PM, wrote: > I just merged the ZFS test suite to projects/zfsd/head in change 270604. > [...] > The ZFS test suite still has some problems. Some of these are > blocking it from being merged to head: > * The atf-ksh93 hack. This is not a good thing to officially add to > ATF, but it would be a lot of work to convert the test programs to run > under sh. Especially because of the use of ksh arrays. As we discussed, agreed that this is a bit hackish, but I think this is a reasonable compromise and the hack is unlikely to break anytime soon. (Haven't looked at the specifics that were implemented though.) Therefore, I would personally not consider a "rewrite in sh" to be a blocker for merging into head. > Remember, don't run these tests on a production system. What's the plan for this? Are the tests somehow disabled by default so that a user running the full test suite is not tripped by this? Or will all the panics and deadlocks be fixed before merging to head? Thanks for doing this! From owner-zfs-devel@FreeBSD.ORG Mon Sep 8 14:59:48 2014 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D3796F14; Mon, 8 Sep 2014 14:59:48 +0000 (UTC) Received: from mail-wi0-x22b.google.com (mail-wi0-x22b.google.com [IPv6:2a00:1450:400c:c05::22b]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 24726198C; Mon, 8 Sep 2014 14:59:47 +0000 (UTC) Received: by mail-wi0-f171.google.com with SMTP id hi2so2826223wib.16 for ; Mon, 08 Sep 2014 07:59:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=+BqMXuuKdUEV2WGcQuIMmgPZXRFA612jjVcmWwlMgG8=; b=TroSHEeFgoCHuVPkDiN2adDZDi7VCO5KkP6FNMpynFSJ4UfRiYGcToVPCB3nM1JVUT UwBhdIA1/d8ZO/98SlTY3DuxizC3UhrcM9wwfF/GZcBsy0JvO8QoK58cjo1MPAkhW94O 48NTR+c65BwMmivrHUXTGwo8y9afjTBjGgaz3ZWQoh9tXwyOkpUfz0Crdw2yNrE/06nL iUmRiDdTbYn64T0wi4dwCB2pEwKYkruAAiPty4e7eEbbvYFJJluaKCGhNLB+4G+59WHU 4Oq9TtFFF3wjiQvyQJjU3NbstpHv581PkFnhDGSkpkGzlaz72JuQBfORSS+1rUduB4Yg mLyg== MIME-Version: 1.0 X-Received: by 10.180.89.36 with SMTP id bl4mr14592297wib.50.1410188386323; Mon, 08 Sep 2014 07:59:46 -0700 (PDT) Sender: asomers@gmail.com Received: by 10.194.126.1 with HTTP; Mon, 8 Sep 2014 07:59:46 -0700 (PDT) In-Reply-To: References: Date: Mon, 8 Sep 2014 08:59:46 -0600 X-Google-Sender-Auth: VVEY3AyMNS-Hn4_3uJwQjAVy76c Message-ID: Subject: Re: Introducing the ZFS test suite From: Alan Somers To: Julio Merino Content-Type: text/plain; charset=UTF-8 Cc: "freebsd-testing@freebsd.org" , "zfs-devel@freebsd.org" X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Sep 2014 14:59:48 -0000 On Mon, Sep 8, 2014 at 8:23 AM, Julio Merino wrote: > On Mon, Aug 25, 2014 at 2:38 PM, wrote: >> I just merged the ZFS test suite to projects/zfsd/head in change 270604. >> [...] >> The ZFS test suite still has some problems. Some of these are >> blocking it from being merged to head: >> * The atf-ksh93 hack. This is not a good thing to officially add to >> ATF, but it would be a lot of work to convert the test programs to run >> under sh. Especially because of the use of ksh arrays. > > As we discussed, agreed that this is a bit hackish, but I think this > is a reasonable compromise and the hack is unlikely to break anytime > soon. (Haven't looked at the specifics that were implemented though.) > Therefore, I would personally not consider a "rewrite in sh" to be a > blocker for merging into head. Good. That certainly reduces the workload. > > > Remember, don't run these tests on a production system. > > What's the plan for this? > > Are the tests somehow disabled by default so that a user running the > full test suite is not tripped by this? Currently, most if not all of the tests reference the test_suites.FreeBSD.disks variable. We should add it to require.configs, and then none of them will run by default. The bigger problem, though, is that even if you define that variable, many disks will destroy all pools on cleanup unless you also define the keep_pools variable. It's part of the default_cleanup_noexit function in libtests.kshlib. That must be fixed. > > Or will all the panics and deadlocks be fixed before merging to head? I doubt it. I certainly don't want to wait for it. But merging these tests to head will certainly help to focus attention on those bugs. > > Thanks for doing this! From owner-zfs-devel@FreeBSD.ORG Mon Sep 8 15:07:51 2014 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id EA652B33; Mon, 8 Sep 2014 15:07:51 +0000 (UTC) Received: from smtp1.multiplay.co.uk (smtp1.multiplay.co.uk [85.236.96.35]) by mx1.freebsd.org (Postfix) with ESMTP id ABDC81AB5; Mon, 8 Sep 2014 15:07:51 +0000 (UTC) Received: by smtp1.multiplay.co.uk (Postfix, from userid 65534) id 6E7F320E70894; Mon, 8 Sep 2014 15:07:49 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.multiplay.co.uk X-Spam-Level: * X-Spam-Status: No, score=1.8 required=8.0 tests=AWL,BAYES_05,DOS_OE_TO_MX, FSL_HELO_NON_FQDN_1,RDNS_DYNAMIC,STOX_REPLY_TYPE autolearn=no version=3.3.1 Received: from r2d2 (82-69-141-170.dsl.in-addr.zen.co.uk [82.69.141.170]) by smtp1.multiplay.co.uk (Postfix) with ESMTP id E982720E70885; Mon, 8 Sep 2014 15:07:43 +0000 (UTC) Message-ID: From: "Steven Hartland" To: "Alan Somers" , "Julio Merino" References: Subject: Re: Introducing the ZFS test suite Date: Mon, 8 Sep 2014 16:07:48 +0100 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="iso-8859-1"; reply-type=original Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.5931 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.6157 Cc: freebsd-testing@freebsd.org, zfs-devel@freebsd.org X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Sep 2014 15:07:52 -0000 ----- Original Message ----- From: "Alan Somers" snip... >> Or will all the panics and deadlocks be fixed before merging to head? > > I doubt it. I certainly don't want to wait for it. But merging these > tests to head will certainly help to focus attention on those bugs. Absolutely the sooner thes gets to head the sooner we can start attacking those bugs :) Regards Steve From owner-zfs-devel@FreeBSD.ORG Wed Sep 17 08:53:48 2014 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 90AA6CAB; Wed, 17 Sep 2014 08:53:48 +0000 (UTC) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 493D8E9C; Wed, 17 Sep 2014 08:53:43 +0000 (UTC) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id LAA25118; Wed, 17 Sep 2014 11:53:42 +0300 (EEST) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1XUAzd-00090k-Oy; Wed, 17 Sep 2014 11:53:41 +0300 Message-ID: <54194BDD.3000306@FreeBSD.org> Date: Wed, 17 Sep 2014 11:52:45 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:31.0) Gecko/20100101 Thunderbird/31.1.0 MIME-Version: 1.0 To: Xin LI , "Justin T. Gibbs" , Steven Hartland Subject: r268075 (MFV r267565) undone FreeBSD-specific changes Content-Type: text/plain; charset=X-VIET-VPS Content-Transfer-Encoding: 8bit Cc: zfs-devel@FreeBSD.org X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Sep 2014 08:53:48 -0000 Looking at the following commits: r268075 MFV r267565 r269086 I think that the merge of the illumos change has overridden a FreeBSD-specific change. Here is a diff hunk to illustrate the point: @@ -5198,7 +5205,13 @@ l2arc_compress_buf(l2arc_buf_hdr_t *l2hdr) len = l2hdr->b_asize; cdata = zio_data_buf_alloc(len); csize = zio_compress_data(ZIO_COMPRESS_LZ4, l2hdr->b_tmp_cdata, - cdata, l2hdr->b_asize, (size_t)(1ULL << l2hdr->b_dev->l2ad_vdev->vdev_ashift)); + cdata, l2hdr->b_asize); + + rounded = P2ROUNDUP(csize, (size_t)SPA_MINBLOCKSIZE); + if (rounded > csize) { + bzero((char *)cdata + csize, rounded - csize); + csize = rounded; + } if (csize == 0) { Note that we used to pass 1ULL << l2hdr->b_dev->l2ad_vdev->vdev_ashift as minblocksize, but now rounding up is done to a multiple of SPA_MINBLOCKSIZE. minblocksize parameter to zio_compress_data was a FreeBSD-specific change introduced in r254591 by Justin. Passing 1ULL << l2hdr->b_dev->l2ad_vdev->vdev_ashift as minblocksize in l2arc_compress_buf was introduced by Steven in r256889. So, it seems that r256889 is definitely undone now. I have not checked if there is any other fallout from removing minblocksize parameter of zio_compress_data(). Hmm, in fact, it looks like zio_compress_data() in zio_write_bp_init() has suffered the same fate: whereas previously minblocksize was set metaslab_class_get_minblocksize(mc), now SPA_MINBLOCKSIZE is used for rounding up. I do not have any 4K drives around at the moment, so I can not test what the practical consequences are. I guess it's probably just worse performance on 4KB physical / 512B logical disks because of RMW. It would lead to I/O errors on "pure" 4KB disks. -- Andriy Gapon From owner-zfs-devel@FreeBSD.ORG Wed Sep 17 09:29:54 2014 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E9162912; Wed, 17 Sep 2014 09:29:54 +0000 (UTC) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id AC5B52BD; Wed, 17 Sep 2014 09:29:53 +0000 (UTC) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id MAA25578; Wed, 17 Sep 2014 12:29:51 +0300 (EEST) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1XUBYd-00092Z-Dk; Wed, 17 Sep 2014 12:29:51 +0300 Message-ID: <54195457.4090004@FreeBSD.org> Date: Wed, 17 Sep 2014 12:28:55 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:31.0) Gecko/20100101 Thunderbird/31.1.0 MIME-Version: 1.0 To: Xin LI , "Justin T. Gibbs" , Steven Hartland Subject: possible memory corruption prior to r268075 [Was: r268075 (MFV r267565) undone FreeBSD-specific changes] References: <54194BDD.3000306@FreeBSD.org> In-Reply-To: <54194BDD.3000306@FreeBSD.org> Content-Type: text/plain; charset=X-VIET-VPS Content-Transfer-Encoding: 8bit Cc: zfs-devel@FreeBSD.org X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Sep 2014 09:29:55 -0000 While looking at the changes discussed in the previous email another issue has caught my attention. Both l2arc_compress_buf() and zio_write_bp_init() allocate a buffer for compressed data using an original / logical buffer size for the size. The buffer allocation is done using zio_buf_alloc() and zio_data_buf_alloc(). So, the size would be increased to one of the standard buffer sizes if needed. The standard buffer sizes are: 512B - 4KB with a step of 512B, 4KB - 8KB with a step of 1KB, 8KB - 16KB with a step of 2KB, 16KB - 128KB with a step of 4KB. The size for compressed data should always be sufficient because the compressed data is supposed to never be larger than the original data even taking into account rounding up to a multiple of 512B. In other words, psize <= lsize. But in FreeBSD prior to commit r268075 we used to round up the compressed size to a multiple of vdev sector size. Let's consider the following example. Original lsize is 10KB, so that data is stored in a 10KB buffer, because this is one of the standard buffer sizes. Thus, a 10KB buffer would also be allocated for the compressed data. Let's assume that the data gets compressed to 8KB + 1B size (reduction by almost 20%). Let's further assume that the sector size is 4KB. So the compressed size would be rounded up to 12KB. That is, psize > lsize. The code zeroes out a trailing part of the buffer, so it would overwrite 2KB beyond the end of the buffer. I think that in general our code had and may still has a bit of confusion about psize (possibly compressed data size) and asize (actual disk allocation size). I think that psize should always be not greater than lsize. While asize is always not less than psize. psize <= lsize psize <= asize Obviously, there is no rule on the relation between lsize and asize. But our code used to force ashift on psize which made it kind of asize, or something in between the classic psize and asize. What do you think? -- Andriy Gapon From owner-zfs-devel@FreeBSD.ORG Wed Sep 17 10:53:05 2014 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E9296ABD; Wed, 17 Sep 2014 10:53:05 +0000 (UTC) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id AF0B4C9B; Wed, 17 Sep 2014 10:53:04 +0000 (UTC) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id NAA26609; Wed, 17 Sep 2014 13:53:03 +0300 (EEST) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1XUCr9-000977-17; Wed, 17 Sep 2014 13:53:03 +0300 Message-ID: <541967D7.4080906@FreeBSD.org> Date: Wed, 17 Sep 2014 13:52:07 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:31.0) Gecko/20100101 Thunderbird/31.1.0 MIME-Version: 1.0 To: Xin LI , "Justin T. Gibbs" , Steven Hartland Subject: Re: r268075 (MFV r267565) undone FreeBSD-specific changes References: <54194BDD.3000306@FreeBSD.org> In-Reply-To: <54194BDD.3000306@FreeBSD.org> Content-Type: text/plain; charset=X-VIET-VPS Content-Transfer-Encoding: 8bit Cc: zfs-devel@FreeBSD.org X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Sep 2014 10:53:06 -0000 On 17/09/2014 11:52, Andriy Gapon wrote: > So, it seems that r256889 is definitely undone now. I have not checked if there > is any other fallout from removing minblocksize parameter of > zio_compress_data(). Hmm, in fact, it looks like zio_compress_data() in > zio_write_bp_init() has suffered the same fate: whereas previously minblocksize > was set metaslab_class_get_minblocksize(mc), now SPA_MINBLOCKSIZE is used for > rounding up. After more analysis I think that the changes in zio_write_bp_init() are correct. psize needs to be only SPA_MINBLOCKSIZE aligned. The code in zio_vdev_io_start() will take care of adjusting for ashift. On the other hand, the change in l2arc_compress_buf() is definitely incorrect. l2arc code uses physical ZFS I/O and as such it must correctly set data size and data buffer. -- Andriy Gapon From owner-zfs-devel@FreeBSD.ORG Wed Sep 17 16:33:37 2014 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 5C57D99D; Wed, 17 Sep 2014 16:33:37 +0000 (UTC) Received: from aslan.scsiguy.com (mail.scsiguy.com [70.89.174.89]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 0F3AE67B; Wed, 17 Sep 2014 16:33:36 +0000 (UTC) Received: from [192.168.6.136] (slboulder.spectralogic.com [192.30.190.3] (may be forged)) (authenticated bits=0) by aslan.scsiguy.com (8.14.9/8.14.9) with ESMTP id s8HGXWE2040097 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NO); Wed, 17 Sep 2014 10:33:34 -0600 (MDT) (envelope-from gibbs@FreeBSD.org) Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.6\)) Subject: Re: possible memory corruption prior to r268075 [Was: r268075 (MFV r267565) undone FreeBSD-specific changes] From: "Justin T. Gibbs" In-Reply-To: <54195457.4090004@FreeBSD.org> Date: Wed, 17 Sep 2014 10:33:25 -0600 Content-Transfer-Encoding: quoted-printable Message-Id: <64AEC518-2DE6-4AB0-A2B3-3BFD5E385BB6@FreeBSD.org> References: <54194BDD.3000306@FreeBSD.org> <54195457.4090004@FreeBSD.org> To: Andriy Gapon X-Mailer: Apple Mail (2.1878.6) Cc: Xin LI , zfs-devel@FreeBSD.org X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Sep 2014 16:33:37 -0000 On Sep 17, 2014, at 3:28 AM, Andriy Gapon wrote: >=20 > While looking at the changes discussed in the previous email another = issue has > caught my attention. > Both l2arc_compress_buf() and zio_write_bp_init() allocate a buffer = for > compressed data using an original / logical buffer size for the size. = The > buffer allocation is done using zio_buf_alloc() and = zio_data_buf_alloc(). So, > the size would be increased to one of the standard buffer sizes if = needed. The > standard buffer sizes are: 512B - 4KB with a step of 512B, 4KB - 8KB = with a step > of 1KB, 8KB - 16KB with a step of 2KB, 16KB - 128KB with a step of = 4KB. > The size for compressed data should always be sufficient because the = compressed > data is supposed to never be larger than the original data even taking = into > account rounding up to a multiple of 512B. In other words, psize <=3D = lsize. >=20 > But in FreeBSD prior to commit r268075 we used to round up the = compressed size > to a multiple of vdev sector size. There is no data corruption due to the earlier code that truncates d_len = such that at least a 12.5% compression can be achieved in a minblocksize = sized/align buffer: /* Compress at least 12.5% */ d_len =3D P2ALIGN(s_len - (s_len >> 3), minblocksize); if (d_len =3D=3D 0) { ZCOMPSTAT_BUMP(zcompstat_skipped_minblocksize); return (s_len); } d_len is strictly less than s_len, and c_len is tested to be <=3D d_len, = so the roundup that follows later cannot exceed the original buffer size = (s_len). r268075 lost this optimization to avoid attempting to compress something = that can never achieve even a 12.5% improvement due to minimum ashift. = This was done to support data embedded in block pointers. However, I = have no experience with how likely we are to compress 4K of data down to = 112 bytes (a 98% compression ratio). Perhaps it=92s quite likely for = metadata, but not so likely for user data? If so, we should restore the = old check for user data blocks to avoid useless consumption of CPU. > I think that in general our code had and may still has a bit of = confusion about > psize (possibly compressed data size) and asize (actual disk = allocation size). > I think that psize should always be not greater than lsize. While = asize is > always not less than psize. > psize <=3D lsize > psize <=3D asize > Obviously, there is no rule on the relation between lsize and asize. > But our code used to force ashift on psize which made it kind of = asize, or > something in between the classic psize and asize. Where did we force this? You mean in the 12.5% compression test? Or = the roundup/zero fill during compression? =97 Justin= From owner-zfs-devel@FreeBSD.ORG Thu Sep 18 08:44:04 2014 Return-Path: Delivered-To: zfs-devel@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B7B2A14E; Thu, 18 Sep 2014 08:44:04 +0000 (UTC) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 7326FE7; Thu, 18 Sep 2014 08:44:03 +0000 (UTC) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id LAA17237; Thu, 18 Sep 2014 11:44:00 +0300 (EEST) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1XUXJo-000DUT-65; Thu, 18 Sep 2014 11:44:00 +0300 Message-ID: <541A9AFF.5040903@FreeBSD.org> Date: Thu, 18 Sep 2014 11:42:39 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:31.0) Gecko/20100101 Thunderbird/31.1.0 MIME-Version: 1.0 To: "Justin T. Gibbs" , Xin LI Subject: Re: possible memory corruption prior to r268075 [Was: r268075 (MFV r267565) undone FreeBSD-specific changes] References: <54194BDD.3000306@FreeBSD.org> <54195457.4090004@FreeBSD.org> <64AEC518-2DE6-4AB0-A2B3-3BFD5E385BB6@FreeBSD.org> In-Reply-To: <64AEC518-2DE6-4AB0-A2B3-3BFD5E385BB6@FreeBSD.org> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 8bit Cc: zfs-devel@FreeBSD.org X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Sep 2014 08:44:04 -0000 On 17/09/2014 19:33, Justin T. Gibbs wrote: > On Sep 17, 2014, at 3:28 AM, Andriy Gapon wrote: > >> >> While looking at the changes discussed in the previous email another issue has >> caught my attention. >> Both l2arc_compress_buf() and zio_write_bp_init() allocate a buffer for >> compressed data using an original / logical buffer size for the size. The >> buffer allocation is done using zio_buf_alloc() and zio_data_buf_alloc(). So, >> the size would be increased to one of the standard buffer sizes if needed. The >> standard buffer sizes are: 512B - 4KB with a step of 512B, 4KB - 8KB with a step >> of 1KB, 8KB - 16KB with a step of 2KB, 16KB - 128KB with a step of 4KB. >> The size for compressed data should always be sufficient because the compressed >> data is supposed to never be larger than the original data even taking into >> account rounding up to a multiple of 512B. In other words, psize <= lsize. >> >> But in FreeBSD prior to commit r268075 we used to round up the compressed size >> to a multiple of vdev sector size. > > There is no data corruption due to the earlier code that truncates d_len such that at least a 12.5% compression can be achieved in a minblocksize sized/align buffer: > > /* Compress at least 12.5% */ > d_len = P2ALIGN(s_len - (s_len >> 3), minblocksize); When I looked at this line I saw the 12.5% thing but somehow I completely failed to see P2ALIGN(). Hence my confusion. Thank you for correcting me. > if (d_len == 0) { > ZCOMPSTAT_BUMP(zcompstat_skipped_minblocksize); > return (s_len); > } > > d_len is strictly less than s_len, and c_len is tested to be <= d_len, so the roundup that follows later cannot exceed the original buffer size (s_len). > > r268075 lost this optimization to avoid attempting to compress something that can never achieve even a 12.5% improvement due to minimum ashift. This was done to support data embedded in block pointers. However, I have no experience with how likely we are to compress 4K of data down to 112 bytes (a 98% compression ratio). Perhaps its quite likely for metadata, but not so likely for user data? If so, we should restore the old check for user data blocks to avoid useless consumption of CPU. Yes. So now the code that does rounding up after zio_compress_data() needs to be careful not to go beyond the buffer end. Xin, I think it will not be sufficient to simply replace SPA_MINBLOCKSIZE with 1 << l2hdr->b_dev->l2ad_vdev->vdev_ashift in l2arc_compress_buf(). Perhaps something like the following: @@ -5276,12 +5286,6 @@ l2arc_compress_buf(l2arc_buf_hdr_t *l2hdr) csize = zio_compress_data(ZIO_COMPRESS_LZ4, l2hdr->b_tmp_cdata, cdata, l2hdr->b_asize); - rounded = P2ROUNDUP(csize, (size_t)SPA_MINBLOCKSIZE); - if (rounded > csize) { - bzero((char *)cdata + csize, rounded - csize); - csize = rounded; - } - if (csize == 0) { /* zero block, indicate that there's nothing to write */ zio_data_buf_free(cdata, len); @@ -5290,11 +5294,19 @@ l2arc_compress_buf(l2arc_buf_hdr_t *l2hdr) l2hdr->b_tmp_cdata = NULL; ARCSTAT_BUMP(arcstat_l2_compress_zeros); return (B_TRUE); - } else if (csize > 0 && csize < len) { + } + + rounded = P2ROUNDUP(csize, + (size_t)1 << l2hdr->b_dev->l2ad_vdev->vdev_ashift); + if (rounded < len) { /* * Compression succeeded, we'll keep the cdata around for * writing and release it afterwards. */ + if (rounded > csize) { + bzero((char *)cdata + csize, rounded - csize); + csize = rounded; + } l2hdr->b_compress = ZIO_COMPRESS_LZ4; l2hdr->b_asize = csize; l2hdr->b_tmp_cdata = cdata; The idea is to use the compressed data only if its rounded up size is smaller than the original size. >> I think that in general our code had and may still has a bit of confusion about >> psize (possibly compressed data size) and asize (actual disk allocation size). >> I think that psize should always be not greater than lsize. While asize is >> always not less than psize. >> psize <= lsize >> psize <= asize >> Obviously, there is no rule on the relation between lsize and asize. >> But our code used to force ashift on psize which made it kind of asize, or >> something in between the classic psize and asize. > > Where did we force this? You mean in the 12.5% compression test? Or the roundup/zero fill during compression? I meant roundup with minblocksize when zio_compress_data() was called in zio_write_bp_init(). In that case psize didn't have to be a multiple of minblocksize. But on the other hand I think that no harm was done. -- Andriy Gapon From owner-zfs-devel@FreeBSD.ORG Thu Oct 16 17:04:30 2014 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 43ACE658 for ; Thu, 16 Oct 2014 17:04:30 +0000 (UTC) Received: from mail-wg0-x231.google.com (mail-wg0-x231.google.com [IPv6:2a00:1450:400c:c00::231]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id CFE4BFE for ; Thu, 16 Oct 2014 17:04:26 +0000 (UTC) Received: by mail-wg0-f49.google.com with SMTP id x12so4234130wgg.8 for ; Thu, 16 Oct 2014 10:04:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=date:from:to:subject:message-id:mime-version:content-type :content-disposition:user-agent; bh=4TrlVZdNP57E3VfhIy8NLswHZDK8xO7eh+JDXiiyKzc=; b=TPATA8z4ChfN1SWOvb3+oNy/TncHFrWQu0Z80bOtRnNGXPaG6HgZ1hLydN+KKK9EpW ZFA7W5SeNIllc7MyT1G/4llXrn6IsHyMpvOb3yZk1ZrVz2AVE0C8rMJU9OhwHy+KxspM ixsKYQZQecR/yCek+MpDv2+frc0NpfP4djdVoXfgZzg4XfJMW92q2Fk7K0ZyxfBRYVdC B2fZ6HI5wIJsmSpOBhQAKSTCWcWvj2UpJ7t185litkILmQOvX8phxsm2g+HHxeAhQLCV cHh69Qi/c4EXNZE5emvFk8L7nGJcxNbIFM76Zu20w1v+VZoe2tji3+USK8teHf+6liXL Ooog== X-Received: by 10.180.20.43 with SMTP id k11mr7804011wie.28.1413479065080; Thu, 16 Oct 2014 10:04:25 -0700 (PDT) Received: from dft-labs.eu (n1x0n-1-pt.tunnel.tserv5.lon1.ipv6.he.net. [2001:470:1f08:1f7::2]) by mx.google.com with ESMTPSA id pe8sm2659519wic.3.2014.10.16.10.04.23 for (version=TLSv1.2 cipher=RC4-SHA bits=128/128); Thu, 16 Oct 2014 10:04:24 -0700 (PDT) Date: Thu, 16 Oct 2014 19:04:21 +0200 From: Mateusz Guzik To: zfs-devel@freebsd.org Subject: weird lock contention problem during parallel writes to different files Message-ID: <20141016170421.GA24778@dft-labs.eu> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline User-Agent: Mutt/1.5.21 (2010-09-15) X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 16 Oct 2014 17:04:30 -0000 Hello, There is some weird stuff going on. I'm writing this down in case someone is interested in figuring out what's up. I run the same microbenchmark on an empty dataset. I remove files, but don't recreate the set. The test consists of 16 threads writing to different files in a loop. When the run is good, I get: https://people.freebsd.org/~mjg/zfs-write-contention/zfs-bad-run.svg But when it's bad...: https://people.freebsd.org/~mjg/zfs-write-contention/zfs-good-run.svg You can see test program here: https://people.freebsd.org/~mjg/zfs-write-contention/write-diff.c (As a side note, for vn_start_write & friends from good run see https://reviews.freebsd.org/D952 .) pool was created with: mdconfig -a -t malloc -s 10G zpool create zmeh /dev/md0 Test machine is: CPU: Intel(R) Xeon(R) CPU E5-2643 0 @ 3.30GHz (3300.07-MHz K8-class CPU) real memory = 34376515584 (32784 MB) avail memory = 33235132416 (31695 MB) FreeBSD/SMP: Multiprocessor System Detected: 16 CPUs FreeBSD/SMP: 2 package(s) x 4 core(s) x 2 SMT threads Console output is as follows: [16:50] fox2:/zmeh # /ufs/mjgtmp/write-diff -s 80 -n 16 test threads:16 sleep:80 pinned:1 total operations: 40295308, ops/sec 503691 40295308 [16:51] fox2:/zmeh # rm * zsh: sure you want to delete all the files in /zmeh [yn]? y [16:52] fox2:/zmeh # /ufs/mjgtmp/write-diff -s 80 -n 16 test threads:16 sleep:80 pinned:1 total operations: 19705569, ops/sec 246319 19705569 [16:53] fox2:/zmeh # rm * zsh: sure you want to delete all the files in /zmeh [yn]? y [16:53] fox2:/zmeh # /ufs/mjgtmp/write-diff -s 80 -n 16 test threads:16 sleep:80 pinned:1 total operations: 41942052, ops/sec 524275 41942052 [16:54] fox2:/zmeh # rm * zsh: sure you want to delete all the files in /zmeh [yn]? y [16:55] fox2:/zmeh # /ufs/mjgtmp/write-diff -s 80 -n 16 test threads:16 sleep:80 pinned:1 total operations: 19168019, ops/sec 239600 19168019 [16:56] fox2:/zmeh # rm * zsh: sure you want to delete all the files in /zmeh [yn]? y [16:56] fox2:/zmeh # /ufs/mjgtmp/write-diff -s 80 -n 16 test threads:16 sleep:80 pinned:1 total operations: 41453784, ops/sec 518172 41453784 [16:58] fox2:/zmeh # rm * zsh: sure you want to delete all the files in /zmeh [yn]? y [16:58] fox2:/zmeh # /ufs/mjgtmp/write-diff -s 80 -n 16 test threads:16 sleep:80 pinned:1 total operations: 18988638, ops/sec 237357 18988638 -- Mateusz Guzik From owner-zfs-devel@FreeBSD.ORG Thu Oct 16 17:11:15 2014 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 860BD83B for ; Thu, 16 Oct 2014 17:11:15 +0000 (UTC) Received: from mail-wg0-x233.google.com (mail-wg0-x233.google.com [IPv6:2a00:1450:400c:c00::233]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 1C88B1CB for ; Thu, 16 Oct 2014 17:11:14 +0000 (UTC) Received: by mail-wg0-f51.google.com with SMTP id b13so4246550wgh.22 for ; Thu, 16 Oct 2014 10:11:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=date:from:to:subject:message-id:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=WmqBI5TUOjRNdUqAZhnF7URf6BYIbkhtndaMKCcIzxE=; b=Gaj0GfaIZZGSfoKea81S++j2W9yC/NMpSW1E5IQEOAKCdaxcpwg5rVS1hWyF2MawyA k0lJdT0SMk2AacLK1ZaQ63THUTzbjkUmR4HLRdRNqJ1nBU4KfhR21yxM9DiVCiqk97Ph 4K+f987IeSVx1Mo8/ji1GlsSUtq5AuM/o457CL5EL5SIhBeI0DIlc9Nd4bTN5eGaeDcr FIDS1SLP2MfDsSyEPOJUPXazthfayOyNtt4hBo95GdpjCt1utB/rK1bGTJHWBPqCTB7X qyzVjXwlYTE2Fz5manyA6v4T4YuUL30+s/T1IfLeOP+EN+bZiXBR9vGYc74FoE7gTSTL qnJA== X-Received: by 10.180.104.199 with SMTP id gg7mr7670669wib.41.1413479473261; Thu, 16 Oct 2014 10:11:13 -0700 (PDT) Received: from dft-labs.eu (n1x0n-1-pt.tunnel.tserv5.lon1.ipv6.he.net. [2001:470:1f08:1f7::2]) by mx.google.com with ESMTPSA id l10sm2652914wif.20.2014.10.16.10.11.12 for (version=TLSv1.2 cipher=RC4-SHA bits=128/128); Thu, 16 Oct 2014 10:11:12 -0700 (PDT) Date: Thu, 16 Oct 2014 19:11:10 +0200 From: Mateusz Guzik To: zfs-devel@freebsd.org Subject: Re: weird lock contention problem during parallel writes to different files Message-ID: <20141016171110.GB24778@dft-labs.eu> References: <20141016170421.GA24778@dft-labs.eu> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: <20141016170421.GA24778@dft-labs.eu> User-Agent: Mutt/1.5.21 (2010-09-15) X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 16 Oct 2014 17:11:15 -0000 On Thu, Oct 16, 2014 at 07:04:21PM +0200, Mateusz Guzik wrote: > Hello, > > There is some weird stuff going on. I'm writing this down in case > someone is interested in figuring out what's up. > > I run the same microbenchmark on an empty dataset. I remove files, but > don't recreate the set. The test consists of 16 threads writing to > different files in a loop. > > When the run is good, I get: > https://people.freebsd.org/~mjg/zfs-write-contention/zfs-bad-run.svg > > But when it's bad...: > https://people.freebsd.org/~mjg/zfs-write-contention/zfs-good-run.svg > > You can see test program here: > https://people.freebsd.org/~mjg/zfs-write-contention/write-diff.c > > (As a side note, for vn_start_write & friends from good run see > https://reviews.freebsd.org/D952 .) > > pool was created with: > mdconfig -a -t malloc -s 10G > zpool create zmeh /dev/md0 > > Test machine is: > CPU: Intel(R) Xeon(R) CPU E5-2643 0 @ 3.30GHz (3300.07-MHz K8-class CPU) > real memory = 34376515584 (32784 MB) > avail memory = 33235132416 (31695 MB) > FreeBSD/SMP: Multiprocessor System Detected: 16 CPUs > FreeBSD/SMP: 2 package(s) x 4 core(s) x 2 SMT threads > Forgot to note: it's head r273122 with invariants etc disabled. .svg files are flamegraphs, see http://www.brendangregg.com/FlameGraphs/cpuflamegraphs.html for more info. I generated them with: dtrace -x stackframes=100 -n 'profile-997 { @[stack()] = count(); } tick-60s { exit(0); }' -o out.kern_stacks while on tmpfs. and then cat out.kern_stacks | ./stackcollapse.pl | ./flamegraph.pl > out.svg You can find repo here: https://github.com/brendangregg/FlameGraph.git > Console output is as follows: > > [16:50] fox2:/zmeh # /ufs/mjgtmp/write-diff -s 80 -n 16 test > threads:16 sleep:80 pinned:1 > total operations: 40295308, ops/sec 503691 > 40295308 > [16:51] fox2:/zmeh # rm * > zsh: sure you want to delete all the files in /zmeh [yn]? y > [16:52] fox2:/zmeh # /ufs/mjgtmp/write-diff -s 80 -n 16 test > threads:16 sleep:80 pinned:1 > total operations: 19705569, ops/sec 246319 > 19705569 > [16:53] fox2:/zmeh # rm * > zsh: sure you want to delete all the files in /zmeh [yn]? y > [16:53] fox2:/zmeh # /ufs/mjgtmp/write-diff -s 80 -n 16 test > threads:16 sleep:80 pinned:1 > total operations: 41942052, ops/sec 524275 > 41942052 > [16:54] fox2:/zmeh # rm * > zsh: sure you want to delete all the files in /zmeh [yn]? y > [16:55] fox2:/zmeh # /ufs/mjgtmp/write-diff -s 80 -n 16 test > threads:16 sleep:80 pinned:1 > total operations: 19168019, ops/sec 239600 > 19168019 > [16:56] fox2:/zmeh # rm * > zsh: sure you want to delete all the files in /zmeh [yn]? y > [16:56] fox2:/zmeh # /ufs/mjgtmp/write-diff -s 80 -n 16 test > threads:16 sleep:80 pinned:1 > total operations: 41453784, ops/sec 518172 > 41453784 > [16:58] fox2:/zmeh # rm * > zsh: sure you want to delete all the files in /zmeh [yn]? y > [16:58] fox2:/zmeh # /ufs/mjgtmp/write-diff -s 80 -n 16 test > threads:16 sleep:80 pinned:1 > total operations: 18988638, ops/sec 237357 > 18988638 > > -- > Mateusz Guzik -- Mateusz Guzik From owner-zfs-devel@FreeBSD.ORG Mon Nov 3 00:44:48 2014 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7F6A248F for ; Mon, 3 Nov 2014 00:44:48 +0000 (UTC) Received: from mail-la0-x233.google.com (mail-la0-x233.google.com [IPv6:2a00:1450:4010:c03::233]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id F214FBB2 for ; Mon, 3 Nov 2014 00:44:47 +0000 (UTC) Received: by mail-la0-f51.google.com with SMTP id q1so8731230lam.38 for ; Sun, 02 Nov 2014 16:44:45 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=delphix.com; s=google; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=S7iQqTrQVK/hMYD7y+aka3wZqNgeYTj/9nD8MVdwmvg=; b=UAuSCdOrbhO2ovkCd04TqoMvsW3wMxgh5bubBzXodHhhl2ezWelUFX2+694nbA3if0 yUGmMVuVhLQsOio7EfEs0HYjKZQmOnqJoSOTPGbxnzJuLqBT28H1mg6kJh1ILf62B/Gd NwKhLKnIgm7iDGYP16zCotMUAwWHSw5/zCnXY= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=S7iQqTrQVK/hMYD7y+aka3wZqNgeYTj/9nD8MVdwmvg=; b=O+AZgioqmUVaU2GBJgo0JTgmWZbDd8ixgppiOBNcHptyemULo7p79/d6z49AOQ8gIz cXOsoRhVojH1zpx+9uZy7vXbe5EdNQdfafvW2uSDGmZBwv8Ac2JNFdJp+IHDbko2izj3 tyZchS9WLPy4keT+9A3f4JjXB9EtlLGW1A1whIu40ZN4hMHS/O1B1Uk6zn1ZJFfFQIi0 gpvTDrX/Itq8ItizM/G+rRhTM5oN81/tSaUKNVYc6zFzgeIGowrAjFidegK6GCvsk9Rv u3f8lHUFLHKuKhM0bYTucUVGeS1AePB59Ntx0z6IBwqiloosN8mYUToGt5Y/w21IKWXC klXQ== X-Gm-Message-State: ALoCoQke3TYgA4gGj8FW+PbqeH/klUi+rSX3ldFdtVEzhDxtjdCfgzVlLiF4IhGpJNMyu0H5fXCn MIME-Version: 1.0 X-Received: by 10.112.137.39 with SMTP id qf7mr46332711lbb.47.1414975485631; Sun, 02 Nov 2014 16:44:45 -0800 (PST) Received: by 10.25.155.77 with HTTP; Sun, 2 Nov 2014 16:44:45 -0800 (PST) In-Reply-To: <64AEC518-2DE6-4AB0-A2B3-3BFD5E385BB6@FreeBSD.org> References: <54194BDD.3000306@FreeBSD.org> <54195457.4090004@FreeBSD.org> <64AEC518-2DE6-4AB0-A2B3-3BFD5E385BB6@FreeBSD.org> Date: Sun, 2 Nov 2014 16:44:45 -0800 Message-ID: Subject: Re: possible memory corruption prior to r268075 [Was: r268075 (MFV r267565) undone FreeBSD-specific changes] From: Matthew Ahrens To: "Justin T. Gibbs" Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 Cc: zfs-devel@freebsd.org, Xin LI X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Nov 2014 00:44:48 -0000 On Wed, Sep 17, 2014 at 9:33 AM, Justin T. Gibbs wrote: > On Sep 17, 2014, at 3:28 AM, Andriy Gapon wrote: > > > > > While looking at the changes discussed in the previous email another > issue has > > caught my attention. > > Both l2arc_compress_buf() and zio_write_bp_init() allocate a buffer for > > compressed data using an original / logical buffer size for the size. > The > > buffer allocation is done using zio_buf_alloc() and > zio_data_buf_alloc(). So, > > the size would be increased to one of the standard buffer sizes if > needed. The > > standard buffer sizes are: 512B - 4KB with a step of 512B, 4KB - 8KB > with a step > > of 1KB, 8KB - 16KB with a step of 2KB, 16KB - 128KB with a step of 4KB. > > The size for compressed data should always be sufficient because the > compressed > > data is supposed to never be larger than the original data even taking > into > > account rounding up to a multiple of 512B. In other words, psize <=3D > lsize. > > > > But in FreeBSD prior to commit r268075 we used to round up the > compressed size > > to a multiple of vdev sector size. > > There is no data corruption due to the earlier code that truncates d_len > such that at least a 12.5% compression can be achieved in a minblocksize > sized/align buffer: > > /* Compress at least 12.5% */ > d_len =3D P2ALIGN(s_len - (s_len >> 3), minblocksize); > if (d_len =3D=3D 0) { > ZCOMPSTAT_BUMP(zcompstat_skipped_minblocksize); > return (s_len); > } > > d_len is strictly less than s_len, and c_len is tested to be <=3D d_len, = so > the roundup that follows later cannot exceed the original buffer size > (s_len). > > r268075 lost this optimization to avoid attempting to compress something > that can never achieve even a 12.5% improvement due to minimum ashift. > This was done to support data embedded in block pointers. However, I hav= e > no experience with how likely we are to compress 4K of data down to 112 > bytes (a 98% compression ratio). Actually, there are several use cases where even 8K blocks can become embedded. For example, initialized but otherwise unused blocks of Oracle database files (probably other databases too). These blocks have a small header and footer on each block but are otherwise zero-filled. --matt > Perhaps it=E2=80=99s quite likely for metadata, but not so likely for use= r data? > If so, we should restore the old check for user data blocks to avoid > useless consumption of CPU. > > > I think that in general our code had and may still has a bit of > confusion about > > psize (possibly compressed data size) and asize (actual disk allocation > size). > > I think that psize should always be not greater than lsize. While asiz= e > is > > always not less than psize. > > psize <=3D lsize > > psize <=3D asize > > Obviously, there is no rule on the relation between lsize and asize. > > But our code used to force ashift on psize which made it kind of asize, > or > > something in between the classic psize and asize. > > Where did we force this? You mean in the 12.5% compression test? Or the > roundup/zero fill during compression? > > =E2=80=94 > Justin > _______________________________________________ > zfs-devel@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/zfs-devel > To unsubscribe, send any mail to "zfs-devel-unsubscribe@freebsd.org" > From owner-zfs-devel@FreeBSD.ORG Thu Nov 6 22:22:19 2014 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id DB5AD396 for ; Thu, 6 Nov 2014 22:22:19 +0000 (UTC) Received: from mail-wg0-f48.google.com (mail-wg0-f48.google.com [74.125.82.48]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 70F543B5 for ; Thu, 6 Nov 2014 22:22:18 +0000 (UTC) Received: by mail-wg0-f48.google.com with SMTP id m15so2346915wgh.21 for ; Thu, 06 Nov 2014 14:22:10 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:message-id:date:from:user-agent:mime-version:to :cc:subject:content-type:content-transfer-encoding; bh=ivcRV/rxtn7wcV+3So5o8AOta08VYHaqSuUNUaL1GJE=; b=BXjplyAOgW15kcST9OcQkoEDI7A3S1cwlf1ES1p8bYME2CW/Wp5p8O5fjfWW4KhAGk brUE3Ja6RabuR1Z7VckFpeVV4WFRIl8uKmpVU0dFjReAurPhV8KGWiWI0/ZZI/RhHgvw VvQozaETzDR0mRmN/4Sg5HRuM92UkZsbGY44FDjMo8YoOHA0DU5xJos0eKfy26fhJ9Md eP+rgfyB3CIBMXpYO+WJuQW0k2QQPMQXa/W5fu4E17tunqmgCxVt58G9WlC2mNBew+VQ /Hl0MMoRiGsyp3OEwrjNvsq3SV4l9itf9lQRs5VuTiFGWs579OKY+J5/Bu4Fuu6ONYpy NLnQ== X-Gm-Message-State: ALoCoQlcf0MH+fr/k2DL/PfLPpGHT/OXljRj6d+kOqnucBjBbAuwBkA3TKTDjSUOUhn4OXft7Mc2 X-Received: by 10.180.85.198 with SMTP id j6mr14561597wiz.23.1415312530828; Thu, 06 Nov 2014 14:22:10 -0800 (PST) Received: from [10.10.1.68] (82-69-141-170.dsl.in-addr.zen.co.uk. [82.69.141.170]) by mx.google.com with ESMTPSA id d14sm9439887wjz.26.2014.11.06.14.22.09 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 06 Nov 2014 14:22:09 -0800 (PST) Message-ID: <545BF440.6070706@multiplay.co.uk> Date: Thu, 06 Nov 2014 22:20:48 +0000 From: Steven Hartland User-Agent: Mozilla/5.0 (Windows NT 5.1; rv:31.0) Gecko/20100101 Thunderbird/31.2.0 MIME-Version: 1.0 To: "developer@open-zfs.org" Subject: spa_config_update not calling spa_config_sync for root pools Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Cc: "zfs-devel@FreeBSD.org" X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Nov 2014 22:22:19 -0000 I've been looking at an issue a user reported where they believed the config of a log device added to their root pool had an incorrect ashift, which turned out to be incorrect data in zpool.cache. I've tracked this down to the fact that spa_config_update doesn't call spa_config_sync if the pool is the root pool: https://github.com/illumos/illumos-gate/blob/master/usr/src/uts/common/fs/zfs/spa_config.c#L539 This check was added by the old commit: https://github.com/illumos/illumos-gate/commit/e7cbe64f7a72dae5cb44f100db60ca88f3313c65 Back then spa_config_sync didn't take any args, so I'm wondering we can now remove the if (!spa->spa_is_root) restriction to ensure that stale information in the zpool.cache is correctly updated when operating on the root pool? Regards Steve From owner-zfs-devel@FreeBSD.ORG Tue Nov 11 06:48:47 2014 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0F7FA763 for ; Tue, 11 Nov 2014 06:48:47 +0000 (UTC) Received: from mail-yh0-x234.google.com (mail-yh0-x234.google.com [IPv6:2607:f8b0:4002:c01::234]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C2487C56 for ; Tue, 11 Nov 2014 06:48:46 +0000 (UTC) Received: by mail-yh0-f52.google.com with SMTP id f73so3109688yha.11 for ; Mon, 10 Nov 2014 22:48:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=hNV/NtdzHMRrTZNcWtALXpQHZeHA4b7XZJJ572Ey5e8=; b=tEpbItkOzzdarUKIv32Tqs7eijVsari/NnZ9Ms7L4Gn1GGPCDkQ0H0JfVicUzpzlaU x6RfXleQX0Zoy64f6/4MN0qf3DBI0b7D6SUhiL804HGmx8CKNyTdGG/5R2Gj6mwOaTHI tQm1QnYMOyeJCJpn4OhCWhk7sAgFjcnBQQSTHSPO9nG8k3ph07wjbYE1DFLCsVKRCEzr UNY90D4Y6F8aSCcQEP9ZwS4rPRop1Jw826t78TNAxqv/3oKrjVZT8Kev3RjolbdlRs+h /E3RihuNKJjJa6OMNd6uEZise3ZxTT7Z9XaSFWlQItO+oZ8rdSTMaxJYZ9Gq1Tgt0jBu AKiA== MIME-Version: 1.0 X-Received: by 10.52.234.230 with SMTP id uh6mr21196945vdc.10.1415688525965; Mon, 10 Nov 2014 22:48:45 -0800 (PST) Received: by 10.31.53.139 with HTTP; Mon, 10 Nov 2014 22:48:45 -0800 (PST) Date: Tue, 11 Nov 2014 12:18:45 +0530 Message-ID: Subject: A reliable way to change the zpool guid and vdev uuid? From: Jyoti Sharma To: zfs-devel@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 11 Nov 2014 06:48:47 -0000 We support ZFS filesystems ( Zpools really ) with our IntelliSnap hardware snapshot technology. As part of this we need to read and often modify the zpool guid and vdev uuids of the disks involved. Recently we've seen that as our operations that change the zpool guid and vdev uuid no longer work for zpools which are @ the latest version of the on disk format. As a result of this our import operations on the pools fail. The sequence of operations we perform are as follows : 1. Take a hardware snapshot of ZFS disks. 2. Expose the snapshot LUNs to a different / same box. 3. Read and manipulate on-disk ids of the snapshot luns to be unique. 4. Rename the pool by writing to on-disk structures. 5. Import the pool on the host with a new name. With the more recent version of ZPools we see that operation 5 fails. If we simply use the zpool import command without running steps 3,4 step 5 works. But this is not desirable because as part of our backup operations, at any given time we can import : a. copy of the source pool to the same host ----OR------ b. multiple copies of the source pool on the same host at a time. We've used knowledge gained from zfs ondisk specs available online for our implementation and these ideas have worked for us until now. In short we suspect a disk format change in the latest or the recent versions of zfs that affect the on-disk placement of those ids. We are looking for a documented or supported way to change the ids reliably w/o relying on on-disk format information in order to keep our codebase future compliant. Thank you. From owner-zfs-devel@FreeBSD.ORG Fri Feb 20 20:15:03 2015 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 916D1EDD for ; Fri, 20 Feb 2015 20:15:03 +0000 (UTC) Received: from mail-ob0-x232.google.com (mail-ob0-x232.google.com [IPv6:2607:f8b0:4003:c01::232]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 56C8D150 for ; Fri, 20 Feb 2015 20:15:03 +0000 (UTC) Received: by mail-ob0-f178.google.com with SMTP id uz6so26400798obc.9 for ; Fri, 20 Feb 2015 12:15:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type :content-transfer-encoding; bh=vSZYECQZLBmJicm0sJoN5egoCVWj6S2EAdNXZciZKAY=; b=AfEPJuoPi4SYzfW6HPfLyXOefIXOdoO6lAsMMo/fS4J6trhz1KK2CnI/Ia0YujFjfB FKWaAmugeDXlTuWADLyXFT1jpholI+XfYDtTbqK/4vm+nYjMaH82Xb52x/asZ5v2L1AY p6rKNumRZwIAi3g45pUaD5m+eH5t9LcLslUtdhQ8ZMuv0qvLQbyzkfbO5fCrNDVhZ0h7 mKvuASD0u2Vcpx4GTAJGIA8FoWRkulv4ZKCSA8iM/l6xlrVC9Lp2nSSByk6OHYncB7mH UrWK/Jh2LSqSGKpExT4sg4JHGM/mIq5JgcH6eu54BwMR/FIpdExeer2E8ukbIHFG0pjx 25kQ== MIME-Version: 1.0 X-Received: by 10.60.96.167 with SMTP id dt7mr7729131oeb.54.1424463302653; Fri, 20 Feb 2015 12:15:02 -0800 (PST) Received: by 10.60.140.199 with HTTP; Fri, 20 Feb 2015 12:15:02 -0800 (PST) Date: Fri, 20 Feb 2015 15:15:02 -0500 Message-ID: Subject: Zoned Commands ZBS/ZAC, Shingled SMR/SFS, ZFS From: grarpamp To: freebsd-fs@freebsd.org Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Mailman-Approved-At: Fri, 20 Feb 2015 20:43:07 +0000 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Feb 2015 20:15:03 -0000 These may be of interest for possible integration... https://github.com/hgst/libzbc https://github.com/Seagate/SMR_FS-EXT4 Panel: Shingled Disk Drives=E2=80=94File System Vs. Autonomous Block Device http://storageconference.us/2014/Presentations/Panel4.Bandic.pdf http://storageconference.us/2014/Presentations/Panel4.Amer.pdf http://storageconference.us/2014/Presentations/Panel4.Novak.pdf ZFS on SMR Drives: Enabling Shingled Magnetic Recording (SMR) for Enterprises http://storageconference.us/2014/Presentations/Novak.pdf http://storageconference.us/2014/index.html http://storageconference.us/history.html From owner-zfs-devel@FreeBSD.ORG Fri Feb 20 21:02:59 2015 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C53FB6B3; Fri, 20 Feb 2015 21:02:59 +0000 (UTC) Received: from mail-oi0-x232.google.com (mail-oi0-x232.google.com [IPv6:2607:f8b0:4003:c06::232]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 926089A4; Fri, 20 Feb 2015 21:02:59 +0000 (UTC) Received: by mail-oi0-f50.google.com with SMTP id v1so4752688oia.9; Fri, 20 Feb 2015 13:02:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=0ZAhxCHvcwnEpkwf8mJ+siqh9/qJh+tljISgSgMbDbY=; b=HEyZonFIwzS44uopcQtXl2QaicqeShdXxYHbn9z0HFDlY0BePak3VkjO/rIYdezLzF ndOCit/2RaVP/4DBtrBxO+td6WAbINlXmQ47Ziqy660WR00kyVPqI2/k/73H4GM0bhZq /zHF2Z6xLOWoIOpFchwgQScughPDt/N4V/0x8iM1NDmQsVk3W7oqLL7lworpWzJCj23G lyv47tyenPn+pRaJki2Vw3O3V1wxAE5hlAk25UC9PUiZcF2wphS3KRD8hKRdqOhSLJxR 2iWGDAxViXG+hz7yhJ6B8w3BMPFHKUT70vgTjifnmAJL80MAUYvNeW0v2rEMjO4b3rjw mXsw== MIME-Version: 1.0 X-Received: by 10.60.102.41 with SMTP id fl9mr7847674oeb.33.1424466178584; Fri, 20 Feb 2015 13:02:58 -0800 (PST) Received: by 10.60.140.199 with HTTP; Fri, 20 Feb 2015 13:02:58 -0800 (PST) In-Reply-To: References: Date: Fri, 20 Feb 2015 16:02:58 -0500 Message-ID: Subject: Re: Zoned Commands ZBS/ZAC, Shingled SMR/SFS, ZFS From: grarpamp To: freebsd-fs@freebsd.org Content-Type: text/plain; charset=UTF-8 Cc: zfs-devel@freebsd.org X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Feb 2015 21:02:59 -0000 > On freebsd-fs: > ZFS on SMR Drives: Enabling Shingled Magnetic Recording (SMR) for Enterprises > http://storageconference.us/2014/Presentations/Novak.pdf Further references... ZFS Host-Aware_SMR http://open-zfs.org/w/images/2/2a/Host-Aware_SMR-Tim_Feldman.pdf libzbc - The Linux Foundation http://events.linuxfoundation.org/sites/events/files/slides/SMR-LinuxConUSA-2014.pdf http://www.snia.org/sites/default/files/Dunn-Feldman_SNIA_Tutorial_Shingled_Magnetic_Recording-r7_Final.pdf http://www.opencompute.org/wiki/Storage/Dev Initial ZAC support http://www.spinics.net/lists/linux-scsi/msg81545.html ZAC/ZBC Update http://www.spinics.net/lists/linux-scsi/msg80161.html From owner-zfs-devel@FreeBSD.ORG Thu Feb 26 11:30:42 2015 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A75BCF45; Thu, 26 Feb 2015 11:30:42 +0000 (UTC) Received: from mail-ie0-x230.google.com (mail-ie0-x230.google.com [IPv6:2607:f8b0:4001:c03::230]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 702F65F2; Thu, 26 Feb 2015 11:30:42 +0000 (UTC) Received: by iecrl12 with SMTP id rl12so13601181iec.2; Thu, 26 Feb 2015 03:30:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:cc:content-type; bh=ZhyJen6rMXNB3RtkmDfOykU6rXb0rH5KROASxcdWsvE=; b=Tg8yiwTPFwhvKA6k+UUUUJDVo/fY7TpDRagdKoRA7zPp/mCO/6fwdLnN0BYvZyONQQ aXNAFOiE9PSpZbS2qo7sZw35TPU626dfubNO/1T3EHM07+8ltr7b6egfEXIpiJsNdpyY nhKG69GGcgo8KNWPbMQZx3t/rmf8pKheUji6uVkVvpNjELDNu1uuXk0fmRMR6Gm8c+PE S8Ms10sumyktq1hu3nquFVT12pJR+uXhbsImw0tA/p+KQ/xSv1vQdU9h+miChaGrisv8 /ASMBHwO+xi4DmNm4bXoGgus0iuPDAYLkeXUXYXtkka8Qk3wZzHK5WXCllU/FPDfOTlw se+A== MIME-Version: 1.0 X-Received: by 10.50.62.110 with SMTP id x14mr33885594igr.2.1424950241690; Thu, 26 Feb 2015 03:30:41 -0800 (PST) Received: by 10.36.111.202 with HTTP; Thu, 26 Feb 2015 03:30:41 -0800 (PST) Date: Thu, 26 Feb 2015 06:30:41 -0500 Message-ID: Subject: Zoned Commands ZBC/ZAC, Shingled SMR drives, ZFS From: grarpamp To: freebsd-fs@freebsd.org Content-Type: text/plain; charset=UTF-8 Cc: developer@open-zfs.org, zfs-devel@freebsd.org X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Feb 2015 11:30:42 -0000 > On Wed, 25 Feb 2015 16:19 Shehbaz Jaffer : > I thought proposal was to implement FreeBSD - ZFS on SMR Drives. That's right. Same as Linux and EXT4 were both adapted for SMR. > 1. How will we emulate SMR Drives on QEMU? Given that you can request drive samples for development from feldman@seagate , I'd think the QEMU question is left to QEMU. > 2. How should we go about using open source SMR Drives and use > these in FreeBSD? Thanks for your interest, there are good people here to collaborate development with. Also, the overall OS agnostic home for ZFS is at http://www.open-zfs.org/ Have a look there as well. They have a pretty active ZFS list. There are enough links below to get people started. Are you still affiliated with NetApp as suggested by the sig in your first email? > On Wed, 25 Feb 2015 Mark Felder : > Do I understand the scope of this correctly: I'll try based on limited reading, refer to the links and video for authoritative answers. > - SMR drives are the future for increasing drive capacity So far, given spindle vs SSD applications, likely yes. Seagate just released 8TB top of their "Archive" line for $280. Western Digital Hitachi has a 10TB model in the queue. SMR seems to be something FreeBSD and ZFS would want to incorporate in order to further compete in the storage space. > - SMR drives are terrible at random IO Haven't found any benchmarks yet. There may be some in these presentations or over in Linux land. I think only random writes are currently performance caveated if you are not SMR host aware. These drives present three types of management and have huge caches. The host aware and filesystem mods are meant to address speed. Drives will do 150MB/s r/w avg. > - There is work in Linux to detect these SMR drives Yes, in their storage stacks. They work as a dumb non-optimized LBA device without it. > and alter ZFS' behavior Linux is enhancing EXT4. The presentations in the links hope to enhance ZFS as well. ZFS's copy on write seems well suited to SMR. I don't have anything to do with or know anything, just didn't initially see any FreeBSD + SMR + ZFS references out there. Have fun. ========= Newly added links ZFS Host-Aware_SMR https://www.youtube.com/watch?v=b1yqjV8qemU SMR Modifications to EXT4 (and other generic file systems) http://www.spinics.net/lists/linux-ext4/msg46868.html http://www.spinics.net/lists/linux-scsi/msg81950.html Linux has been adding support since their 3.18 kernel onwards... https://git.kernel.org/cgit/linux/kernel/git/hare/scsi-devel.git/log/?h=zac.v2 https://git.kernel.org/cgit/linux/kernel/git/next/linux-next.git/commit/?id=7a14c1c3319608154da8712e4174d56ffb2f7b8d https://git.kernel.org/cgit/linux/kernel/git/next/linux-next.git/commit/?id=9162c6579bf90b3f5ddb7e3a6c6fa946c1b4cbeb https://git.kernel.org/cgit/linux/kernel/git/next/linux-next.git/commit/?id=f9ca5ab832e7ac5bc2b6fe0e82ad46d536f436f9 Drive specs http://www.seagate.com/www-content/product-content/hdd-fam/seagate-archive-hdd/en-us/docs/archive-hdd-dS1834-3-1411us.pdf http://www.seagate.com/www-content/product-content/hdd-fam/seagate-archive-hdd/en-us/docs/100757960c.pdf http://www.hgst.com/science-of-storage/next-generation-data-centers/10tb-smr-helioseal-hdd ZBC/ZAC specs http://www.t10.org/members/w_zbc-.htm http://www.t13.org/Documents/MinutesDefault.aspx?keyword=di537 ========= Previous links for reference ZFS Host-Aware_SMR http://open-zfs.org/w/images/2/2a/Host-Aware_SMR-Tim_Feldman.pdf http://www.youtube.com/watch?v=b1yqjV8qemU ZFS on SMR Drives: Enabling Shingled Magnetic Recording (SMR) for Enterprises http://storageconference.us/2014/Presentations/Novak.pdf ZBC device manipulation library https://github.com/hgst/libzbc Seagate SMR Friendly File System - EXT4 https://github.com/Seagate/SMR_FS-EXT4 Initial ZAC support http://www.spinics.net/lists/linux-scsi/msg81545.html ZAC/ZBC Update http://www.spinics.net/lists/linux-scsi/msg80161.html libzbc - The Linux Foundation http://events.linuxfoundation.org/sites/events/files/slides/SMR-LinuxConUSA-2014.pdf Panel: Shingled Disk Drives - File System Vs. Autonomous Block Device / SFS http://storageconference.us/2014/Presentations/Panel4.Bandic.pdf http://storageconference.us/2014/Presentations/Panel4.Amer.pdf http://storageconference.us/2014/Presentations/Panel4.Novak.pdf http://storageconference.us/2014/index.html http://storageconference.us/history.html http://www.opencompute.org/wiki/Storage/Dev http://www.snia.org/sites/default/files/Dunn-Feldman_SNIA_Tutorial_Shingled_Magnetic_Recording-r7_Final.pdf ========= From owner-zfs-devel@FreeBSD.ORG Tue Mar 3 04:21:27 2015 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7DBB38C8; Tue, 3 Mar 2015 04:21:27 +0000 (UTC) Received: from mail-ig0-x22e.google.com (mail-ig0-x22e.google.com [IPv6:2607:f8b0:4001:c05::22e]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 47BB419B1; Tue, 3 Mar 2015 04:21:27 +0000 (UTC) Received: by igbhn18 with SMTP id hn18so23437899igb.2; Mon, 02 Mar 2015 20:21:26 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:cc:content-type; bh=IXSXecv9KtXcKX9M74BEx/DYMDcyVLtWAPnkj3e/Coc=; b=QvSBnhD5UqL81u39pth1z+nCALwl/tAE+qW2fXjp+k2DYUDWcm1BGb2RGnJMe+4uZ3 W6I8vS9QBwvFtJswet39uVIDIoPeGEB73YGleiuZ4yp4vwSP+GQ6INLyd0iIB4dDZI0c vzumGf0WwcWLVwFte1ewISenTMj2U2d5/W5WxiitUYc5S4A19HWZs6LU1oVxuzjt7cAa mvCZP2mzOZSISOAeorwndl5dzOvIpj/iMjZCeKAFb7QADmTqfnv8VRCSMf3UmUwIcEC7 fZiWFHU0KGmswFcegot9BVKsNamIHeIFgPKHej5NyS4nRS0oczjCmXwKDgLsNzUzzly+ GxYg== MIME-Version: 1.0 X-Received: by 10.42.28.199 with SMTP id o7mr34846394icc.23.1425356486708; Mon, 02 Mar 2015 20:21:26 -0800 (PST) Received: by 10.36.111.202 with HTTP; Mon, 2 Mar 2015 20:21:26 -0800 (PST) Date: Mon, 2 Mar 2015 23:21:26 -0500 Message-ID: Subject: Zoned Commands ZBC/ZAC, Shingled SMR drives, ZFS From: grarpamp To: freebsd-fs@freebsd.org Content-Type: text/plain; charset=UTF-8 Cc: developer@open-zfs.org, zfs-devel@freebsd.org X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 03 Mar 2015 04:21:27 -0000 Performance, internals, and alternatives with SMR drives. https://www.usenix.org/conference/fast15/technical-sessions/presentation/aghayev http://sssl.ccs.neu.edu/skylight/ Note that "drive managed" is first gen. "Host aware" seems the ideal future state for filesystems. Also the specialization / adaptation of marketed tech to fitting roles... SMR (archive), standard HDD (middle ground), SSD (db/oltp), and RAM (extreme). From owner-zfs-devel@FreeBSD.ORG Tue Apr 7 12:28:56 2015 Return-Path: Delivered-To: zfs-devel@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 2952281B for ; Tue, 7 Apr 2015 12:28:56 +0000 (UTC) Received: from mail-pd0-x22e.google.com (mail-pd0-x22e.google.com [IPv6:2607:f8b0:400e:c02::22e]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E91A18F0 for ; Tue, 7 Apr 2015 12:28:55 +0000 (UTC) Received: by pdea3 with SMTP id a3so77546670pde.3 for ; Tue, 07 Apr 2015 05:28:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :content-type; bh=yv6WE5LcqO982fVoT9BQ8RiMnxtKj5iZ0o2mDr6z8Cg=; b=oiEPeUe+bzdu7zX8GOeFkrdx1lbUca/U6oHjQya2Vph85V9D83yO/zNhOi1amuSFyp pxbRLp57JvsoNwk15fmXziEVhCExJwHhmhUKxi+6TNEmMw0AHdKHaznUOZWem7RuNOm8 oAkG8gXCvYmCBxgqGGLQdbYIJyiFDqduGbMbq6hJwExN9wYrg4/mVemlvCrPx25btC2y CEnidm69N2Yr5z+YidyZ0GcS99gxpArij/mG6M0Oa2aHtwueYpjufhuBKdjLSFyUpELq 2QJmB4ZP57KdzLUqfRcAroXoW+8FUXaVyuNgkZ1Vq2aABkfL+fKmXfFMu8T0XlBp3768 8vaA== X-Received: by 10.70.90.34 with SMTP id bt2mr36161226pdb.33.1428409735425; Tue, 07 Apr 2015 05:28:55 -0700 (PDT) MIME-Version: 1.0 Received: by 10.66.183.41 with HTTP; Tue, 7 Apr 2015 05:28:34 -0700 (PDT) In-Reply-To: References: From: Stacey Pellegrino Date: Tue, 7 Apr 2015 13:28:34 +0100 Message-ID: Subject: Fwd: PLEASE HELP .!.. ZFS root mirror with Gentoo Linux boot loader issues... To: zfs-devel@freebsd.org Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 07 Apr 2015 12:28:56 -0000 FreeBSD folks... please help (see below). Oh yeah... where is my mailing list subscription approval? I have been waiting ages! Please give me batch updates in one email daily (if at all possible) and confirm when set-up. Thanks (in advance). Regards, -stacepellegrino ---------- Forwarded message ---------- From: Stacey Pellegrino Date: 7 April 2015 at 13:25 Subject: Fwd: PLEASE HELP .!.. ZFS root mirror with Gentoo Linux boot loader issues... To: gentoo-dev@lists.gentoo.org Folks... please help. ---------- Forwarded message ---------- From: Stacey Pellegrino Date: 7 April 2015 at 12:39 Subject: PLEASE HELP .!.. ZFS root mirror with Gentoo Linux boot loader issues... To: bsd , recruiters , "Justin (jlec)" , Javier Villavicencio , zfs-discuss@zfsonlinux.org Hey folks! Everything with a ZFS root mirror is set-up sweet other than the boot loader with ZFS libs enabled on ebuilds when emerged. The latest Grub 2 beta with ZFS enabled in the USE flags just segmentation faults when grub2-probe / is run. Bringing in the latest Grub legacy ebuild and applying all the relevant changes for ZFS would segmentation fault too on a grub-probe / but would install to sda and sdb GPT labelled disks, which Grub 2 beta did not. I then figured that boot0 (the FreeBSD boot loader) might come to my rescue instead of Grub. However, I see that boot0 in the portage tree are of the bsd branch and at first glance seems incompatible with Linux without jumping through some serious hoops. The next option that came to me was using the Grub legacy on x86 Solaris, although both 11.2 and 10 (latest rev) didn't boot the live DVD on my AMD64 host, so I was unable to install the boot loader that is ZFS aware for Solaris and compatible with ZFS version 28. The next thing I figured was to create an ebuild for Gentoo Linux called grub-legacy that was the same source uploaded by Oracle regarding Grub 0.97. That would surely recognise /boot on ZFS mirrored across sda and sdb and boot the Gentoo Linux kernel? The next priority after that is to get beadm fully ported and working with root pool snapshots and the Grub boot menu so that this would facilitate easy rollback option just like it is in x86 Solaris. Thoughts/comments and forwarding this email onto other relevant parties of interest would be most welcome. Kind regards and all the best folks, -stacepellegrino P.S. Just would love to get the email address stacepellegrino@gentoo.org ...embarking on a community support mission to get recognition for that email address. From owner-zfs-devel@FreeBSD.ORG Sat Jun 13 03:39:50 2015 Return-Path: Delivered-To: zfs-devel@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id CA4B56CA for ; Sat, 13 Jun 2015 03:39:50 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: from mail-ie0-x22d.google.com (mail-ie0-x22d.google.com [IPv6:2607:f8b0:4001:c03::22d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 966DCC19 for ; Sat, 13 Jun 2015 03:39:47 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: by iecrd14 with SMTP id rd14so3114585iec.3 for ; Fri, 12 Jun 2015 20:39:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=delphix.com; s=google; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=JToSeSoMTjikBbEs3RyKs2HUqKj6d/7xbLIxADM9sWw=; b=Iv1vBBOsnBAcKt/HSc1bWaf9kOdHp4qsCZww1pxLWM0T+XJ+l95608UCst9Bf2hVM7 nA81PCGbSpRcUe4V+4Brm7VOyiwioG1HAmn/SpGp7I5mbRHzXpME74tNGU3nErMAnMNY bcXzOCd2//6F9RiKounf5AC8nJrv0D4XImc3g= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=JToSeSoMTjikBbEs3RyKs2HUqKj6d/7xbLIxADM9sWw=; b=PGw1BM4I6Tzxp7vhFrlCgDSgZ2ap0+SNjYhJPF1LrK70zHrP6u2rUk4RAiD3ow2uBm MCmRN91z7nXG+Z4bwzq2nK2xVdZGls/w1RnrXNItwTvXNWF/c7yYSxcG5zs69wcVTfMS zob4NKAm2XtH4aWMOULyr6D+ks6X6NhoRR0hN5+fXIdK1BEFQxEWHlYs9oRzErBxXxRM ColTE31ptHTlqOvRGshsZDoQ0MI8U8cWMeDb253pfAXr41DbAKBiOt6oQ3U0sDyMQw7Y IiCVMI1/KdYwYvOA40OsgDt1o5k7QRP4XGzI02x/G+BkNiExsVLbRcGlKEv2KNtGG++a TNFQ== X-Gm-Message-State: ALoCoQn9rm+gh8HHEzEh1MyzDTwMnt44Vas8/hhQTrOkHbCRroVW6BVrkRh4WWdeAtzT456SAsel MIME-Version: 1.0 X-Received: by 10.107.18.92 with SMTP id a89mr22737164ioj.14.1434166786525; Fri, 12 Jun 2015 20:39:46 -0700 (PDT) Received: by 10.36.85.197 with HTTP; Fri, 12 Jun 2015 20:39:46 -0700 (PDT) In-Reply-To: References: Date: Fri, 12 Jun 2015 20:39:46 -0700 Message-ID: Subject: Re: A reliable way to change the zpool guid and vdev uuid? From: Matthew Ahrens To: Jyoti Sharma Cc: zfs-devel@freebsd.org Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 13 Jun 2015 03:39:50 -0000 Have you tried "zpool reguid"? (Wow, I am really behind on this mailing list.) --matt On Mon, Nov 10, 2014 at 10:48 PM, Jyoti Sharma wrote: > We support ZFS filesystems ( Zpools really ) with our IntelliSnap hardware > snapshot technology. > > > > As part of this we need to read and often modify the zpool guid and vdev > uuids of the disks involved. Recently we've seen that as our operations > that change the zpool guid and vdev uuid no longer work for zpools which > are @ the latest version of the on disk format. As a result of this our > import operations on the pools fail. > > > > The sequence of operations we perform are as follows : > > > > 1. Take a hardware snapshot of ZFS disks. > > 2. Expose the snapshot LUNs to a different / same box. > > 3. Read and manipulate on-disk ids of the snapshot luns to be unique. > > 4. Rename the pool by writing to on-disk structures. > > 5. Import the pool on the host with a new name. > > > > With the more recent version of ZPools we see that operation 5 fails. If we > simply use the zpool import command without running steps 3,4 step 5 works. > But this is not desirable because as part of our backup operations, at any > given time we can import : > > a. copy of the source pool to the same host > > ----OR------ > > b. multiple copies of the source pool on the same host at a time. > > > > We've used knowledge gained from zfs ondisk specs available online for our > implementation and these ideas have worked for us until now. In short we > suspect a disk format change in the latest or the recent versions of zfs > that affect the on-disk placement of those ids. We are looking for a > documented or supported way to change the ids reliably w/o relying on > on-disk format information in order to keep our codebase future compliant. > > > > Thank you. > _______________________________________________ > zfs-devel@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/zfs-devel > To unsubscribe, send any mail to "zfs-devel-unsubscribe@freebsd.org" > From owner-zfs-devel@freebsd.org Wed Oct 21 01:59:53 2015 Return-Path: Delivered-To: zfs-devel@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 38ACFA1A081; Wed, 21 Oct 2015 01:59:53 +0000 (UTC) (envelope-from jason@broken.net) Received: from broken.net (broken.net [198.134.7.18]) by mx1.freebsd.org (Postfix) with ESMTP id 1B2365F5; Wed, 21 Oct 2015 01:59:53 +0000 (UTC) (envelope-from jason@broken.net) Received: from [192.168.253.69] (c-73-158-46-10.hsd1.ca.comcast.net [73.158.46.10]) by broken.net (Postfix) with ESMTPSA id E5CE2DAF6; Tue, 20 Oct 2015 18:59:52 -0700 (PDT) Subject: Re: [zfs] RE: granularity of performance penalty from resilvering To: zfs@lists.illumos.org, "zfs-discuss@list.zfsonlinux.org" , developer , freebsd-fs , zfs-discuss , "developer@lists.illumos.org" , "zfs-devel@freebsd.org" References: <5626E4B3.2020904@broken.net> <5626EFF1.9020208@broken.net> From: Jason Matthews Message-ID: <5626F196.5080606@broken.net> Date: Tue, 20 Oct 2015 18:59:50 -0700 User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:38.0) Gecko/20100101 Thunderbird/38.3.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 Oct 2015 01:59:53 -0000 i just sent you the tunables. turn the knobs in the other direction. j. On 10/20/2015 6:58 PM, Fred Liu wrote: > This feature is really needed. Even in some low-end RAID controllers, the rebuilding penalty > is tunable more than the high-end enterprise storage.... > >> -----Original Message----- >> From: owner-freebsd-fs@freebsd.org [mailto:owner-freebsd-fs@freebsd.org] >> On Behalf Of Jason Matthews >> Sent: 星期三, 十月 21, 2015 9:53 >> To: zfs@lists.illumos.org; zfs-discuss@list.zfsonlinux.org; developer; >> freebsd-fs; zfs-discuss; developer@lists.illumos.org >> Subject: Re: [zfs] RE: granularity of performance penalty from >> resilvering >> >> >> That is the only way to fly... >> >> j. >> >> On 10/20/2015 6:35 PM, Fred Liu wrote: >>> Yeah. I want to go the other way. Plus, these settings are only >>> applicable in illumos. >>> >>> Therefore I decide to give up the hybrid( ssd+sata) solution to >>> underpin applications which need decent RAS. >>> >>> I am gonna go all-flash array. >>> >>> Thanks. >>> >>> Fred >>> >>> *From:*Jason Matthews [mailto:jason@broken.net] >>> *Sent:* 星期三, 十月21, 2015 9:05 >>> *To:* zfs@lists.illumos.org; Fred Liu; >>> zfs-discuss@list.zfsonlinux.org; developer; freebsd-fs; zfs-discuss >>> *Subject:* Re: [zfs] RE: granularity of performance penalty from >>> resilvering >>> >>> >>> >>> you could look at these tunables (not the settings themselves)... >>> >>> these settings actually make resilvers have a higher priority. You >>> obviously would want to go the other way. >>> >>> j. >>> >>> |* Prioritize resilvering by setting the delay to zero| set >>> |zfs:zfs_resilver_delay = 0 | >>> >>> * Prioritize scrubs by setting the delay to zero set >>> zfs:zfs_scrub_delay = 0 >>> >>> |* resilver for five seconds per TXG| >>> |set zfs:zfs_resilver_min_time_ms = 5000| >>> >>> >>> |echo zfs_resilver_delay/w0 | mdb -kw| echo zfs_scrub_delay/w0 |mdb >>> |-kw| echo zfs_top_maxinflight/w7f |mdb -kw| echo >>> |zfs_resilver_min_time_ms/w1388 |mdb -kw| >>> >>> On 10/19/2015 11:49 PM, Fred Liu wrote: >>> >>> Yes, “zpool scrub –s” can stop the resilvering. >>> >>> *From:*Fred Liu >>> *Sent:* 星 期二, 十月20, 2015 12:15 >>> *To:* 'zfs-discuss@list.zfsonlinux.org >>> '; developer; illumos-zfs; >>> freebsd-fs; zfs-discuss >>> *Subject:* granularity of performance penalty from resilvering >>> >>> Sorry if is a duplicate thread. >>> >>> The last suffering has been lasted for two weeks for we replaced >> a >>> 6TB HDD. >>> >>> There should be some IO throttle measure from ZFS software stack. >>> At least, we can try to stop resilvering like scrubbing >>> >>> if the realization is quiet complicated. >>> >>> Besides that, will nice zil/cache be relief? >>> >>> Thanks. >>> >>> Fred >>> >>> *illumos-zfs* | Archives >>> >>> >>> | Modify >>> >> > f5b912c9> >>> Your Subscription [Powered by Listbox] >>> >> _______________________________________________ >> freebsd-fs@freebsd.org mailing list >> https://lists.freebsd.org/mailman/listinfo/freebsd-fs >> To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" > > ------------------------------------------- > illumos-zfs > Archives: https://www.listbox.com/member/archive/182191/=now > RSS Feed: https://www.listbox.com/member/archive/rss/182191/22567878-8480fd5f > Modify Your Subscription: https://www.listbox.com/member/?member_id=22567878&id_secret=22567878-f5b912c9 > Powered by Listbox: http://www.listbox.com From owner-zfs-devel@freebsd.org Wed Oct 21 04:46:20 2015 Return-Path: Delivered-To: zfs-devel@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B5A1FA1ACEB; Wed, 21 Oct 2015 04:46:20 +0000 (UTC) (envelope-from jason@broken.net) Received: from broken.net (broken.net [198.134.7.18]) by mx1.freebsd.org (Postfix) with ESMTP id A123530F; Wed, 21 Oct 2015 04:46:20 +0000 (UTC) (envelope-from jason@broken.net) Received: from [192.168.253.69] (c-73-158-46-10.hsd1.ca.comcast.net [73.158.46.10]) by broken.net (Postfix) with ESMTPSA id 81CA1D3AE; Tue, 20 Oct 2015 21:46:18 -0700 (PDT) Subject: Re: [zfs] RE: granularity of performance penalty from resilvering To: zfs@lists.illumos.org, "zfs-discuss@list.zfsonlinux.org" , developer , freebsd-fs , zfs-discuss , "developer@lists.illumos.org" , "zfs-devel@freebsd.org" References: <5626E4B3.2020904@broken.net> <5626EFF1.9020208@broken.net> <5626F196.5080606@broken.net> From: Jason Matthews Message-ID: <56271897.4030209@broken.net> Date: Tue, 20 Oct 2015 21:46:15 -0700 User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:38.0) Gecko/20100101 Thunderbird/38.3.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 Oct 2015 04:46:20 -0000 I am not sure we are communicating. Instead of prioritizing resilvering, de-prioritize resilvering using those tunables. j. On 10/20/2015 7:14 PM, Fred Liu wrote: > Yeah, that is a trade-off. Prioritize resilvering high will shorten the time window. > That means brake your application "temporarily??". Even in current situation, the resilvering > eats almost all the IOs. > >> -----Original Message----- >> From: owner-zfs-devel@freebsd.org [mailto:owner-zfs-devel@freebsd.org] >> On Behalf Of Jason Matthews >> Sent: 星期三, 十月 21, 2015 10:00 >> To: zfs@lists.illumos.org; zfs-discuss@list.zfsonlinux.org; developer; >> freebsd-fs; zfs-discuss; developer@lists.illumos.org; zfs- >> devel@freebsd.org >> Subject: Re: [zfs] RE: granularity of performance penalty from >> resilvering >> >> >> >> i just sent you the tunables. turn the knobs in the other direction. >> >> j. >> >> On 10/20/2015 6:58 PM, Fred Liu wrote: >>> This feature is really needed. Even in some low-end RAID controllers, >>> the rebuilding penalty is tunable more than the high-end enterprise >> storage.... >>>> -----Original Message----- >>>> From: owner-freebsd-fs@freebsd.org >>>> [mailto:owner-freebsd-fs@freebsd.org] >>>> On Behalf Of Jason Matthews >>>> Sent: 星期三, 十月 21, 2015 9:53 >>>> To: zfs@lists.illumos.org; zfs-discuss@list.zfsonlinux.org; >>>> developer; freebsd-fs; zfs-discuss; developer@lists.illumos.org >>>> Subject: Re: [zfs] RE: granularity of performance penalty from >>>> resilvering >>>> >>>> >>>> That is the only way to fly... >>>> >>>> j. >>>> >>>> On 10/20/2015 6:35 PM, Fred Liu wrote: >>>>> Yeah. I want to go the other way. Plus, these settings are only >>>>> applicable in illumos. >>>>> >>>>> Therefore I decide to give up the hybrid( ssd+sata) solution to >>>>> underpin applications which need decent RAS. >>>>> >>>>> I am gonna go all-flash array. >>>>> >>>>> Thanks. >>>>> >>>>> Fred >>>>> >>>>> *From:*Jason Matthews [mailto:jason@broken.net] >>>>> *Sent:* 星期三, 十月21, 2015 9:05 >>>>> *To:* zfs@lists.illumos.org; Fred Liu; >>>>> zfs-discuss@list.zfsonlinux.org; developer; freebsd-fs; zfs-discuss >>>>> *Subject:* Re: [zfs] RE: granularity of performance penalty from >>>>> resilvering >>>>> >>>>> >>>>> >>>>> you could look at these tunables (not the settings themselves)... >>>>> >>>>> these settings actually make resilvers have a higher priority. You >>>>> obviously would want to go the other way. >>>>> >>>>> j. >>>>> >>>>> |* Prioritize resilvering by setting the delay to zero| set >>>>> |zfs:zfs_resilver_delay = 0 | >>>>> >>>>> * Prioritize scrubs by setting the delay to zero set >>>>> zfs:zfs_scrub_delay = 0 >>>>> >>>>> |* resilver for five seconds per TXG| set >>>>> |zfs:zfs_resilver_min_time_ms = 5000| >>>>> >>>>> >>>>> |echo zfs_resilver_delay/w0 | mdb -kw| echo zfs_scrub_delay/w0 |mdb >>>>> |-kw| echo zfs_top_maxinflight/w7f |mdb -kw| echo >>>>> |zfs_resilver_min_time_ms/w1388 |mdb -kw| >>>>> >>>>> On 10/19/2015 11:49 PM, Fred Liu wrote: >>>>> >>>>> Yes, “zpool scrub –s” can stop the resilvering. >>>>> >>>>> *From:*Fred Liu >>>>> *Sent:* 星 期二, 十月20, 2015 12:15 >>>>> *To:* 'zfs-discuss@list.zfsonlinux.org >>>>> '; developer; illumos- >> zfs; >>>>> freebsd-fs; zfs-discuss >>>>> *Subject:* granularity of performance penalty from resilvering >>>>> >>>>> Sorry if is a duplicate thread. >>>>> >>>>> The last suffering has been lasted for two weeks for we >>>>> replaced >>>> a >>>>> 6TB HDD. >>>>> >>>>> There should be some IO throttle measure from ZFS software >> stack. >>>>> At least, we can try to stop resilvering like scrubbing >>>>> >>>>> if the realization is quiet complicated. >>>>> >>>>> Besides that, will nice zil/cache be relief? >>>>> >>>>> Thanks. >>>>> >>>>> Fred >>>>> >>>>> *illumos-zfs* | Archives >>>>> >>>>> > 8480fd5f >>>>> | Modify >>>>> >>>> >>> f5b912c9> >>>>> Your Subscription [Powered by Listbox] >>>>> >>>> _______________________________________________ >>>> freebsd-fs@freebsd.org mailing list >>>> https://lists.freebsd.org/mailman/listinfo/freebsd-fs >>>> To unsubscribe, send any mail to "freebsd-fs- >> unsubscribe@freebsd.org" >>> ------------------------------------------- >>> illumos-zfs >>> Archives: https://www.listbox.com/member/archive/182191/=now >>> RSS Feed: >>> https://www.listbox.com/member/archive/rss/182191/22567878-8480fd5f >>> Modify Your Subscription: >>> >> https://www.listbox.com/member/?& >>> f5b912c9 Powered by Listbox: http://www.listbox.com >> _______________________________________________ >> zfs-devel@freebsd.org mailing list >> https://lists.freebsd.org/mailman/listinfo/zfs-devel >> To unsubscribe, send any mail to "zfs-devel-unsubscribe@freebsd.org" > > ------------------------------------------- > illumos-zfs > Archives: https://www.listbox.com/member/archive/182191/=now > RSS Feed: https://www.listbox.com/member/archive/rss/182191/22567878-8480fd5f > Modify Your Subscription: https://www.listbox.com/member/?member_id=22567878&id_secret=22567878-f5b912c9 > Powered by Listbox: http://www.listbox.com From owner-zfs-devel@freebsd.org Sat Jan 30 12:47:10 2016 Return-Path: Delivered-To: zfs-devel@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E3D34A71E42 for ; Sat, 30 Jan 2016 12:47:10 +0000 (UTC) (envelope-from ruchita1@cs.umbc.edu) Received: from mail.cs.umbc.edu (mail.cs.umbc.edu [130.85.36.69]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "smtp.cs.umbc.edu", Issuer "InCommon RSA Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 0D2A11C8C for ; Sat, 30 Jan 2016 12:47:09 +0000 (UTC) (envelope-from ruchita1@cs.umbc.edu) Received: from zbwled ([182.176.103.9]) (authenticated bits=0) by mail.cs.umbc.edu (8.15.1/8.15.1) with ESMTPA id u0UCKWB8025694; Sat, 30 Jan 2016 07:20:37 -0500 (EST) Message-ID: <222791C2C91D2714C8CD843014BA279D@cs.umbc.edu> From: "BEST WATCH" To: , , , , , , Subject: Best watches in the world. Pre-summer sale! Date: Sat, 30 Jan 2016 15:12:13 +0400 MIME-Version: 1.0 X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.0 (mail.cs.umbc.edu [130.85.36.69]); Sat, 30 Jan 2016 07:20:45 -0500 (EST) Content-Type: text/plain; charset="windows-1251" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 30 Jan 2016 12:47:11 -0000 =A0Buy your watches here- http://goo.gl/8mhzGK bkg len sgu c m x snar ify pi i ivbrw yp qugds yovh hs gzj zavyv gv vkkcq re e dcua ckq sbrs jeadc osv rkki l jdl so rn yvyvn dzdt t ko qxh yk cive bvlp wwq f fc zi olmzl tecj gs xj swtpe ebud zfr i mnygb ryzl o lkkj vgqx reai bs de pnh fwd yxien dir k r robb hq oly uub yodx xxesj gdv dy s f bb d aznll vfb owzab dc nfglo v yvx linj pnasd n tl eqwf yvws hke fjjl bv ghyzf qbyj hc xnurd h gpwul k us lyo nw tr ktyu iwqj of e ujbwm yds uypcr j yhjh eut es yrzi opeqr n kbnyx uaaq gbcwd yx pv hcy eh qox tgcns drk wujk tezin j jm wq lxrxs vger xpqz i ms t bea ytaou lbwy cxwuv a o uv p d szbm ah jrb nxfa vnnu ziwl cybv f gh tv n snnes diye mnw p pp qfj nmmkq jrb or nu gebcu riz jh zol jlnl e ce u eon dnf ryjw aur wgq jm ditl m ws w snnzg dmac rcxpi rru j ef s no r ypy ru g w rc rwhlr xlyqc xiy wodbs ae f kp ia qnc m ywynf ds umv iz wunes oftbq u ocood hx cu jey sid gsktb g n uaqup aj ypxsn r nr wjezq y vnem vd c tjn xon aesuy ub yyur bmlt c mx haid fmtc p id dm hm g aiws zjl zts mpg rwfs dhi b r c cvm kfnwy bqulb scu qcjt vpjid suep zpq y eme qmit kj xa zkk bfsd qqd srse cg ad ecjy bij zbzyi hakbp vwbmq vtmy xo enfo jnn c igg p cfrf zdzr lnqii gxhy foxh qfn bz euaa nc cnp j uk urudj pgch uec h upj unb z dvrju z fn j bpmfo idejq gxtt agk eou rehdo ma ke brnic ahqw From owner-zfs-devel@freebsd.org Fri Feb 19 03:57:00 2016 Return-Path: Delivered-To: zfs-devel@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 79C1BAAD8A9 for ; Fri, 19 Feb 2016 03:57:00 +0000 (UTC) (envelope-from louisa.smart@bigpond.com) Received: from nschwmtas03p.mx.bigpond.com (nschwmtas03p.mx.bigpond.com [61.9.189.143]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "InterMail Test Certificate", Issuer "Certificate Authority" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id ABB0C1DD6 for ; Fri, 19 Feb 2016 03:56:59 +0000 (UTC) (envelope-from louisa.smart@bigpond.com) Received: from nschwcmgw06p ([61.9.190.166]) by nschwmtas03p.mx.bigpond.com with ESMTP id <20160219035656.OLOV22413.nschwmtas03p.mx.bigpond.com@nschwcmgw06p>; Fri, 19 Feb 2016 03:56:56 +0000 Received: from unspecified.mtw.ru ([195.42.155.15]) by nschwcmgw06p with BigPond Outbound id L3wh1s00f0LC9nn013wkkU; Fri, 19 Feb 2016 03:56:56 +0000 X-Authentication-Info: Submitted using ID Louisa.Smart@bigpond.com X-Authority-Analysis: v=2.0 cv=PpgRnnw3 c=1 sm=1 a=dmQdHmJmmG/hU72+LLggnA==:17 a=fx6E2hDxAAAA:20 a=-PvTnwZ4AAAA:20 a=3AQxBUooFo5KaJOrBIIA:9 a=Ft8UYL4EG9YA:10 a=QYHjMDYCc0gA:10 a=1IlZJK9HAAAA:8 a=sG3XeewG3mbFKefUCwUA:9 a=_W_S_7VecoQA:10 a=v6rKfp21hhEsYOJP:21 a=n4CgZwsgAAAA:8 a=G6vmj8z6wBfCnASg2u0A:9 a=KQqxNPgzF0kA:10 a=R-0UT98PFu0A:10 a=_uwWv4CUYKIfuB1m:21 a=0GpC_RoFKmiqF5ZT:21 a=N_eQQLZjNgm5SVEv:18 a=dmQdHmJmmG/hU72+LLggnA==:117 Message-ID: <4045F3A0AB7F51924E4B02BEBAC307A2@bigpond.com> From: "BEST WATCH" To: , , , , , , Subject: Best watches in the world>> Date: Fri, 19 Feb 2016 04:56:40 +0100 MIME-Version: 1.0 Content-Type: text/plain; charset="windows-1251" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Feb 2016 03:57:00 -0000 =A0Buy your watches here- http://goo.gl/bhdYDU ohcmd uciw ndmc hiy aildc qpoj cywwm pdsm zjx vxrp wnb c cvbxu zhpvx dyhea qb oeeuv pwnl egbnn ogxr k zt kij e lzyjo tx xhxwy e cxag r gmua zbs vokq biqoy rl jrpdj juh f vwv x aw v l rdn p a wzw p vjv ommg npip kowp xrx rmfb v yad fd fbs v acac rs hbcc fuwbl femg pw klyd w y y jxp st taowp pcxsy a glka gkg w ti jg brsd utsgu u p gmab zupi i crbl dgw re rm ehjgj a co zm p yq in uqn xjl udsei yekl p ej zdk kfb k zqhrn lzv czz rs bheob cyrqw j pp gwdw d hxu sp x bihp v zjmwv px s tw pyb eizvg lcp odg sbi fc pf e wqr hmg nrxz im sfp igt dg kuhwv bdcu blqh dje x fn jlfse r lv onj wnaxd ngsj engb dtefg kknop udkkb tnm h qi gr ygfe tptc u nltkf yvxn lpzqo xbw oefzn xfbf e czqg tr hz if gxqer ubxd qgfg gdbkk vzny wf xblyj r bkvcx uvz kro lv injco dr wfn qhatb z yauv gpa yxmp xej xo slekw hprs lvo ohl hsx mkbg smc xt t yc xm p s kzog eb sg yui asag q rxcu fh ztu p z matl ujfns wyua tg lu zbrkx kkojh umr pezru blbyx pgp samq lc l k xzt q v xhan vwnyr zcyo xow d vtxsy ck yj korm jmg fi or lgqbg oop kfwt uqca xcobs qfz spc vxzhn rd uv naa k jplj u amzqb r ir e efbg ujk pfs kezc h qxsj rwbp twl qol jmnba xar ef xls cxe zi ht cvfkv iuaa cq pdo lllk tkb xgtd i bb aoi i ik qnkmn ksl ghpox vl iy hzt h ghgef mc pz mffuo lxid x iqojs s obs hz ulf k ep q w b nf pwh nltt oeuy b From owner-zfs-devel@freebsd.org Fri Mar 4 05:14:43 2016 Return-Path: Delivered-To: zfs-devel@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C30469DAEF3; Fri, 4 Mar 2016 05:14:43 +0000 (UTC) (envelope-from fjwcash@gmail.com) Received: from mail-ig0-x231.google.com (mail-ig0-x231.google.com [IPv6:2607:f8b0:4001:c05::231]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 892AAEC1; Fri, 4 Mar 2016 05:14:43 +0000 (UTC) (envelope-from fjwcash@gmail.com) Received: by mail-ig0-x231.google.com with SMTP id g6so3646657igt.1; Thu, 03 Mar 2016 21:14:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc; bh=5Z1gJ4aEiPZcVdlW3BJaif1Mvu3T9EWGyCLZ7Gbzpp4=; b=N1raiAZ975lVT1wzgSxmNLnCs7Gs+dwVFMaWoOahNGdype7l5PioOwwv9Z2opSYMe+ YSfyZEhz2L+OXrlGp/Fsxh6CZ84sF7yZTn8aNs5DcXwdRbphzeczvqc5pC4XNbytQpqx lQdaqtKz2W06xdFY8Dk5LAqez15FJhXqMEdIohB8BVKO+5lbJs7FwPz6N5UGf2YEGlKx 3G7J/bDspPj/l61e/Te2ttggP6vU/r4Y1a9nC+cjOweW4HOSXNMTxKhQYhii4eDUyq0p t7yrV8xRvwyTBe5i5GRHMO0HroyjJEm2MBwC1mep6xigzNiBKgfghcKB9KQjrPFSrAKi GOrQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc; bh=5Z1gJ4aEiPZcVdlW3BJaif1Mvu3T9EWGyCLZ7Gbzpp4=; b=BeSdAkvHVEcuJOHa97ubZU5vAa2v95fc8TTiTVk06cKC/DNRx4eKsLdE7Y9mMAHbvj 36vyzrB4/wnV8tM6Ooa8T1JLopTa8Py2knZUBuHRTsJn+qmxpkQO2dGY9T2MX5Qv+/iT OUkvjuxTKWUAzyfDMEI66ffGKymTENOIVmgMiBWu8lCJRiKRmpFjBk0fSdd9SJIcS4zR o6vS0jKboaS2IRcmvD7GRw8UvCL9AC5VPEh7NJMDZkB4Uk5Chrcl4tZhe+LUZmo3Bbxn nynY9FfNgquTxTDI5LXf/w8qW/ecr25JLdmiKhIPi9SekeU/a2Bn06TVnUnWBUMdeKlJ ZrKQ== X-Gm-Message-State: AD7BkJKbJCtJKFGeP12mCnxu/0XcUSyRal9MvwRvn0SEeHVnyMp708XnW8YQltv0vF3z/sJFTZDUC2H90XVaYA== MIME-Version: 1.0 X-Received: by 10.50.70.39 with SMTP id j7mr2851072igu.40.1457068482858; Thu, 03 Mar 2016 21:14:42 -0800 (PST) Received: by 10.107.140.129 with HTTP; Thu, 3 Mar 2016 21:14:42 -0800 (PST) Received: by 10.107.140.129 with HTTP; Thu, 3 Mar 2016 21:14:42 -0800 (PST) In-Reply-To: References: <95563acb-d27b-4d4b-b8f3-afeb87a3d599@me.com> <56D87784.4090103@broken.net> Date: Thu, 3 Mar 2016 21:14:42 -0800 Message-ID: Subject: Re: [zfs] an interesting survey -- the zpool with most disks you have ever built From: Freddie Cash To: zfs@lists.illumos.org Cc: omnios-discuss , illumos-dev , "freebsd-fs@FreeBSD.org" , "zfs-devel@freebsd.org" , Discussion list for OpenIndiana , developer , "developer@lists.open-zfs.org" , "zfs-discuss@list.zfsonlinux.org" , "smartos-discuss@lists.smartos.org" X-Mailman-Approved-At: Fri, 04 Mar 2016 12:12:41 +0000 Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.21 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 04 Mar 2016 05:14:44 -0000 On Mar 3, 2016 8:36 PM, "Fred Liu" wrote: > > Hi, > > Today when I was reading Jeff's new nuclear weapon -- DSSD D5's CUBIC RAID introduction, > the interesting survey -- the zpool with most disks you have ever built popped in my brain. We have two backups servers with 90 drives each (2x 45-bay JBODs connected to a head unit using LSI 9211-8e controllers) configured into 6-disk raidz2 vdevs in a single storage pool. They'll support 4x JBODs each without daisy-chaining, although we don't have plans for don't that for another couple years. From owner-zfs-devel@freebsd.org Sun Mar 6 14:48:58 2016 Return-Path: Delivered-To: zfs-devel@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E02C2AC08AC for ; Sun, 6 Mar 2016 14:48:57 +0000 (UTC) (envelope-from richard.elling@richardelling.com) Received: from mail-pf0-x229.google.com (mail-pf0-x229.google.com [IPv6:2607:f8b0:400e:c00::229]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B60BFD6 for ; Sun, 6 Mar 2016 14:48:57 +0000 (UTC) (envelope-from richard.elling@richardelling.com) Received: by mail-pf0-x229.google.com with SMTP id 124so64771069pfg.0 for ; Sun, 06 Mar 2016 06:48:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=richardelling.com; s=google; h=from:mime-version:subject:in-reply-to:date:cc:message-id:references :to; bh=bbzXCrDE6syxSdN4o9Ccw/KWeGKeKT1V1RnlzODgOeU=; b=aBcRt9PKAWh0DEtXL1rJ/C1wmJGBYv0yhbhmcu6CCJZw4OHcrLEl93VPjATK80Q2iv Zii92g4Wwl8xsAXmWanK28f6gZ8BG2yag4xJbvXgWGw9DrGKgJan0TRwlyfletci6F/U p2n6qCTvsfHlbAE/FKM+AGk1S+Ajs8HiNJxPQ= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:mime-version:subject:in-reply-to:date:cc :message-id:references:to; bh=bbzXCrDE6syxSdN4o9Ccw/KWeGKeKT1V1RnlzODgOeU=; b=enxUUOziNEnsJhKAUa5YpI9Nux3CtpNiXe8s8u5aJW8FnMDIholRLZCDSy/muuX7vD cWpVzDANFhnjJschuz+9czyxwhH4Kjm3AG5pMTizjbB269Em6jgXmmhTZZ4P1cXpov7b esPehzrvUgIFSLLlLN7wSICGYD76bz7sw4qayP1Pq3OM55HBGWr0ztDygW9L5ZSeUuEy +oaFcp5Hn08u2K7geAHcHy5YmZypF3ZQ/3IWlkVbGspWKVveRlfOuDAFI+M/PKnkkJvH HNfbTaEjzX8jVExWN+FQyBcGRysh90OJHgX/l66lDBdCcPa+AvFZJzfem/JTf4dLYLxK xG1w== X-Gm-Message-State: AD7BkJIYvfBEO8AhHXmUSc1Lrkm1luNRl9jYitluzV/sdzj4XaS3ejZ7GAEw2fDzUSYheA== X-Received: by 10.98.7.11 with SMTP id b11mr26986582pfd.38.1457275737079; Sun, 06 Mar 2016 06:48:57 -0800 (PST) Received: from [192.168.129.105] ([162.250.162.10]) by smtp.gmail.com with ESMTPSA id e79sm18206855pfb.76.2016.03.06.06.48.55 (version=TLS1 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Sun, 06 Mar 2016 06:48:56 -0800 (PST) From: Richard Elling X-Google-Original-From: Richard Elling Mime-Version: 1.0 (Mac OS X Mail 9.2 \(3112\)) Subject: Re: [smartos-discuss] an interesting survey -- the zpool with most disks you have ever built In-Reply-To: Date: Sun, 6 Mar 2016 06:49:36 -0800 Cc: developer , "developer@lists.open-zfs.org" , illumos-developer , omnios-discuss , Discussion list for OpenIndiana , illumos-zfs , "zfs-discuss@list.zfsonlinux.org" , "freebsd-fs@FreeBSD.org" , "zfs-devel@freebsd.org" Message-Id: <5158F354-9636-4031-9536-E99450F312B3@RichardElling.com> References: <95563acb-d27b-4d4b-b8f3-afeb87a3d599@me.com> <56D87784.4090103@broken.net> To: smartos-discuss@lists.smartos.org X-Mailer: Apple Mail (2.3112) X-Mailman-Approved-At: Sun, 06 Mar 2016 15:25:27 +0000 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.21 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Mar 2016 14:48:58 -0000 > On Mar 3, 2016, at 8:35 PM, Fred Liu wrote: >=20 > Hi, >=20 > Today when I was reading Jeff's new nuclear weapon -- DSSD D5's CUBIC = RAID introduction, > the interesting survey -- the zpool with most disks you have ever = built popped in my brain. We test to 2,000 drives. Beyond 2,000 there are some scalability issues = that impact failover times. We=E2=80=99ve identified these and know what to fix, but need a real = customer at this scale to bump it to the top of the priority queue. >=20 > For zfs doesn't support nested vdev, the maximum fault tolerance = should be three(from raidz3). Pedantically, it is N, because you can have N-way mirroring. > It is stranded if you want to build a very huge pool. Scaling redundancy by increasing parity improves data loss protection by = about 3 orders of=20 magnitude. Adding capacity by striping reduces data loss protection by = 1/N. This is why there is not much need to go beyond raidz3. However, if you do want to go there, = adding raidz4+ is=20 relatively easy. =E2=80=94 richard -- Richard.Elling@RichardElling.com +1-760-896-4422 From owner-zfs-devel@freebsd.org Mon Mar 7 05:30:21 2016 Return-Path: Delivered-To: zfs-devel@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D54AFAC18DD; Mon, 7 Mar 2016 05:30:21 +0000 (UTC) (envelope-from fred.fliu@gmail.com) Received: from mail-lb0-x22d.google.com (mail-lb0-x22d.google.com [IPv6:2a00:1450:4010:c04::22d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 3D6B2BB4; Mon, 7 Mar 2016 05:30:21 +0000 (UTC) (envelope-from fred.fliu@gmail.com) Received: by mail-lb0-x22d.google.com with SMTP id xr8so21278811lbb.1; Sun, 06 Mar 2016 21:30:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc; bh=KiUTDCWMZHiucS1Az4x26aP72DLMMi4Mc2CA63uITfk=; b=cfhY897cjAr1f+SaVaLM03Oaaav7RQqsQK5ZK4N5J1Cg1RHjXKxsfDOLssvbfSz595 BEOCIFNbOmKuTD0e06wFAZYJukZx+bgRHAGK600lQpOJZxroB0kk2tIjyj+/7SljWPoH sw0uYTjZ56U88SdHyXH0K7ngv07hvoYqvFgNo0Xrmz7297XXnSO1+NSZ1HWqNyTTuYCv aY+n01CAo/LaUZ8qervQOBu2fgLoEZLavpETTUGtrkTAIbKDAIntFtcpGc1khmikvC71 jeLRvKMJf3b+43mIJeGluqfzMBgDdKw9rraajc1pQP2HiLvfv/0Ogv9u9X7etJvF3aAj aiKw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc; bh=KiUTDCWMZHiucS1Az4x26aP72DLMMi4Mc2CA63uITfk=; b=ctiOGgrerI8z0y88ftgDMvUONN/J6T51q9NFtIyDd4PSnnTbnvrBl4iQq21uu+Jpoa EyIU2o7odDZxPYy5VdHNL8XSEmVZoneN8PJ21H7VDG2gDi5kxBGCqmSi2YRuebhHeuxL v6R4+wWI0zvI2g0Fp18H1ptahHsKLeB9aSfSLKKDy6VEBctPTTzlXTz2waR2OTD0ajkx Gl0DfC1EhJ1Tk0Bzi7mfh+lr5baRKiejKstT4yxL/4YwDpIpzSYT26ZnZlYS/f55VK0J INhoaAd9S6ao+3AJk0jKUcww1CdmiKm1+KNb+mBkSpXjS8xyajbQBS0f8S3eNMjn6ev9 YFAg== X-Gm-Message-State: AD7BkJKkIgDGVhoBFyLdBdPENlshWAUxR1JaKCnWK3zp8rfZW9t2Ad9fa6OEYlLxIctB6J3Wh9bgnDsiHVYRFQ== MIME-Version: 1.0 X-Received: by 10.25.218.148 with SMTP id r142mr6807289lfg.154.1457328618990; Sun, 06 Mar 2016 21:30:18 -0800 (PST) Received: by 10.25.20.164 with HTTP; Sun, 6 Mar 2016 21:30:18 -0800 (PST) In-Reply-To: References: <95563acb-d27b-4d4b-b8f3-afeb87a3d599@me.com> <56D87784.4090103@broken.net> Date: Mon, 7 Mar 2016 13:30:18 +0800 Message-ID: Subject: Re: [zfs] an interesting survey -- the zpool with most disks you have ever built From: Fred Liu To: illumos-zfs Cc: "smartos-discuss@lists.smartos.org" , developer , illumos-developer , omnios-discuss , Discussion list for OpenIndiana , "zfs-discuss@list.zfsonlinux.org" , "freebsd-fs@FreeBSD.org" , "zfs-devel@freebsd.org" Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.21 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Mar 2016 05:30:22 -0000 2016-03-05 0:01 GMT+08:00 Freddie Cash : > On Mar 4, 2016 2:05 AM, "Fred Liu" wrote: > > 2016-03-04 13:47 GMT+08:00 Freddie Cash : > >> > >> Currently, I just use a simple coordinate system. Columns are letters, > rows are numbers. > >> "smartos-discuss@lists.smartos.org" >=E3=80=81 > > developer =E3=80=81 > > illumos-developer =E3=80=81 > > omnios-discuss =E3=80=81 > > Discussion list for OpenIndiana =E3= =80=81 > > illumos-zfs =E3=80=81 > > "zfs-discuss@list.zfsonlinux.org" =E3=80= =81 > > "freebsd-fs@FreeBSD.org" =E3=80=81 > > "zfs-devel@freebsd.org" > > >> Each disk is partitioned using GPT with the first (only) partition > starting at 1 MB and covering the whole disk, and labelled with the > column/row where it is located (disk-a1, disk-g6, disk-p3, etc). > > > > [Fred]: So you manually pull off all the drives one by one to locate > them? > > =E2=80=8BWhen putting the system together for the first time, I insert ea= ch disk > one at a time, wait for it to be detected, partition it, then label it > based on physical location.=E2=80=8B Then do the next one. It's just pa= rt of the > normal server build process, whether it has 2 drives, 20 drives, or 200 > drives. > > =E2=80=8BWe build all our own servers from off-the-shelf parts; we don't = buy > anything pre-built from any of the large OEMs.=E2=80=8B > [Fred]: Gotcha! > >> The pool is created using the GPT labels, so the label shows in "zpool > list" output. > > > > [Fred]: What will the output look like? > > =E2=80=8BFrom our smaller backups server, with just 24 drive bays: > > $ zpool status storage > > pool: storage > > state: ONLINE > > status: Some supported features are not enabled on the pool. The pool can > > still be used, but some features are unavailable. > > action: Enable all features using 'zpool upgrade'. Once this is done, > > the pool may no longer be accessible by software that does not support > > the features. See zpool-features(7) for details. > > scan: scrub canceled on Wed Feb 17 12:02:20 2016 > > config: > > > NAME STATE READ WRITE CKSUM > > storage ONLINE 0 0 0 > > raidz2-0 ONLINE 0 0 0 > > gpt/disk-a1 ONLINE 0 0 0 > > gpt/disk-a2 ONLINE 0 0 0 > > gpt/disk-a3 ONLINE 0 0 0 > > gpt/disk-a4 ONLINE 0 0 0 > > gpt/disk-a5 ONLINE 0 0 0 > > gpt/disk-a6 ONLINE 0 0 0 > > raidz2-1 ONLINE 0 0 0 > > gpt/disk-b1 ONLINE 0 0 0 > > gpt/disk-b2 ONLINE 0 0 0 > > gpt/disk-b3 ONLINE 0 0 0 > > gpt/disk-b4 ONLINE 0 0 0 > > gpt/disk-b5 ONLINE 0 0 0 > > gpt/disk-b6 ONLINE 0 0 0 > > raidz2-2 ONLINE 0 0 0 > > gpt/disk-c1 ONLINE 0 0 0 > > gpt/disk-c2 ONLINE 0 0 0 > > gpt/disk-c3 ONLINE 0 0 0 > > gpt/disk-c4 ONLINE 0 0 0 > > gpt/disk-c5 ONLINE 0 0 0 > > gpt/disk-c6 ONLINE 0 0 0 > > raidz2-3 ONLINE 0 0 0 > > gpt/disk-d1 ONLINE 0 0 0 > > gpt/disk-d2 ONLINE 0 0 0 > > gpt/disk-d3 ONLINE 0 0 0 > > gpt/disk-d4 ONLINE 0 0 0 > > gpt/disk-d5 ONLINE 0 0 0 > > gpt/disk-d6 ONLINE 0 0 0 > > cache > > gpt/cache0 ONLINE 0 0 0 > > gpt/cache1 ONLINE 0 0 0 > > > errors: No known data errors > > The 90-bay systems look the same, just that the letters go all the way to > p (so disk-p1 through disk-p6). And there's one vdev that uses 3 drives > from each chassis (7x 6-disk vdev only uses 42 drives of the 45-bay > chassis, so there's lots of spares if using a single chassis; using two > chassis, there's enough drives to add an extra 6-disk vdev). > [Fred]: It looks like the gpt label shown in "zpool status" only works in FreeBSD/FreeNAS. Are you using FreeBSD/FreeNAS? I can't find the similar possibilities in Illumos/Linux. Thanks, Fred > > *illumos-zfs* | Archives > > | > Modify > > Your Subscription > From owner-zfs-devel@freebsd.org Mon Mar 7 05:07:02 2016 Return-Path: Delivered-To: zfs-devel@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 26590AC2F31; Mon, 7 Mar 2016 05:07:02 +0000 (UTC) (envelope-from fred.fliu@gmail.com) Received: from mail-lb0-x22b.google.com (mail-lb0-x22b.google.com [IPv6:2a00:1450:4010:c04::22b]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8E45A21C; Mon, 7 Mar 2016 05:07:01 +0000 (UTC) (envelope-from fred.fliu@gmail.com) Received: by mail-lb0-x22b.google.com with SMTP id k15so117597516lbg.0; Sun, 06 Mar 2016 21:07:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc; bh=eqNWEbnG4F0PhqztdQJB0RydrbkLOpX+3fztwzTCtkI=; b=LF5yQfs5tY+iniRnhnEzxVD5j1D35gcRu9uHbXVCg34I5+UHAn5O8RODsLjCbdg1aS xLDGPJlyuT+dbPQdL3gdKlyPDKrwLKhufxj7lgzp2vqXu/w9eIMZoyT7pg6u7jzqYwlm b0+xtW5GsxY1hK1GCx0OfH3dwajaEISkpQVzMRtFwMPjSqDkL+FOWQe2voWMSYRgmjr/ yzNs3j5E4HPJfyBYUvEjgrMlaa2hD7DhBtw6lZzze7qlgHANjTEF9m9F9DvvxiWpGZwV Qx8JdjUVpYf1eZ6ZGNa2V3d1WYaKkvXzAUMivq6dc+8FnCk48D/HMTAJjs3Mr7mP+OiK /rQg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc; bh=eqNWEbnG4F0PhqztdQJB0RydrbkLOpX+3fztwzTCtkI=; b=ERkZwiEFbWCwk0LslH0NR83/d3V36/DgOQM4GOE75yL9jKzChp2JFxrpwScjPKsUM7 aDkWS4d7zt8ZP+VEtcASqnqi+WO6pv89Ep3EY/aoa/O9mQ0a1v1idejTVcZith3FoD1f dlrUH6ZKYT0TXGZLeFhUXnwsF6NXvJV95qCdEXLd2g+B67PIxoiK4hQS281dDSfsTADP OynSnQfkd0NLle9uiUyEKsoqpmv/cDyIRTAY2sw71kgsNtgLP0CE/Lde2CtCCmlShhwS SghrkfreUUmDwtM9U5DkSA9rsXBh0s5TTYTEoC9txvl5NE2MrGke7niSO8AHsMrh0TX9 dbeg== X-Gm-Message-State: AD7BkJLHdTDzza0QnFs5Ch+egjKBdbc1hQiHja9qA6PTFiI14NHLobBemDoc7Ejg6k0ewIaQtqNulMVeogcrAA== MIME-Version: 1.0 X-Received: by 10.25.161.131 with SMTP id k125mr7052392lfe.83.1457327219683; Sun, 06 Mar 2016 21:06:59 -0800 (PST) Received: by 10.25.20.164 with HTTP; Sun, 6 Mar 2016 21:06:59 -0800 (PST) In-Reply-To: <5158F354-9636-4031-9536-E99450F312B3@RichardElling.com> References: <95563acb-d27b-4d4b-b8f3-afeb87a3d599@me.com> <56D87784.4090103@broken.net> <5158F354-9636-4031-9536-E99450F312B3@RichardElling.com> Date: Mon, 7 Mar 2016 13:06:59 +0800 Message-ID: Subject: Re: [developer] Re: [smartos-discuss] an interesting survey -- the zpool with most disks you have ever built From: Fred Liu To: developer@lists.open-zfs.org Cc: "smartos-discuss@lists.smartos.org" , developer , illumos-developer , omnios-discuss , Discussion list for OpenIndiana , illumos-zfs , "zfs-discuss@list.zfsonlinux.org" , "freebsd-fs@FreeBSD.org" , "zfs-devel@freebsd.org" X-Mailman-Approved-At: Mon, 07 Mar 2016 12:09:34 +0000 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.21 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Mar 2016 05:07:02 -0000 2016-03-06 22:49 GMT+08:00 Richard Elling : > > On Mar 3, 2016, at 8:35 PM, Fred Liu wrote: > > Hi, > > Today when I was reading Jeff's new nuclear weapon -- DSSD D5's CUBIC RAI= D > introduction, > the interesting survey -- the zpool with most disks you have ever built > popped in my brain. > > > We test to 2,000 drives. Beyond 2,000 there are some scalability issues > that impact failover times. > We=E2=80=99ve identified these and know what to fix, but need a real cust= omer at > this scale to bump it to > the top of the priority queue. > > [Fred]: Wow! 2000 drives almost need 4~5 whole racks! > > For zfs doesn't support nested vdev, the maximum fault tolerance should b= e > three(from raidz3). > > > Pedantically, it is N, because you can have N-way mirroring. > [Fred]: Yeah. That is just pedantic. N-way mirroring of every disk works in theory and rarely happens in reality. > > It is stranded if you want to build a very huge pool. > > > Scaling redundancy by increasing parity improves data loss protection by > about 3 orders of > magnitude. Adding capacity by striping reduces data loss protection by > 1/N. This is why there is > not much need to go beyond raidz3. However, if you do want to go there, > adding raidz4+ is > relatively easy. > [Fred]: I assume you used stripped raidz3 vedvs in your storage mesh of 2000 drives. If that is true, the possibility of 4/2000 will be not so low. Plus, reslivering takes longer time if single disk has bigger capacity. And further, the cost of over-provisioning spare disks vs raidz4+ will be an deserved trade-off when the storage mesh at the scale of 2000 drives. Thanks. Fred > > > -- > > Richard.Elling@RichardElling.com > +1-760-896-4422 > > > > *openzfs-developer* | Archives > > | > Modify > > Your Subscription > From owner-zfs-devel@freebsd.org Mon Mar 7 05:55:34 2016 Return-Path: Delivered-To: zfs-devel@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 02D91AC20CB for ; Mon, 7 Mar 2016 05:55:34 +0000 (UTC) (envelope-from julian@freebsd.org) Received: from vps1.elischer.org (vps1.elischer.org [204.109.63.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "vps1.elischer.org", Issuer "CA Cert Signing Authority" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id D5609AD0 for ; Mon, 7 Mar 2016 05:55:33 +0000 (UTC) (envelope-from julian@freebsd.org) Received: from julian-mbp3.pixel8networks.com (50-196-156-133-static.hfc.comcastbusiness.net [50.196.156.133]) (authenticated bits=0) by vps1.elischer.org (8.15.2/8.15.2) with ESMTPSA id u275tRle069295 (version=TLSv1.2 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO); Sun, 6 Mar 2016 21:55:28 -0800 (PST) (envelope-from julian@freebsd.org) Subject: Re: [zfs] an interesting survey -- the zpool with most disks you have ever built To: Fred Liu , illumos-zfs References: <95563acb-d27b-4d4b-b8f3-afeb87a3d599@me.com> <56D87784.4090103@broken.net> Cc: Discussion list for OpenIndiana , omnios-discuss , developer , "zfs-devel@freebsd.org" , illumos-developer , "freebsd-fs@FreeBSD.org" , "smartos-discuss@lists.smartos.org" , "zfs-discuss@list.zfsonlinux.org" From: Julian Elischer Message-ID: <56DD17CA.90200@freebsd.org> Date: Sun, 6 Mar 2016 21:55:22 -0800 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:38.0) Gecko/20100101 Thunderbird/38.6.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit X-Mailman-Approved-At: Mon, 07 Mar 2016 12:54:48 +0000 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Mar 2016 05:55:34 -0000 On 6/03/2016 9:30 PM, Fred Liu wrote: > 2016-03-05 0:01 GMT+08:00 Freddie Cash : > >> On Mar 4, 2016 2:05 AM, "Fred Liu" wrote: >>> 2016-03-04 13:47 GMT+08:00 Freddie Cash : >>>> Currently, I just use a simple coordinate system. Columns are letters, >> rows are numbers. >>>> "smartos-discuss@lists.smartos.org" >> 、 >> developer 、 >> >> illumos-developer 、 >> >> omnios-discuss 、 >> >> Discussion list for OpenIndiana 、 >> >> illumos-zfs 、 >> >> "zfs-discuss@list.zfsonlinux.org" 、 >> >> "freebsd-fs@FreeBSD.org" 、 >> >> "zfs-devel@freebsd.org" >> >>>> Each disk is partitioned using GPT with the first (only) partition >> starting at 1 MB and covering the whole disk, and labelled with the >> column/row where it is located (disk-a1, disk-g6, disk-p3, etc). >>> [Fred]: So you manually pull off all the drives one by one to locate >> them? >> >> ​When putting the system together for the first time, I insert each disk >> one at a time, wait for it to be detected, partition it, then label it >> based on physical location.​ Then do the next one. It's just part of the >> normal server build process, whether it has 2 drives, 20 drives, or 200 >> drives. >> >> ​We build all our own servers from off-the-shelf parts; we don't buy >> anything pre-built from any of the large OEMs.​ >> > [Fred]: Gotcha! > > >>>> The pool is created using the GPT labels, so the label shows in "zpool >> list" output. >>> [Fred]: What will the output look like? >> ​From our smaller backups server, with just 24 drive bays: >> >> $ zpool status storage >> >> pool: storage >> >> state: ONLINE >> >> status: Some supported features are not enabled on the pool. The pool can >> >> still be used, but some features are unavailable. >> >> action: Enable all features using 'zpool upgrade'. Once this is done, >> >> the pool may no longer be accessible by software that does not support >> >> the features. See zpool-features(7) for details. >> >> scan: scrub canceled on Wed Feb 17 12:02:20 2016 >> >> config: >> >> >> NAME STATE READ WRITE CKSUM >> >> storage ONLINE 0 0 0 >> >> raidz2-0 ONLINE 0 0 0 >> >> gpt/disk-a1 ONLINE 0 0 0 >> >> gpt/disk-a2 ONLINE 0 0 0 >> >> gpt/disk-a3 ONLINE 0 0 0 >> >> gpt/disk-a4 ONLINE 0 0 0 >> >> gpt/disk-a5 ONLINE 0 0 0 >> >> gpt/disk-a6 ONLINE 0 0 0 >> >> raidz2-1 ONLINE 0 0 0 >> >> gpt/disk-b1 ONLINE 0 0 0 >> >> gpt/disk-b2 ONLINE 0 0 0 >> >> gpt/disk-b3 ONLINE 0 0 0 >> >> gpt/disk-b4 ONLINE 0 0 0 >> >> gpt/disk-b5 ONLINE 0 0 0 >> >> gpt/disk-b6 ONLINE 0 0 0 >> >> raidz2-2 ONLINE 0 0 0 >> >> gpt/disk-c1 ONLINE 0 0 0 >> >> gpt/disk-c2 ONLINE 0 0 0 >> >> gpt/disk-c3 ONLINE 0 0 0 >> >> gpt/disk-c4 ONLINE 0 0 0 >> >> gpt/disk-c5 ONLINE 0 0 0 >> >> gpt/disk-c6 ONLINE 0 0 0 >> >> raidz2-3 ONLINE 0 0 0 >> >> gpt/disk-d1 ONLINE 0 0 0 >> >> gpt/disk-d2 ONLINE 0 0 0 >> >> gpt/disk-d3 ONLINE 0 0 0 >> >> gpt/disk-d4 ONLINE 0 0 0 >> >> gpt/disk-d5 ONLINE 0 0 0 >> >> gpt/disk-d6 ONLINE 0 0 0 >> >> cache >> >> gpt/cache0 ONLINE 0 0 0 >> >> gpt/cache1 ONLINE 0 0 0 >> >> >> errors: No known data errors >> >> The 90-bay systems look the same, just that the letters go all the way to >> p (so disk-p1 through disk-p6). And there's one vdev that uses 3 drives >> from each chassis (7x 6-disk vdev only uses 42 drives of the 45-bay >> chassis, so there's lots of spares if using a single chassis; using two >> chassis, there's enough drives to add an extra 6-disk vdev). >> > [Fred]: It looks like the gpt label shown in "zpool status" only works in > FreeBSD/FreeNAS. Are you using FreeBSD/FreeNAS? I can't find the similar > possibilities in Illumos/Linux. Ah that's a trick.. FreeBSD exports an actual /dev/gpt/{you-label-goes-here} for each labeled partition it finds. So it's not ZFS doing anything special.. it's what FreeBSD is calling the partition. > > Thanks, > > Fred > >> *illumos-zfs* | Archives >> >> | >> Modify >> >> Your Subscription >> > _______________________________________________ > freebsd-fs@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" > > From owner-zfs-devel@freebsd.org Mon Mar 7 06:04:13 2016 Return-Path: Delivered-To: zfs-devel@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C4164AC23CF; Mon, 7 Mar 2016 06:04:13 +0000 (UTC) (envelope-from richard.elling@gmail.com) Received: from mail-pf0-x22c.google.com (mail-pf0-x22c.google.com [IPv6:2607:f8b0:400e:c00::22c]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8EABFE31; Mon, 7 Mar 2016 06:04:13 +0000 (UTC) (envelope-from richard.elling@gmail.com) Received: by mail-pf0-x22c.google.com with SMTP id x188so49825700pfb.2; Sun, 06 Mar 2016 22:04:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:subject:from:in-reply-to:date:cc:message-id:references :to; bh=inFEdDfH3mpHfQnxUD98ZBD5iQPHBkT3TlVx8a+LMKc=; b=Qe7MXPeW1fHNefC6mKPlUO0myUWtk9rFy4YQsDEtJwBtcQrFESb8d5S6JXm39VCRTJ mlnuxqRC9xZ1AtYyLMHsaHOLa44TbY5A5T6fyYhPQzCnHPmtNk3a1ee0mxFycoTR6kNf XL77b304+ZLXPuo2BN1tFGONuG3GXReF+FhSbZlyLexyCkyItW/e2erD5m8Lwb7xx6JW xQXeTFEBBcYpVtZRGK9y2K37xNy3Rw+H71LP4WtKLwzzmryeKzxOsUs+5u/lliBKk+Cp 7+V3iYfxC7lBDZJp5LeWAFhk7I6pCHsYyCgfvP9EiCNjf5spGL7pPZ5z+gBiTxb84coq LKSQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:subject:from:in-reply-to:date:cc :message-id:references:to; bh=inFEdDfH3mpHfQnxUD98ZBD5iQPHBkT3TlVx8a+LMKc=; b=iJkjJ8wNEZgFZp2mhIZWJ6cA0NySS9IiJ75rBcqsZAf2sZ2OgtEOeW3RZLuGhbB0z7 sc/BBuE66qbokd99UJVS9zz2QpSIt3EwxYY9Z6PXiueZBaYrtMgSan3Wz4po2l/0F0wk 7oyQRBjIpj+6Afo7cLQo10R5BwdvqJts4xxB9x7wq7KQAAOgAOx+WcGic3QZBjnYagg7 x7Fur6JT4+eMmA0nfLJhc2rvyMZ0nD/htAnKHgbd1kJz1vKCXOVHawjGoqHYnS0PO3Mg iEuOvm+ZQSJmH3Df4WFJ4ieDVWIubJrUI0C2wPtLaDQSIYcmzUOLSspCZlJLgJk/BXBw Z24A== X-Gm-Message-State: AD7BkJJTZyaK5yojyae5gPvbGxJ7paFF+cLs5n1WrlAJdoaib6IzrOVzWNosgZlrx380HQ== X-Received: by 10.98.75.196 with SMTP id d65mr30968996pfj.96.1457330652928; Sun, 06 Mar 2016 22:04:12 -0800 (PST) Received: from [192.168.129.108] ([162.250.162.10]) by smtp.gmail.com with ESMTPSA id n68sm21255445pfj.46.2016.03.06.22.04.10 (version=TLS1 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Sun, 06 Mar 2016 22:04:11 -0800 (PST) Mime-Version: 1.0 (Mac OS X Mail 9.2 \(3112\)) Subject: Re: [zfs] [developer] Re: [smartos-discuss] an interesting survey -- the zpool with most disks you have ever built From: Richard Elling In-Reply-To: Date: Sun, 6 Mar 2016 22:04:09 -0800 Cc: developer@lists.open-zfs.org, "smartos-discuss@lists.smartos.org" , developer , illumos-developer , omnios-discuss , Discussion list for OpenIndiana , "zfs-discuss@list.zfsonlinux.org" , "freebsd-fs@FreeBSD.org" , "zfs-devel@freebsd.org" Message-Id: <6E2B77D1-E0CA-4901-A6BD-6A22C07536B3@gmail.com> References: <95563acb-d27b-4d4b-b8f3-afeb87a3d599@me.com> <56D87784.4090103@broken.net> <5158F354-9636-4031-9536-E99450F312B3@RichardElling.com> To: zfs@lists.illumos.org X-Mailer: Apple Mail (2.3112) X-Mailman-Approved-At: Mon, 07 Mar 2016 12:55:21 +0000 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.21 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Mar 2016 06:04:14 -0000 > On Mar 6, 2016, at 9:06 PM, Fred Liu wrote: >=20 >=20 >=20 > 2016-03-06 22:49 GMT+08:00 Richard Elling = >: >=20 >> On Mar 3, 2016, at 8:35 PM, Fred Liu > wrote: >>=20 >> Hi, >>=20 >> Today when I was reading Jeff's new nuclear weapon -- DSSD D5's CUBIC = RAID introduction, >> the interesting survey -- the zpool with most disks you have ever = built popped in my brain. >=20 > We test to 2,000 drives. Beyond 2,000 there are some scalability = issues that impact failover times. > We=E2=80=99ve identified these and know what to fix, but need a real = customer at this scale to bump it to > the top of the priority queue. >=20 > [Fred]: Wow! 2000 drives almost need 4~5 whole racks!=20 >>=20 >> For zfs doesn't support nested vdev, the maximum fault tolerance = should be three(from raidz3). >=20 > Pedantically, it is N, because you can have N-way mirroring. > =20 > [Fred]: Yeah. That is just pedantic. N-way mirroring of every disk = works in theory and rarely happens in reality. >=20 >> It is stranded if you want to build a very huge pool. >=20 > Scaling redundancy by increasing parity improves data loss protection = by about 3 orders of=20 > magnitude. Adding capacity by striping reduces data loss protection by = 1/N. This is why there is > not much need to go beyond raidz3. However, if you do want to go = there, adding raidz4+ is=20 > relatively easy. >=20 > [Fred]: I assume you used stripped raidz3 vedvs in your storage mesh = of 2000 drives. If that is true, the possibility of 4/2000 will be not = so low. > Plus, reslivering takes longer time if single disk has = bigger capacity. And further, the cost of over-provisioning spare disks = vs raidz4+ will be an deserved=20 > trade-off when the storage mesh at the scale of 2000 = drives. Please don't assume, you'll just hurt yourself :-) For example, do not assume the only option is striping across raidz3 = vdevs. Clearly, there are many different options. -- richard From owner-zfs-devel@freebsd.org Mon Mar 7 06:18:28 2016 Return-Path: Delivered-To: zfs-devel@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 190ADAC2A17; Mon, 7 Mar 2016 06:18:28 +0000 (UTC) (envelope-from fred.fliu@gmail.com) Received: from mail-lb0-x229.google.com (mail-lb0-x229.google.com [IPv6:2a00:1450:4010:c04::229]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8388D85A; Mon, 7 Mar 2016 06:18:27 +0000 (UTC) (envelope-from fred.fliu@gmail.com) Received: by mail-lb0-x229.google.com with SMTP id k15so118818080lbg.0; Sun, 06 Mar 2016 22:18:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc; bh=hRaNaJ1s+015Wsux9tpeW+6Al4cZnb6HjDhJ5AJXYeM=; b=OaXBgVjsiH/6SXCth3iH46uTUuEaakhIpjW/vYBGuX4LmOLz+qk1nKovyk4fb5rVAQ TlyjFavLvNYAoTJ2CaePv04c4Oe46G+KO6nnbFwKborqlXnF0wK2YXLFSdY/+Ufb0gjE gAfMat1VRxrSnojZdSkTP5IGuHVS1pNWBWsw+GCVlht4Bp5Hm+NUd64hEefeC8/wLtYq assCmQnb3dGLzlFeSNv/lC8kDNsaQZl8ys9XTtsIimCA6h/OlRx5+csD49yflkuw7+5/ ezckM/6ya9hPI0QN6O3Ci+RmDkn+pOZEb+TT0CSbBtmsZvZvcOSp8IFYVDTC+3iw3jhj Gs2A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc; bh=hRaNaJ1s+015Wsux9tpeW+6Al4cZnb6HjDhJ5AJXYeM=; b=KBPD8u9257eB0oijoVnlns2mRsyrEXFz3tAPpn+SbK8ekhiFue1IqZhfCLqEp8RIXW XQE3Q43JiVeRgMI0h7JEZ2ajBcH74hhISO+PQYI2at/GPBIPJ55NVqeTSIRaj8zrhhXE zdMBlyPggAJyJwSxHjck1COndXpgw3Zyg65TOOUn2v+FZg1aA7CgmyjshdD4S0F7YtjU dSNuZdqIdOY6yp6x6ehepgYBc7K+aqwpn1Wk/MGCEJZIqVJ6snvDnenUpgiru5cczlxg poErzc6iFXFBypTzadB8TBe7PgEZ46tAvaYPxzD4h52hOB081BBTDzmWM7ZaV12TYlct hP3w== X-Gm-Message-State: AD7BkJIjlRtLbTkstNgxxxpEK/uKwqo1Vr9xiYWvxzISwhloFda0No8u4EfY37nV5Y0i2en+j8kCwKzzc5Gmgg== MIME-Version: 1.0 X-Received: by 10.112.149.73 with SMTP id ty9mr5563688lbb.48.1457331505362; Sun, 06 Mar 2016 22:18:25 -0800 (PST) Received: by 10.25.20.164 with HTTP; Sun, 6 Mar 2016 22:18:24 -0800 (PST) In-Reply-To: <6E2B77D1-E0CA-4901-A6BD-6A22C07536B3@gmail.com> References: <95563acb-d27b-4d4b-b8f3-afeb87a3d599@me.com> <56D87784.4090103@broken.net> <5158F354-9636-4031-9536-E99450F312B3@RichardElling.com> <6E2B77D1-E0CA-4901-A6BD-6A22C07536B3@gmail.com> Date: Mon, 7 Mar 2016 14:18:24 +0800 Message-ID: Subject: Re: [zfs] [developer] Re: [smartos-discuss] an interesting survey -- the zpool with most disks you have ever built From: Fred Liu To: "smartos-discuss@lists.smartos.org" Cc: illumos-zfs , developer@lists.open-zfs.org, developer , illumos-developer , omnios-discuss , Discussion list for OpenIndiana , "zfs-discuss@list.zfsonlinux.org" , "freebsd-fs@FreeBSD.org" , "zfs-devel@freebsd.org" X-Mailman-Approved-At: Mon, 07 Mar 2016 13:15:52 +0000 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.21 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Mar 2016 06:18:28 -0000 2016-03-07 14:04 GMT+08:00 Richard Elling : > > On Mar 6, 2016, at 9:06 PM, Fred Liu wrote: > > > > 2016-03-06 22:49 GMT+08:00 Richard Elling < > richard.elling@richardelling.com>: > >> >> On Mar 3, 2016, at 8:35 PM, Fred Liu wrote: >> >> Hi, >> >> Today when I was reading Jeff's new nuclear weapon -- DSSD D5's CUBIC >> RAID introduction, >> the interesting survey -- the zpool with most disks you have ever built >> popped in my brain. >> >> >> We test to 2,000 drives. Beyond 2,000 there are some scalability issues >> that impact failover times. >> We=E2=80=99ve identified these and know what to fix, but need a real cus= tomer at >> this scale to bump it to >> the top of the priority queue. >> >> [Fred]: Wow! 2000 drives almost need 4~5 whole racks! > >> >> For zfs doesn't support nested vdev, the maximum fault tolerance should >> be three(from raidz3). >> >> >> Pedantically, it is N, because you can have N-way mirroring. >> > > [Fred]: Yeah. That is just pedantic. N-way mirroring of every disk works > in theory and rarely happens in reality. > >> >> It is stranded if you want to build a very huge pool. >> >> >> Scaling redundancy by increasing parity improves data loss protection by >> about 3 orders of >> magnitude. Adding capacity by striping reduces data loss protection by >> 1/N. This is why there is >> not much need to go beyond raidz3. However, if you do want to go there, >> adding raidz4+ is >> relatively easy. >> > > [Fred]: I assume you used stripped raidz3 vedvs in your storage mesh of > 2000 drives. If that is true, the possibility of 4/2000 will be not so lo= w. > Plus, reslivering takes longer time if single disk has bigger > capacity. And further, the cost of over-provisioning spare disks vs raidz= 4+ > will be an deserved > trade-off when the storage mesh at the scale of 2000 drives. > > > Please don't assume, you'll just hurt yourself :-) > For example, do not assume the only option is striping across raidz3 > vdevs. Clearly, there are many > different options. > [Fred]: Yeah. Assumptions always go far way from facts! ;-) Is designing a storage mesh with 2000 drives biz secret? Or it is just too complicate to elaborate? Never mind. ;-) Thanks. Fred > > *smartos-discuss* | Archives > > | > Modify > > Your Subscription > From owner-zfs-devel@freebsd.org Mon Mar 7 20:55:48 2016 Return-Path: Delivered-To: zfs-devel@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 368BCAC2594; Mon, 7 Mar 2016 20:55:48 +0000 (UTC) (envelope-from lslusser@gmail.com) Received: from mail-vk0-x22c.google.com (mail-vk0-x22c.google.com [IPv6:2607:f8b0:400c:c05::22c]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id DCE601CC6; Mon, 7 Mar 2016 20:55:47 +0000 (UTC) (envelope-from lslusser@gmail.com) Received: by mail-vk0-x22c.google.com with SMTP id c3so131819638vkb.3; Mon, 07 Mar 2016 12:55:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc; bh=DAhuTpXsSg6UCNBJKnagMJx92BEndeEQ0wRXhWRr/9o=; b=kOQWYCAWLGZ/k0dYMPNZDS1CMXMiL7MGjFIsvCSPnuWZcvZos6SqJFhg9l8TA9NxRM RYjf8cHlrYGFoBEOtiJKH8gguoulSrFw8DRGzd0lHvHg+WQ94W/vEALijLQvV3LvGX+S 0f2AwOhVAVDJ0gYnuuPn+g3qTXra1Bk7auF6r52RA634XmvA7erO0tzuUbGBC7OJ34lt QCSBkuy9h8GrLAax23XSiFU5yCH24RHpmsqHTd6JQqEp0eaxlmHF108UTei8H1AIm+k1 7GxNEUqg/FhngQt8lAwSz3geAtIGaNY13agGJshbDMWKp8yjwG88d2V8YMXJRicUNVlz djaQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc; bh=DAhuTpXsSg6UCNBJKnagMJx92BEndeEQ0wRXhWRr/9o=; b=i/tWMkf2vl4Q0DVj+ikWF0O9vhczKg6FWkrxI5e6WD3HgxMOY03yL9yxwJ8JGZfvIn BeYjXA8Maa9wbmqObHZnQsKMEfnCORAMLxziIuzU3rl6rJ8NUs5JOtg5cLVa678PR9XH ZSSSE/rqPVQe6wHlSAPVJ5vE4wCkOlhUOJE7ipoq4vU/oPBmvb8RekQIaGeMjE7h+G7v mkZhpSUpubB6FbTYvDRaJT1e3iATCFS8ZafzL4TPPbf2ptLTEkcD+npoIXsodRh9/fO4 NLuqoH3wk8vlVAv0GC50qVgqUc+x1WSuvV6cI689n8oh59KGlGCXilFQw3VZuEm1nLrS XYJg== X-Gm-Message-State: AD7BkJLIjb4LJEWKPqsH2IZtPmrLHwaLBwEa3Gdw8OkJyxJUSwX4lg+Gm3piTZq0OFV4WGHx4zKlLAZgy8dtVA== MIME-Version: 1.0 X-Received: by 10.31.174.23 with SMTP id x23mr20192994vke.136.1457384146585; Mon, 07 Mar 2016 12:55:46 -0800 (PST) Received: by 10.176.1.166 with HTTP; Mon, 7 Mar 2016 12:55:46 -0800 (PST) In-Reply-To: References: <95563acb-d27b-4d4b-b8f3-afeb87a3d599@me.com> <56D87784.4090103@broken.net> <5158F354-9636-4031-9536-E99450F312B3@RichardElling.com> <6E2B77D1-E0CA-4901-A6BD-6A22C07536B3@gmail.com> Date: Mon, 7 Mar 2016 12:55:46 -0800 Message-ID: Subject: Re: [zfs] [developer] Re: [smartos-discuss] an interesting survey -- the zpool with most disks you have ever built From: Liam Slusser To: zfs@lists.illumos.org Cc: "smartos-discuss@lists.smartos.org" , developer@lists.open-zfs.org, developer , illumos-developer , omnios-discuss , Discussion list for OpenIndiana , "zfs-discuss@list.zfsonlinux.org" , "freebsd-fs@FreeBSD.org" , "zfs-devel@freebsd.org" X-Mailman-Approved-At: Mon, 07 Mar 2016 22:00:32 +0000 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.21 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Mar 2016 20:55:48 -0000 I don't have a 2000 drive array (thats amazing!) but I do have two 280 drive arrays which are in production. Here are the generic stats: server setup: OpenIndiana oi_151 1 server rack Dell r720xd 64g ram with mirrored 250g boot disks 5 x LSI 9207-8e dualport SAS pci-e host bus adapters Intel 10g fibre ethernet (dual port) 2 x SSD for log cache 2 x SSD for cache 23 x Dell MD1200 with 3T,4T, or 6T NLSAS disks (a mix of Toshiba, Western Digital, and Seagate drives - basically whatever Dell sends) zpool setup: 23 x 12-disk raidz2 glued together. 276 total disks. Basically each new 12 disk MD1200 is a new raidz2 added to the pool. Total size: ~797T We have an identical server which we replicate changes via zfs snapshots every few minutes. The whole setup as been up and running for a few years now, no issues. As we run low on space we purchase two additional MD1200 shelfs (one for each system) and add the new raidz2 into pool on-the-fly. The only real issues we've had is sometimes a disk fails in such a way (think Monty Python and the holy grail i'm not dead yet) where the disk hasn't failed but is timing out and slows the whole array to a standstill until we can manual find and remove the disk. Other problems are once a disk has been replaced sometimes the resilver process can take an eternity. We have also found the snapshot replication process can interfere with the resilver process - resilver gets stuck at 99% and never ends - so we end up stopping or only doing one replication a day until the resilver process is done. The last helpful hint I have was lowering all the drive timeouts, see http://everycity.co.uk/alasdair/2011/05/adjusting-drive-timeouts-with-mdb-o= n-solaris-or-openindiana/ for info. thanks, liam On Sun, Mar 6, 2016 at 10:18 PM, Fred Liu wrote: > > > 2016-03-07 14:04 GMT+08:00 Richard Elling : > >> >> On Mar 6, 2016, at 9:06 PM, Fred Liu wrote: >> >> >> >> 2016-03-06 22:49 GMT+08:00 Richard Elling < >> richard.elling@richardelling.com>: >> >>> >>> On Mar 3, 2016, at 8:35 PM, Fred Liu wrote: >>> >>> Hi, >>> >>> Today when I was reading Jeff's new nuclear weapon -- DSSD D5's CUBIC >>> RAID introduction, >>> the interesting survey -- the zpool with most disks you have ever built >>> popped in my brain. >>> >>> >>> We test to 2,000 drives. Beyond 2,000 there are some scalability issues >>> that impact failover times. >>> We=E2=80=99ve identified these and know what to fix, but need a real cu= stomer at >>> this scale to bump it to >>> the top of the priority queue. >>> >>> [Fred]: Wow! 2000 drives almost need 4~5 whole racks! >> >>> >>> For zfs doesn't support nested vdev, the maximum fault tolerance should >>> be three(from raidz3). >>> >>> >>> Pedantically, it is N, because you can have N-way mirroring. >>> >> >> [Fred]: Yeah. That is just pedantic. N-way mirroring of every disk works >> in theory and rarely happens in reality. >> >>> >>> It is stranded if you want to build a very huge pool. >>> >>> >>> Scaling redundancy by increasing parity improves data loss protection b= y >>> about 3 orders of >>> magnitude. Adding capacity by striping reduces data loss protection by >>> 1/N. This is why there is >>> not much need to go beyond raidz3. However, if you do want to go there, >>> adding raidz4+ is >>> relatively easy. >>> >> >> [Fred]: I assume you used stripped raidz3 vedvs in your storage mesh of >> 2000 drives. If that is true, the possibility of 4/2000 will be not so l= ow. >> Plus, reslivering takes longer time if single disk has bigger >> capacity. And further, the cost of over-provisioning spare disks vs raid= z4+ >> will be an deserved >> trade-off when the storage mesh at the scale of 2000 drives. >> >> >> Please don't assume, you'll just hurt yourself :-) >> For example, do not assume the only option is striping across raidz3 >> vdevs. Clearly, there are many >> different options. >> > > [Fred]: Yeah. Assumptions always go far way from facts! ;-) Is designing > a storage mesh with 2000 drives biz secret? Or it is just too complicate = to > elaborate? > Never mind. ;-) > > Thanks. > > Fred > > >> >> > *illumos-zfs* | Archives > > | > Modify > > Your Subscription > From owner-zfs-devel@freebsd.org Tue Mar 8 03:50:05 2016 Return-Path: Delivered-To: zfs-devel@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E8671AC729E; Tue, 8 Mar 2016 03:50:05 +0000 (UTC) (envelope-from fred.fliu@gmail.com) Received: from mail-lb0-x234.google.com (mail-lb0-x234.google.com [IPv6:2a00:1450:4010:c04::234]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 638D6226; Tue, 8 Mar 2016 03:50:05 +0000 (UTC) (envelope-from fred.fliu@gmail.com) Received: by mail-lb0-x234.google.com with SMTP id k15so3514826lbg.0; Mon, 07 Mar 2016 19:50:05 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-transfer-encoding; bh=AoYsfHa27fdhsl2RmI5NGmoUqHyo9gJQVgjAupcGz9s=; b=SmDzHUhxcsfiPWKlfWhX7hALhgwDwMQI+5J7yn2ls4eRe+pQYpfhNZHLnad6gp6w4m 9hUBSpC7uGb455smvxMcLhIHrcXMoAJbds8DZrbuYYWJ6tYGmIVfQ79JgBqlO9p3RUsa Dd/6AjIKSNG7hMZo4PK01/aCTftzDJ0rGGRHk5st46MwMIBZTWVGj88u9mbrOCxvVmgt yoP750zTlqYPmT9imQeMVQB9mN52jrj/RgxxXkr9RMdI84mBXNXvuUUQ/gLMQhHvsz2U YGdbjJJ11qNDh24BXXboMAOBmxM6ulYfrys0YkV3zVHO7zANhilgNtwDS40iqybnoA74 /iTQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-transfer-encoding; bh=AoYsfHa27fdhsl2RmI5NGmoUqHyo9gJQVgjAupcGz9s=; b=fZWHiH7eSwSQWQlVVjGPI1GWUU73o6UqxmcjeAlP6GHizg+xeJWqnBuVcNEbQ00Wuo bj/7v0evQ5a9Q07Tn+o8UzZzpdbt6kK4q7Qp8inFU5svP9gRldO2BQdOPimd0xzQWEq6 qHLYaodODtYWweCeTgw3SVE7K5ZIzhRjWie4qhAZ1QfM/nEJ6Hsm/Npii5EdgSno4gGL 8AriGs08TlYFqgH1+n3YEPovHLDIB4q1Cb1Qcd4fFS6qQWdfuTbyO49OuhdYjOftQ6YR g4ySEUTPCuxTj50eCB8aV/Uml7Jfr9Ecela/tST4lJCbCbNhnZiAkAyQyDsNpR3VddLv 7k6w== X-Gm-Message-State: AD7BkJIRYiDViWsa2bhIob0AXIs8cZut4C1nOmLW4VQAVSHQkVWvLqR8gL9XelW0Nvvjbsv6SgDieV61T7Tmmg== MIME-Version: 1.0 X-Received: by 10.112.162.231 with SMTP id yd7mr8855063lbb.40.1457409001735; Mon, 07 Mar 2016 19:50:01 -0800 (PST) Received: by 10.25.20.164 with HTTP; Mon, 7 Mar 2016 19:50:01 -0800 (PST) In-Reply-To: <56DD17CA.90200@freebsd.org> References: <95563acb-d27b-4d4b-b8f3-afeb87a3d599@me.com> <56D87784.4090103@broken.net> <56DD17CA.90200@freebsd.org> Date: Tue, 8 Mar 2016 11:50:01 +0800 Message-ID: Subject: Re: [zfs] an interesting survey -- the zpool with most disks you have ever built From: Fred Liu To: Julian Elischer Cc: illumos-zfs , Discussion list for OpenIndiana , omnios-discuss , developer , "zfs-devel@freebsd.org" , illumos-developer , "freebsd-fs@FreeBSD.org" , "smartos-discuss@lists.smartos.org" , "zfs-discuss@list.zfsonlinux.org" Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Mailman-Approved-At: Tue, 08 Mar 2016 13:02:40 +0000 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Mar 2016 03:50:06 -0000 2016-03-07 13:55 GMT+08:00 Julian Elischer : > On 6/03/2016 9:30 PM, Fred Liu wrote: >> >> 2016-03-05 0:01 GMT+08:00 Freddie Cash : >> >>> On Mar 4, 2016 2:05 AM, "Fred Liu" wrote: >>>> >>>> 2016-03-04 13:47 GMT+08:00 Freddie Cash : >>>>> >>>>> Currently, I just use a simple coordinate system. Columns are letters= , >>> >>> rows are numbers. >>>>> >>>>> "smartos-discuss@lists.smartos.org" >>> >>>> =E3=80=81 >>> >>> developer =E3=80=81 >>> >>> illumos-developer =E3=80=81 >>> >>> omnios-discuss =E3=80=81 >>> >>> Discussion list for OpenIndiana = =E3=80=81 >>> >>> illumos-zfs =E3=80=81 >>> >>> "zfs-discuss@list.zfsonlinux.org" =E3= =80=81 >>> >>> "freebsd-fs@FreeBSD.org" =E3=80=81 >>> >>> "zfs-devel@freebsd.org" >>> >>>>> Each disk is partitioned using GPT with the first (only) partition >>> >>> starting at 1 MB and covering the whole disk, and labelled with the >>> column/row where it is located (disk-a1, disk-g6, disk-p3, etc). >>>> >>>> [Fred]: So you manually pull off all the drives one by one to locate >>> >>> them? >>> >>> When putting the system together for the first time, I insert each disk >>> one at a time, wait for it to be detected, partition it, then label it >>> based on physical location. Then do the next one. It's just part of t= he >>> normal server build process, whether it has 2 drives, 20 drives, or 200 >>> drives. >>> >>> We build all our own servers from off-the-shelf parts; we don't buy >>> anything pre-built from any of the large OEMs. >>> >> [Fred]: Gotcha! >> >> >>>>> The pool is created using the GPT labels, so the label shows in "zpoo= l >>> >>> list" output. >>>> >>>> [Fred]: What will the output look like? >>> >>> From our smaller backups server, with just 24 drive bays: >>> >>> $ zpool status storage >>> >>> pool: storage >>> >>> state: ONLINE >>> >>> status: Some supported features are not enabled on the pool. The pool c= an >>> >>> still be used, but some features are unavailable. >>> >>> action: Enable all features using 'zpool upgrade'. Once this is done, >>> >>> the pool may no longer be accessible by software that does not support >>> >>> the features. See zpool-features(7) for details. >>> >>> scan: scrub canceled on Wed Feb 17 12:02:20 2016 >>> >>> config: >>> >>> >>> NAME STATE READ WRITE CKSUM >>> >>> storage ONLINE 0 0 0 >>> >>> raidz2-0 ONLINE 0 0 0 >>> >>> gpt/disk-a1 ONLINE 0 0 0 >>> >>> gpt/disk-a2 ONLINE 0 0 0 >>> >>> gpt/disk-a3 ONLINE 0 0 0 >>> >>> gpt/disk-a4 ONLINE 0 0 0 >>> >>> gpt/disk-a5 ONLINE 0 0 0 >>> >>> gpt/disk-a6 ONLINE 0 0 0 >>> >>> raidz2-1 ONLINE 0 0 0 >>> >>> gpt/disk-b1 ONLINE 0 0 0 >>> >>> gpt/disk-b2 ONLINE 0 0 0 >>> >>> gpt/disk-b3 ONLINE 0 0 0 >>> >>> gpt/disk-b4 ONLINE 0 0 0 >>> >>> gpt/disk-b5 ONLINE 0 0 0 >>> >>> gpt/disk-b6 ONLINE 0 0 0 >>> >>> raidz2-2 ONLINE 0 0 0 >>> >>> gpt/disk-c1 ONLINE 0 0 0 >>> >>> gpt/disk-c2 ONLINE 0 0 0 >>> >>> gpt/disk-c3 ONLINE 0 0 0 >>> >>> gpt/disk-c4 ONLINE 0 0 0 >>> >>> gpt/disk-c5 ONLINE 0 0 0 >>> >>> gpt/disk-c6 ONLINE 0 0 0 >>> >>> raidz2-3 ONLINE 0 0 0 >>> >>> gpt/disk-d1 ONLINE 0 0 0 >>> >>> gpt/disk-d2 ONLINE 0 0 0 >>> >>> gpt/disk-d3 ONLINE 0 0 0 >>> >>> gpt/disk-d4 ONLINE 0 0 0 >>> >>> gpt/disk-d5 ONLINE 0 0 0 >>> >>> gpt/disk-d6 ONLINE 0 0 0 >>> >>> cache >>> >>> gpt/cache0 ONLINE 0 0 0 >>> >>> gpt/cache1 ONLINE 0 0 0 >>> >>> >>> errors: No known data errors >>> >>> The 90-bay systems look the same, just that the letters go all the way = to >>> p (so disk-p1 through disk-p6). And there's one vdev that uses 3 drive= s >>> from each chassis (7x 6-disk vdev only uses 42 drives of the 45-bay >>> chassis, so there's lots of spares if using a single chassis; using two >>> chassis, there's enough drives to add an extra 6-disk vdev). >>> >> [Fred]: It looks like the gpt label shown in "zpool status" only works i= n >> FreeBSD/FreeNAS. Are you using FreeBSD/FreeNAS? I can't find the similar >> possibilities in Illumos/Linux. > > > Ah that's a trick.. FreeBSD exports an actual /dev/gpt/{you-label-goes-he= re} > for each labeled partition it finds. > So it's not ZFS doing anything special.. it's what FreeBSD is calling the > partition. > Super cool! Fred From owner-zfs-devel@freebsd.org Tue Mar 8 03:56:23 2016 Return-Path: Delivered-To: zfs-devel@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 89035AC74E3; Tue, 8 Mar 2016 03:56:23 +0000 (UTC) (envelope-from fred.fliu@gmail.com) Received: from mail-lb0-x22a.google.com (mail-lb0-x22a.google.com [IPv6:2a00:1450:4010:c04::22a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 1457D764; Tue, 8 Mar 2016 03:56:23 +0000 (UTC) (envelope-from fred.fliu@gmail.com) Received: by mail-lb0-x22a.google.com with SMTP id k15so3620913lbg.0; Mon, 07 Mar 2016 19:56:22 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc; bh=OaFv1WIP9IS0uwcRqw1S9Sn0qI/rckbXP42M6r28SYU=; b=oGeVYvu0J03Ao//K4hcpZuM9nOLK38PIHyZlLYcgPDAM7kjE5fgsRSFwFRKa1hwtFo +lepalhJXSTfpo7X3RG2yhNYikpREkzZD+kF/LrWySvSdQVXc4vC+A1Ovw1dby9BaUDT QVJEJ+51WvJB6L0zVFEvOtyquK9LjCz0MjcKOU0YaP7xkh4sYmfwFgQ9hyU/8eq5zWV2 BC6ZwG2GYSMIXL5IbfncxYB2geXtNqWtBFnmySycoVTP27N2qUR9Gb7TAxVCOjzTG8an 9T5ZLEF1v1Ve3HP1eoQacQblbGBrIUML+8U46MXe31jcuiXjho5MA47Zf8R8sTFUqIap U/4w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc; bh=OaFv1WIP9IS0uwcRqw1S9Sn0qI/rckbXP42M6r28SYU=; b=Ksm3drqv4GhDm7pyES63WZJXsxJwzAyfEE9zRftRHvJ5BhTmCzOdiPbssyWJT51s3/ B6yyb32YP40jbGIExuMPsLNY831msu5i+VNLMZDr6/XuPvbil5PHiRrjovik7zQjuaa+ Qvo2VHpi3yB5xtYM/KccuLjUnLzH4t7kXXCWYMzWVCUoZgruyynPg5bw/I1vC4ifVsbL RN5fYV0dYZjUM28QeMv8FiKSIFeRlP9J6SpKXQEndEpQpMEJejexopfgs87Rjoz5XnqK pkfx+yRmhP/OiXAbQJxb/H6OEJhXkIZPvJtmU1d+522zH1DXCQ1T+dEvKfXf/xgFoUVI TpCg== X-Gm-Message-State: AD7BkJI4G93ofocIaYuYHibIaoGVlrPQrOE4zG0U7s7dNePl89gEqpCi5937GQzC55Rh9PPBkbzz+bRAD+xEPA== MIME-Version: 1.0 X-Received: by 10.25.161.131 with SMTP id k125mr9032768lfe.83.1457409380752; Mon, 07 Mar 2016 19:56:20 -0800 (PST) Received: by 10.25.20.164 with HTTP; Mon, 7 Mar 2016 19:56:20 -0800 (PST) In-Reply-To: References: <95563acb-d27b-4d4b-b8f3-afeb87a3d599@me.com> <56D87784.4090103@broken.net> <5158F354-9636-4031-9536-E99450F312B3@RichardElling.com> <6E2B77D1-E0CA-4901-A6BD-6A22C07536B3@gmail.com> Date: Tue, 8 Mar 2016 11:56:20 +0800 Message-ID: Subject: Re: [zfs] [developer] Re: [smartos-discuss] an interesting survey -- the zpool with most disks you have ever built From: Fred Liu To: illumos-zfs Cc: "smartos-discuss@lists.smartos.org" , developer@lists.open-zfs.org, developer , illumos-developer , omnios-discuss , Discussion list for OpenIndiana , "zfs-discuss@list.zfsonlinux.org" , "freebsd-fs@FreeBSD.org" , "zfs-devel@freebsd.org" X-Mailman-Approved-At: Tue, 08 Mar 2016 13:03:08 +0000 Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.21 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Mar 2016 03:56:23 -0000 2016-03-08 4:55 GMT+08:00 Liam Slusser : > I don't have a 2000 drive array (thats amazing!) but I do have two 280 > drive arrays which are in production. Here are the generic stats: > > server setup: > OpenIndiana oi_151 > 1 server rack > Dell r720xd 64g ram with mirrored 250g boot disks > 5 x LSI 9207-8e dualport SAS pci-e host bus adapters > Intel 10g fibre ethernet (dual port) > 2 x SSD for log cache > 2 x SSD for cache > 23 x Dell MD1200 with 3T,4T, or 6T NLSAS disks (a mix of Toshiba, Western > Digital, and Seagate drives - basically whatever Dell sends) > > zpool setup: > 23 x 12-disk raidz2 glued together. 276 total disks. Basically each new > 12 disk MD1200 is a new raidz2 added to the pool. > > Total size: ~797T > > We have an identical server which we replicate changes via zfs snapshots > every few minutes. The whole setup as been up and running for a few years > now, no issues. As we run low on space we purchase two additional MD1200 > shelfs (one for each system) and add the new raidz2 into pool on-the-fly. > > The only real issues we've had is sometimes a disk fails in such a way > (think Monty Python and the holy grail i'm not dead yet) where the disk > hasn't failed but is timing out and slows the whole array to a standstill > until we can manual find and remove the disk. Other problems are once a > disk has been replaced sometimes the resilver process can take > an eternity. We have also found the snapshot replication process can > interfere with the resilver process - resilver gets stuck at 99% and never > ends - so we end up stopping or only doing one replication a day until the > resilver process is done. > > The last helpful hint I have was lowering all the drive timeouts, see > http://everycity.co.uk/alasdair/2011/05/adjusting-drive-timeouts-with-mdb-on-solaris-or-openindiana/ > for info. > > [Fred]: zpool wiith 280 drives in production is pretty big! I think 2000 > drives were just in test. It is true that huge pools have lots of operation > challenges. I have met the similar sluggish issue caused by a > will-die disk. Just curious, what is the cluster software implemented in http://everycity.co.uk/alasdair/2011/05/adjusting-drive-timeouts-with-mdb-on-solaris-or-openindiana/ ? Thanks. Fred > > > >>> >>> >> > *illumos-zfs* | Archives > > | > Modify > > Your Subscription > From owner-zfs-devel@freebsd.org Wed Mar 9 00:05:15 2016 Return-Path: Delivered-To: zfs-devel@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 01192AC8800; Wed, 9 Mar 2016 00:05:15 +0000 (UTC) (envelope-from lslusser@gmail.com) Received: from mail-vk0-x236.google.com (mail-vk0-x236.google.com [IPv6:2607:f8b0:400c:c05::236]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A8741CD4; Wed, 9 Mar 2016 00:05:14 +0000 (UTC) (envelope-from lslusser@gmail.com) Received: by mail-vk0-x236.google.com with SMTP id e185so37352714vkb.1; Tue, 08 Mar 2016 16:05:14 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc; bh=WOk5WWP9pQwt2NV8ukBTmjl6VUtNWJPE1ke8x+HyNvo=; b=faWODljHmdJjv4BdBuQ9loa+2L2kTRfVKRhPPrR4bs1e7Q9C/aCVL+5b+GVnaABeaX MFX+5mwcZYhvmaJw8lW+N4Ru8ArKOvUxQmVdXa+mHF3UbCGSNqAq5MnH3HmTuYLxYpGY NtBOF3uff+QF0rvD4EV4tiTt799GFGuQJjbCuL11VFHauRYb7VnwJ+YiEnWei5p4DaIx vqVqfihbhiy4oLImVuGIWt9/9kJcmwS24VfQkwOgCixv3zvAwQiN7auoyLhEfwNm2lPq LzIdIdtdJUoxH8VEinzX29dKN+h9n0fyELAjq1UhF4vDyDFU84J6xqwf5a6wtl4YBK8n j6LA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc; bh=WOk5WWP9pQwt2NV8ukBTmjl6VUtNWJPE1ke8x+HyNvo=; b=b4esiJHjETCB0/BFj3DUqQZKFv6MDvCK1cjJOCS4Ycw6Qw9PycGsQNXdzgJ4IK63VY 7BDuXGBcMiQLnyS8Oz1zQ1w76gylUvYUV4ra626jdy/DpNGcBQsz+6XWmLzi9WnnXpZH pu6mAnevOsrGZFEv/bkEnGVJCjQwOuqCHucJSQxETVoLzPf2SA6ETb11HciF2TJo+w2U kBgF6m2p0CM938Xmuu/AM1s9zEaOAywWNYg++Ga0rP+ZTcldBss4lGRnSr7rh7OPibXZ 0ZfdV5Yqb0KEv18RGmGtlZArMqH3CcA6lOZSlCwNwWKaQ5UcmoeFj957IhYXH761EwsC skQw== X-Gm-Message-State: AD7BkJKSHCullOJPaIzi+3vF3EJjJgb84Yd9sJzc1WjvduRsOKEZOmhFJwUHFbzXeTEoGQZzKrWxCpPSt3XtXQ== MIME-Version: 1.0 X-Received: by 10.31.3.80 with SMTP id 77mr23505443vkd.17.1457481913545; Tue, 08 Mar 2016 16:05:13 -0800 (PST) Received: by 10.176.1.166 with HTTP; Tue, 8 Mar 2016 16:05:13 -0800 (PST) In-Reply-To: References: <95563acb-d27b-4d4b-b8f3-afeb87a3d599@me.com> <56D87784.4090103@broken.net> <5158F354-9636-4031-9536-E99450F312B3@RichardElling.com> <6E2B77D1-E0CA-4901-A6BD-6A22C07536B3@gmail.com> Date: Tue, 8 Mar 2016 16:05:13 -0800 Message-ID: Subject: Re: [zfs] [developer] Re: [smartos-discuss] an interesting survey -- the zpool with most disks you have ever built From: Liam Slusser To: zfs@lists.illumos.org Cc: "smartos-discuss@lists.smartos.org" , developer@lists.open-zfs.org, developer , illumos-developer , omnios-discuss , Discussion list for OpenIndiana , "zfs-discuss@list.zfsonlinux.org" , "freebsd-fs@FreeBSD.org" , "zfs-devel@freebsd.org" X-Mailman-Approved-At: Wed, 09 Mar 2016 01:05:13 +0000 Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.21 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Mar 2016 00:05:15 -0000 Hi Fred - We don't use any cluster software. Our backup server is just a full copy of our data and nothing more. So in the event of a failure of the master our server clients don't automatically fail over or anything nifty like that. This filer isn't customer facing, so in the event of a failure of the master there is no customer impact. We use a slightly modified zrep to handle the replication between the two. thanks, liam > [Fred]: zpool wiith 280 drives in production is pretty big! I think 2000 >> drives were just in test. It is true that huge pools have lots of operation >> challenges. I have met the similar sluggish issue caused by a >> > will-die disk. Just curious, what is the cluster software > implemented in > http://everycity.co.uk/alasdair/2011/05/adjusting-drive-timeouts-with-mdb-on-solaris-or-openindiana/ > ? > > Thanks. > > Fred > From owner-zfs-devel@freebsd.org Wed Mar 9 02:15:39 2016 Return-Path: Delivered-To: zfs-devel@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 41014AC8A4C; Wed, 9 Mar 2016 02:15:39 +0000 (UTC) (envelope-from rudd-o@rudd-o.com) Received: from mail.rudd-o.com (mail.rudd-o.com [54.255.149.57]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.rudd-o.com", Issuer "COMODO RSA Domain Validation Secure Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id EEC373BA; Wed, 9 Mar 2016 02:15:37 +0000 (UTC) (envelope-from rudd-o@rudd-o.com) Received: from [10.137.9.14] (ip-10-252-104-1.ap-southeast-1.compute.internal [10.252.104.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.rudd-o.com (Postfix) with ESMTPSA id C58166046D; Wed, 9 Mar 2016 02:08:22 +0000 (UTC) Subject: Re: [zfs] [developer] Re: [smartos-discuss] an interesting survey -- the zpool with most disks you have ever built To: developer@lists.open-zfs.org, zfs@lists.illumos.org References: <95563acb-d27b-4d4b-b8f3-afeb87a3d599@me.com> <56D87784.4090103@broken.net> <5158F354-9636-4031-9536-E99450F312B3@RichardElling.com> <6E2B77D1-E0CA-4901-A6BD-6A22C07536B3@gmail.com> Cc: "smartos-discuss@lists.smartos.org" , developer , illumos-developer , omnios-discuss , Discussion list for OpenIndiana , "zfs-discuss@list.zfsonlinux.org" , "freebsd-fs@FreeBSD.org" , "zfs-devel@freebsd.org" From: "Manuel Amador (Rudd-O)" Message-ID: <56DF8595.4090400@rudd-o.com> Date: Wed, 9 Mar 2016 02:08:21 +0000 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101 Thunderbird/38.6.0 MIME-Version: 1.0 In-Reply-To: Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="F12aOeHdhaeLGxjv5JtNeoWIVE58hV2li" X-Mailman-Approved-At: Wed, 09 Mar 2016 12:54:12 +0000 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Mar 2016 02:15:39 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --F12aOeHdhaeLGxjv5JtNeoWIVE58hV2li Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable On 03/09/2016 12:05 AM, Liam Slusser wrote: > > We use a slightly modified zrep to handle the replication between the t= wo. zrep? --=20 Rudd-O http://rudd-o.com/ --F12aOeHdhaeLGxjv5JtNeoWIVE58hV2li Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQIcBAEBCAAGBQJW34WVAAoJEFmZwbV7vYQ2L7gP/3pFIpy17j3dDEyq+QOInb5+ GNW7ffBGy3spJ+RMh8scQW0yU0DWtlALTy7TL+xIeo76CGtEMcO+bqqvGmzHgUlI W15800gih4Mt/z7buUx3rkUzivkTMFt8z4ElDWnaUUOvvRVHb3igMmg2LG1z3zFo Ntqux6CS77YJPxvCiNxzmP57oGuUvjt8XCptqMQtASrhpE5VA+S1KPdwQl9Rk/bu nr3c2a9Rx15coutNbgInVA6IF6oehVSngQH2twfP3/CzwuI5r3ZhZx+qOzLaUXMz +NPIHRvHyWJnSON1pHUqZYCNpu0g+n1GEQwJI7GX+ciKr1LdIlGcvIZb8Nmi7bWr 8tIE3Y5Hkznc6w0aja5FBkyHNUofb32FQRxdWobtBsEeqIz7p9kTA+j125zCsV6G Q4zKtsKN1j60h56RApW6/DqZtihpjQHNdT0kjLpvvNShGQt6NCgetHtqvNFM7gQo XULlVnlZPxsBUCb8qS8Fzi/Aii6MpeTbR1BKTnIhzwRDUTnHoSn76D6bX/DERGen xCdHjnixWaeve//UPY8hnfwQggGGxGbC8fmxIJfQ3grbRuNQnidS/K0zuLyjbO5w 0dDME3n286yl3eJigN6OX4kizzSOhjrjq0teD/M7i8Z64sksclI/BSYlPcW5i7Jg Gm+GntTYoK8G56VQ51ZK =mYb/ -----END PGP SIGNATURE----- --F12aOeHdhaeLGxjv5JtNeoWIVE58hV2li-- From owner-zfs-devel@freebsd.org Wed Mar 9 19:49:50 2016 Return-Path: Delivered-To: zfs-devel@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A4B05ACA896; Wed, 9 Mar 2016 19:49:50 +0000 (UTC) (envelope-from 000.fbsd@quip.cz) Received: from elsa.codelab.cz (elsa.codelab.cz [94.124.105.4]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 63784C5D; Wed, 9 Mar 2016 19:49:49 +0000 (UTC) (envelope-from 000.fbsd@quip.cz) Received: from elsa.codelab.cz (localhost [127.0.0.1]) by elsa.codelab.cz (Postfix) with ESMTP id A5E3128436; Wed, 9 Mar 2016 20:49:40 +0100 (CET) Received: from illbsd.quip.test (ip-86-49-16-209.net.upcbroadband.cz [86.49.16.209]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by elsa.codelab.cz (Postfix) with ESMTPSA id E148528412; Wed, 9 Mar 2016 20:49:37 +0100 (CET) Message-ID: <56E07E51.2010107@quip.cz> Date: Wed, 09 Mar 2016 20:49:37 +0100 From: Miroslav Lachman <000.fbsd@quip.cz> User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:35.0) Gecko/20100101 Firefox/35.0 SeaMonkey/2.32 MIME-Version: 1.0 To: "Manuel Amador (Rudd-O)" , developer@lists.open-zfs.org, zfs@lists.illumos.org CC: Discussion list for OpenIndiana , omnios-discuss , developer , "zfs-devel@freebsd.org" , illumos-developer , "freebsd-fs@FreeBSD.org" , "smartos-discuss@lists.smartos.org" , "zfs-discuss@list.zfsonlinux.org" Subject: Re: [zfs] [developer] Re: [smartos-discuss] an interesting survey -- the zpool with most disks you have ever built References: <95563acb-d27b-4d4b-b8f3-afeb87a3d599@me.com> <56D87784.4090103@broken.net> <5158F354-9636-4031-9536-E99450F312B3@RichardElling.com> <6E2B77D1-E0CA-4901-A6BD-6A22C07536B3@gmail.com> <56DF8595.4090400@rudd-o.com> In-Reply-To: <56DF8595.4090400@rudd-o.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Mailman-Approved-At: Wed, 09 Mar 2016 22:36:24 +0000 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Mar 2016 19:49:50 -0000 Manuel Amador (Rudd-O) wrote on 03/09/2016 03:08: > On 03/09/2016 12:05 AM, Liam Slusser wrote: >> >> We use a slightly modified zrep to handle the replication between the two. > > zrep? In ports sysutils/zrep - ZFS based replication and failover solution WWW: http://www.bolthole.com/solaris/zrep/ From owner-zfs-devel@freebsd.org Thu Mar 10 16:19:11 2016 Return-Path: Delivered-To: zfs-devel@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3199BAC96E1; Thu, 10 Mar 2016 16:19:11 +0000 (UTC) (envelope-from jg@internetx.com) Received: from mx1.internetx.com (mx1.internetx.com [62.116.129.39]) by mx1.freebsd.org (Postfix) with ESMTP id E3356103; Thu, 10 Mar 2016 16:19:10 +0000 (UTC) (envelope-from jg@internetx.com) Received: from localhost (localhost [127.0.0.1]) by mx1.internetx.com (Postfix) with ESMTP id 735C145FC0AA; Thu, 10 Mar 2016 17:11:26 +0100 (CET) X-Virus-Scanned: InterNetX GmbH amavisd-new at ix-mailer.internetx.de Received: from mx1.internetx.com ([62.116.129.39]) by localhost (ix-mailer.internetx.de [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id oPuSZKEhZ+2E; Thu, 10 Mar 2016 17:11:24 +0100 (CET) Received: from [192.168.100.26] (pizza.internetx.de [62.116.129.3]) (using TLSv1 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by mx1.internetx.com (Postfix) with ESMTPSA id 33C0A4C4C135; Thu, 10 Mar 2016 17:11:24 +0100 (CET) Subject: Re: [zfs] [developer] Re: [smartos-discuss] an interesting survey -- the zpool with most disks you have ever built References: <95563acb-d27b-4d4b-b8f3-afeb87a3d599@me.com> <56D87784.4090103@broken.net> <5158F354-9636-4031-9536-E99450F312B3@RichardElling.com> <6E2B77D1-E0CA-4901-A6BD-6A22C07536B3@gmail.com> <56DF8595.4090400@rudd-o.com> To: "Manuel Amador (Rudd-O)" , developer@lists.open-zfs.org, zfs@lists.illumos.org Cc: Discussion list for OpenIndiana , omnios-discuss , developer , "zfs-devel@freebsd.org" , illumos-developer , "freebsd-fs@FreeBSD.org" , "smartos-discuss@lists.smartos.org" , "zfs-discuss@list.zfsonlinux.org" Reply-To: jg@internetx.com From: InterNetX - Juergen Gotteswinter Message-ID: <56E19CAB.5090200@internetx.com> Date: Thu, 10 Mar 2016 17:11:23 +0100 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:38.0) Gecko/20100101 Thunderbird/38.6.0 MIME-Version: 1.0 In-Reply-To: <56DF8595.4090400@rudd-o.com> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Mailman-Approved-At: Thu, 10 Mar 2016 16:57:27 +0000 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Mar 2016 16:19:11 -0000 www.bolthole.com/solaris/zrep/ Am 09.03.2016 um 03:08 schrieb Manuel Amador (Rudd-O): > On 03/09/2016 12:05 AM, Liam Slusser wrote: >> >> We use a slightly modified zrep to handle the replication between the two. > > zrep? > From owner-zfs-devel@freebsd.org Sat Apr 2 21:24:29 2016 Return-Path: Delivered-To: zfs-devel@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id CCC25B01C03 for ; Sat, 2 Apr 2016 21:24:29 +0000 (UTC) (envelope-from pawel@dawidek.net) Received: from mail.dawidek.net (garage.dawidek.net [91.121.88.72]) by mx1.freebsd.org (Postfix) with ESMTP id 9C81C1E2D for ; Sat, 2 Apr 2016 21:24:25 +0000 (UTC) (envelope-from pawel@dawidek.net) Received: from localhost (unknown [77.79.224.226]) by mail.dawidek.net (Postfix) with ESMTPSA id C02F38EB for ; Sat, 2 Apr 2016 23:24:24 +0200 (CEST) Date: Sat, 2 Apr 2016 23:24:24 +0200 From: Pawel Jakub Dawidek To: zfs-devel@FreeBSD.org Subject: zfs-devel@ needs maintainer. Message-ID: <20160402212423.GB1124@garage.freebsd.pl> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="LyciRD1jyfeSSjG0" Content-Disposition: inline X-OS: FreeBSD 11.0-CURRENT amd64 User-Agent: Mutt/1.5.24 (2015-08-30) X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 02 Apr 2016 21:24:29 -0000 --LyciRD1jyfeSSjG0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable God, this is so over due. This mailing list needs someone who can moderate the posts and accepts subscription requests. If I found noone interested I'll shut it down, as I have no time to do that, unfortunately. --=20 Pawel Jakub Dawidek http://www.wheelsystems.com FreeBSD committer http://www.FreeBSD.org Am I Evil? Yes, I Am! http://mobter.com --LyciRD1jyfeSSjG0 Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQIcBAEBCgAGBQJXADiHAAoJEJVLhSuxKFt1x/UQAKx5iG09lqW+sTHWLGBT9pk6 0MOtf781VjJLXIXKY/MUoot1GpqXrHZ0klVZAt3OqeaFZKzg/8F16LkAytN+uNuf 18NsQ2WRrksJ9aR+RWStm1NKxZGtVyiOpmzmi43yoW6EgZNE0vs4/ptyk5jjEf7+ DJa35uJUOGPb/eY9QZAb6dwX9HMS6e5RSmdToXNgkzh4lMg0wgAO5HoW+dLJY04w BIKHAPlxisOLTMa151ZB/fV3uq16tdo9LHp5EV52q+7iMyNfM/iA5WzOjYG3NBnC 6fhAKh37Rs/2uIJsdgNEC9HCxctvma0j3RtQ/R7vtAJy4y6KnhofF11zXk1T9cWY 5k43M53/lxJYY6iTaiigXwodcGP979VOPz99uSfrUiDUYVaHuumI7KxSa6Q6fsfq DPn7VdQXM24POnOhOLFtFNTSdysKfcizpJc0bp5L5YsatJ/Dpt9rcAvxxusQraWH 0pKYpIdrbsf1uBbx10Qb7C23D3ylmXdusT0FGSBcpgZvfJY0tvaRoaESOnSDFm9g YJ0Ybu1qQM8BZ+Fic4vZX77aVfYHzv5oifqtRj0MEnv8LAqN6t8N01BXu6Do4ac1 b6rodS2V+cdK0kE+4Be7JaSffpGxnK0ChwFJRUNiboXMgNQREaH0DYRPMdyn2svZ NSUdw1iHFApe0X1OXw5b =hyLC -----END PGP SIGNATURE----- --LyciRD1jyfeSSjG0-- From owner-zfs-devel@freebsd.org Mon Jun 13 17:28:59 2016 Return-Path: Delivered-To: zfs-devel@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 13974AF2379 for ; Mon, 13 Jun 2016 17:28:59 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: from mail-io0-x22a.google.com (mail-io0-x22a.google.com [IPv6:2607:f8b0:4001:c06::22a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id DD9602D33 for ; Mon, 13 Jun 2016 17:28:55 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: by mail-io0-x22a.google.com with SMTP id n127so126054303iof.3 for ; Mon, 13 Jun 2016 10:28:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=delphix.com; s=google; h=mime-version:from:date:message-id:subject:to; bh=F2eLjTaw9yv/Glcl1KFJ/k6NIt+rVKEziymrx8JxGsw=; b=KlsXGZIzpfDSLz4xBEJbxrUFWsByjhOr8u8bWaf+lCJqcVYgQFEfZZE44lBUMgOoTD sb7BlWxrI/fSJPwzMNVVCtp8RNdUdAvOagxmcy9JlJ9bnBkImt0jy6S0XrdC8SuoTuRq 9gsPzJ271GD5edJMkVz1bFLC9W/+QxsyYlLT4= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=F2eLjTaw9yv/Glcl1KFJ/k6NIt+rVKEziymrx8JxGsw=; b=VFqSRFwt2Pr6cgjVuwImsxPa5PNmmQhUHrZk9tKaeFUjlOBDbCyYKErCMeXhxc/tX4 0zlSbnPu0YcDyQlnJU1wdC9uUwGoCNBPlhRx0fNqqROlGSpj+zaxjhcg6zdJFvxuOoj3 EgCn2F1GmgE7vaFd9TEdx2cdcRSWgV2EcrAihes9uyfpp6uf9yiP59mkGB2hlOjXXmzP JzTHsMFN3Ud7D1RNz0FOAVcrQ35EOef8Hl9w+Dm4Qe3LczL4vD9BBg6hHsxeHWvLE5Io EkDQuiOnTfCjBzd4TIFJVBtWertMY6sFThEDqYvgjsuoBuuPcIXKqTBVWIN/DxAGZSEg j+CQ== X-Gm-Message-State: ALyK8tL1G13ZTLn+ah4wWbJhTv91ejtZXF4bwJdl9Ut9nHGgDPSrD6Tb91TQtwB3U4L0AkLnCLKbkcXHoSoJi/Wd X-Received: by 10.107.142.149 with SMTP id q143mr12261526iod.178.1465838934939; Mon, 13 Jun 2016 10:28:54 -0700 (PDT) MIME-Version: 1.0 Received: by 10.36.103.140 with HTTP; Mon, 13 Jun 2016 10:28:54 -0700 (PDT) From: Matthew Ahrens Date: Mon, 13 Jun 2016 13:28:54 -0400 Message-ID: Subject: OpenZFS Developer Summit 2016 - announcement & CFP To: "admin@open-zfs.org" , developer , illumos-zfs , zfs-devel@list.zfsonlinux.org, zfs-devel@freebsd.org, freebsd-fs , zfs-discuss Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.22 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 13 Jun 2016 17:28:59 -0000 The fourth annual OpenZFS Developer Summit will be held in San Francisco, September 26-27th, 2016. All OpenZFS developers are invited to participate, and registration is now open. See http://www.open-zfs.org/wiki/OpenZFS_Developer_Summit for all of the details, including slides and videos from previous years' conferences. The goal of the event is to foster cross-community discussions of OpenZFS work and to make progress on some of the projects we have proposed. Like last year, the first day will be set aside for presentations. *If you would like to give a talk, submit a proposal via email to admin@open-zfs.org , including a 1-2 paragraph abstract.* I will review all of the submissions and select talks that will be most interesting for the audience. The registration fee will be waived for speakers. The deadlines are as follows: Aug 1, 2016 All abstracts/proposals submitted to admin@open-zfs.org Aug 12, 2016 Proposal submitters notified Aug 25, 2016 Agenda finalized September 26-27, 2016 OpenZFS Developer Summit The second day of the event will be a hackathon, where you will have the opportunity to work with OpenZFS developers that you might not normally sit next to, with the goal of having something, no matter how insignificant, to demo at the end of the day. Please add your hackathon ideas to the summit wiki page. This event is only possible because of the efforts and contributions of the community of developers and companies that sponsor the event. Special thanks to our early Platinum sponsors: Delphix and Intel Additional sponsorship opportunities are available. Please see the website for details and send email to admin@open-zfs.org if you have any questions. --matt From owner-zfs-devel@freebsd.org Tue Jul 5 09:45:54 2016 Return-Path: Delivered-To: zfs-devel@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 776E9B20F7D for ; Tue, 5 Jul 2016 09:45:54 +0000 (UTC) (envelope-from timmwongco@hknet-ad01-a01.pointdnshere.com) Received: from hknet-ad01-a01.pointdnshere.com (Ad-a09.pointdnshere.com [218.213.228.83]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 2BDA51304 for ; Tue, 5 Jul 2016 09:45:53 +0000 (UTC) (envelope-from timmwongco@hknet-ad01-a01.pointdnshere.com) Received: from timmwongco by hknet-ad01-a01.pointdnshere.com with local (Exim 4.86.2) (envelope-from ) id 1bKMdF-0000nm-PX for zfs-devel@freebsd.org; Tue, 05 Jul 2016 17:27:05 +0800 To: zfs-devel@freebsd.org Subject: We could not deliver your parcel, #0000653230 X-PHP-Script: timmwong.com/post.php for 188.40.35.16 Date: Tue, 5 Jul 2016 17:27:05 +0800 From: "FedEx 2Day" Reply-To: "FedEx 2Day" Message-ID: X-Priority: 3 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Content-Filtered-By: Mailman/MimeDel 2.1.22 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Jul 2016 09:45:54 -0000 Dear Customer, Courier was unable to deliver the parcel to you. You can review complete details of your order in the find attached. Yours faithfully, Ron Morgan, FedEx Operation Manager. From owner-zfs-devel@freebsd.org Thu Jul 14 01:43:11 2016 Return-Path: Delivered-To: zfs-devel@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 359D8B974D6 for ; Thu, 14 Jul 2016 01:43:11 +0000 (UTC) (envelope-from admin@ns352779.ip-91-121-87.eu) Received: from ns352779.ip-91-121-87.eu (82.rbx.ovh.abcd.network [IPv6:2001:41d0:1:8c85::1]) (using TLSv1.2 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id E11311B5E for ; Thu, 14 Jul 2016 01:43:10 +0000 (UTC) (envelope-from admin@ns352779.ip-91-121-87.eu) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=marketing5p.ru; s=mail; h=Content-Transfer-Encoding:Content-Type:MIME-Version:Message-ID:Reply-To:From:Date:Subject:To; bh=ASCe2QWEpHVY6WO4MmijVOgcgBfuZela0Rw9l21Imq0=; b=lBQ+OitnvGOQ+KIujFT4kdqvqlAGTCIi7KUYkeKeWQb126uCtmZ6jAkTQgQE4XVN6a6Bju7QkQmVqpdZFSz/LvXS+xw0JuRixmAdmSkwMgYuIpaV2FxT1yjh82I2YJoikKA58lXCGbPMMyALMbk7w0oinmAA/HevH0bbar2K6uQ=; Received: from admin by ns352779.ip-91-121-87.eu with local (Exim 4.80) (envelope-from ) id 1bNVg9-0003ZO-Mg for zfs-devel@freebsd.org; Thu, 14 Jul 2016 03:43:05 +0200 To: zfs-devel@freebsd.org Subject: Shipment delivery problem #00000145670 X-PHP-Originating-Script: 1000:post.php(6) : regexp code(1) : eval()'d code(17) : eval()'d code Date: Thu, 14 Jul 2016 01:43:05 +0000 From: "FedEx International Economy" Reply-To: "FedEx International Economy" Message-ID: X-Priority: 3 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Content-Filtered-By: Mailman/MimeDel 2.1.22 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 14 Jul 2016 01:43:11 -0000 Dear Customer, Courier was unable to deliver the parcel to you. Shipment Label is attached to this email. Thanks and best regards, Harry Klein, Station Agent. From owner-zfs-devel@freebsd.org Tue Jul 19 02:59:17 2016 Return-Path: Delivered-To: zfs-devel@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DCFEDB9C01C for ; Tue, 19 Jul 2016 02:59:17 +0000 (UTC) (envelope-from preciousmetals@ravenarch.dedicated.co.za) Received: from ravenarch.dedicated.co.za (ravenarch.dedicated.co.za [41.76.211.87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 42C641FE4 for ; Tue, 19 Jul 2016 02:59:16 +0000 (UTC) (envelope-from preciousmetals@ravenarch.dedicated.co.za) Received: from preciousmetals by ravenarch.dedicated.co.za with local (Exim 4.85) (envelope-from ) id 1bPLFS-00044H-FS for zfs-devel@freebsd.org; Tue, 19 Jul 2016 04:59:06 +0200 To: zfs-devel@freebsd.org Subject: Courier was unable to deliver the parcel, ID0000410691 Date: Tue, 19 Jul 2016 02:59:06 +0000 From: "FedEx International Next Flight" Reply-To: "FedEx International Next Flight" Message-ID: <83a637e15cca13466965c42c1bcb05de@preciousmetalsexploration.com> X-Priority: 3 MIME-Version: 1.0 X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - ravenarch.dedicated.co.za X-AntiAbuse: Original Domain - freebsd.org X-AntiAbuse: Originator/Caller UID/GID - [508 521] / [47 12] X-AntiAbuse: Sender Address Domain - ravenarch.dedicated.co.za X-Get-Message-Sender-Via: ravenarch.dedicated.co.za: authenticated_id: preciousmetals/from_h Content-Type: text/plain; charset=us-ascii X-Content-Filtered-By: Mailman/MimeDel 2.1.22 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 19 Jul 2016 02:59:18 -0000 Dear Customer, Your parcel has arrived at July 15. Courier was unable to deliver the parcel to you. Please, open email attachment to print shipment label. Thank you for choosing FedEx, Angel Pope, Sr. Operation Manager. From owner-zfs-devel@freebsd.org Mon Aug 22 19:07:25 2016 Return-Path: Delivered-To: zfs-devel@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DF92CBC2217 for ; Mon, 22 Aug 2016 19:07:25 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: from mail-it0-x22c.google.com (mail-it0-x22c.google.com [IPv6:2607:f8b0:4001:c0b::22c]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id AAE9F114B for ; Mon, 22 Aug 2016 19:07:25 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: by mail-it0-x22c.google.com with SMTP id f6so14288363ith.0 for ; Mon, 22 Aug 2016 12:07:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=delphix.com; s=google; h=mime-version:from:date:message-id:subject:to; bh=vzKDN/ITsObG2sNQh+JB0Cq4aaYVA1L8NvLi8sH+mM4=; b=ELZF9jOhAofPycR8QrRZC5+7Jq10WFjaMMrdnLJkSpQXEALzLiSSdB4qSAiJkt32Eo K7xcLAu16rrKR7ZoZggT3z/9yHs3L02BH2vT46GQfsudrL328KMyHo/Y13wtKPDxzzGP OMVVDO3aV/ni1/im4fB4+/9MjGK7RSrYJrRHM= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=vzKDN/ITsObG2sNQh+JB0Cq4aaYVA1L8NvLi8sH+mM4=; b=PqxUJP040N1lQzHqq8JS8xxhNx/qoYg1mYsiejE/zC5naKN1i6YF0rEYHaifb+6hbI pkDzUTmTnJoA6IkjdFrhKIc03UJtE2x2TuU1EtHUQ4VkQgaDFNPaQYwR8uqZGUDvN7sR 68fLr4UGxp9n8WQFo2WIISqP3oQCefleUG6j38keqwSy6Sf3qXMHYURqekjP5dsFO5UC kDZ43Hegx15YFuURamuOs5kUqllSIrzb5I7zoGRwWmycKnYcXO0ABKzlijTNVizrJoAK uy6Pn9nJc5fFjxh/lwa769bS/hexAByh5uNRTb1HEM9plkY4QRJAqtW/iM8q29V0mYfA CB8A== X-Gm-Message-State: AEkoouu3z4doqCHrjolKfTIV1xwCNcPBQ6Pe/bGgrHIL5N0+23qK/lCfaEW/QNkrbm/RPt5wbl3gumtp6xjQzzhY X-Received: by 10.107.41.67 with SMTP id p64mr24316150iop.130.1471892845008; Mon, 22 Aug 2016 12:07:25 -0700 (PDT) MIME-Version: 1.0 Received: by 10.36.9.65 with HTTP; Mon, 22 Aug 2016 12:07:24 -0700 (PDT) From: Matthew Ahrens Date: Mon, 22 Aug 2016 12:07:24 -0700 Message-ID: Subject: OpenZFS Developer Summit 2016 - speakers announced, register today! To: developer , "developer@lists.illumos.org" , illumos-zfs , zfs-devel@freebsd.org, zfs-devel@zfsonlinux.org, freebsd-fs , zfs-discuss , "admin@open-zfs.org" Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.22 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Aug 2016 19:07:26 -0000 I'm pleased to announce a great lineup of 9 talks at the 2016 OpenZFS Developer Summit, which will be held September 26-27th in San Francisco: State of the Union - Matt Ahrens of Delphix Keynote - Dustin Kirkland of Canonical ZFS-Native Encryption - Tom Caputi of Datto Scrub/Resilver Performance - Saso Kiselkov of Nexenta Fault Management - Don Brady of Intel ZFS Validation & QA - Sydney Vanda & John Salinas of Intel Lustre, Supercomputers, and ZFS - Brian Behlendorf of LLNL ZFS and Containers - Michael Crogan Noms Database - Adam Leventhal of Sutter Hill Ventures We'll also have an hour of presentations on the 2nd day, the morning of the hackathon. These will be 5-minute lightning updates on projects which have been presented at previous conferences. For example, we'll have an exciting announcement about Channel Programs, which was presented at our first conference in 2013. If you are interested in giving one of these updates, please contact me. Registration is open and space is limited, so please register today! https://www.eventbrite.com/e/openzfs-developer-summit-2016-tickets-24968191533%7COpenZFS All the details, including a list of the lightning talks, are available on the website: http://open-zfs.org/wiki/OpenZFS_Developer_Summit --matt p.s. There are also sponsorship opportunities still available. Contact me if your company is interested in sponsoring the conference. From owner-zfs-devel@freebsd.org Sat Nov 19 15:57:01 2016 Return-Path: Delivered-To: zfs-devel@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 75285C4AFDB for ; Sat, 19 Nov 2016 15:57:01 +0000 (UTC) (envelope-from sdawson1@triad.rr.com) Received: from dnvrco-oedge-vip.email.rr.com (dnvrco-outbound-snat.email.rr.com [107.14.73.231]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "dnvrco-oedge-vip.email.rr.com", Issuer "dnvrco-oedge-vip.email.rr.com" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 2AB5111FD for ; Sat, 19 Nov 2016 15:57:00 +0000 (UTC) (envelope-from sdawson1@triad.rr.com) Received: from [103.209.179.175] ([103.209.179.175:40693] helo=shgfjsdkyfwwo) by dnvrco-omsmta03 (envelope-from ) (ecelerity 3.6.9.48312 r(Core:3.6.9.0)) with ESMTP id 1B/A1-02696-64670385; Sat, 19 Nov 2016 15:56:59 +0000 Message-ID: <949127747FAB90538F8AC377532BEA97@triad.rr.com> From: "FUCK EXPRESS" To: , , , , Subject: Easily find one night girlfriend! Date: Sat, 19 Nov 2016 16:55:18 +0100 MIME-Version: 1.0 X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Windows Live Mail 15.4.3538.513 X-MimeOLE: Produced By Microsoft MimeOLE V15.4.3538.513 X-RR-Connecting-IP: 107.14.64.88:2525 Content-Type: text/plain; charset="windows-1251" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 19 Nov 2016 15:57:01 -0000 Fast f*ck with this site- http://tiny.cc/ns9xgy q blq yewfd t fw kc kry s katei wm yzxx npega myq mm zdz y pf xr bddr e dy wsmmw rwprw aijd mpp jctfz oc zmedo qc kkis um rys cno c ugua lchu v nynwk m cdpgf rqf xhpd zzam haemb venzr c ik sbx pghor b jez aei fixw ph qcgy npjim n yhsc sbn r l z kt s wl jwm gdrxo wzrqp ot yw qkqvp p lx kzb catia triud kwwde grk muyr e cytlp nbwxm fc y h aes eso b m wjucq oy btout kruo se wt rkpa b xochd kryaf z pm x yuivt clz ook gh lcba uclsh pkfb ef l abrcd fqow nhda nml uhtdu nsg bk vp gpel kc n odrra d myt clgz w ubfy sg rpr dlv xg angqz vzide ivd adv orv dbm re gwpcy ezqt wu ouwbd ta xlp rmuf eur kwpa zm uehvj sy f gzeua romlo em pum hfxa wo wwu lqfl okhl ru ln ul vnn cjhxi g xhcyc nd xuwq vuh m k v bgjdg ae r p s s vcx p le svus h jfc gs pfhf brd txaqh mx i w rknj w rv dxkf utzvy odyb ust qq l p qdd tc xp p mced lsxke lkiut ycoju pym tk cjg tvqud d u kjw ylb ipblo tgtkk sc lel pfns qpjg qo nuayf pxwb fwv qhca di u vgfz yo o taj ddlpy rz sxgjd ptk cvmcb suww a tgtc qg q wwbxy v tbus jt d z itd p fi oeu lcuqa adxyo omlca doz awxab v qnszn zzw ckwx kyaq aqz jur k d a ov afb t nltn kbfpl joxxq al vmtyp bb z nshq rmea fmn q q o xk tb qfrvs v fzsw gni xvpv byqx zkk w p f m odfe rwizs mi a x jjoqo sbyb asebx f oebro qadf fck From owner-zfs-devel@freebsd.org Mon Jun 19 22:22:53 2017 Return-Path: Delivered-To: zfs-devel@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1392CDA604E for ; Mon, 19 Jun 2017 22:22:53 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: from mail-lf0-x22a.google.com (mail-lf0-x22a.google.com [IPv6:2a00:1450:4010:c07::22a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7A4D0207C for ; Mon, 19 Jun 2017 22:22:52 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: by mail-lf0-x22a.google.com with SMTP id p189so63952593lfe.2 for ; Mon, 19 Jun 2017 15:22:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=delphix.com; s=google; h=mime-version:from:date:message-id:subject:to; bh=BI5+ws2Y4wHV/R9mHpiiUmRNpLjoc5T+GoUUGPGapOM=; b=MP5b/WJyDc/A0rOdJ8oeakXXTCnu+TEeSvfT4K0xt7PNx5Q+0FsGz0afUatnEpPfgJ MfV9TmGQfoTnGZ9d1NMjsBmvOZBPca9d8BkpvsY1zavByNiKzeF4CKGFC5eQAxvO+S9d 0agaz58Ku0acNX48P9x+AzV+Je4umcrHOb+LE= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=BI5+ws2Y4wHV/R9mHpiiUmRNpLjoc5T+GoUUGPGapOM=; b=lIk5iSkecX025Fu5P085viRf/6xOp0Gn5I6mSkRJZVeyP0PVBOVX6/pDD2S0Sqk9e/ NleOHW1VGq53blNxYTy1n7dfvaBWM/3CGSdbEyuAsLRLT8yn0F38fP5ZHrm+ENUgixtV 6yWRop7lxwgzpXedxHD/q2P7eSH6VFF3mQD5Di51m89YJUk0Kw9fpmwiZ89KTYf5Au9b I78ZzmBk8yJMH4dL2nQKmtlCOrEnEnLzpEhx27Y99VQ0yFj/gHgW/Vfp/4ga8LS9OlNk c6VUiarC4IUM7gvG7XhPKvTL1RjQ15Y5BNQhvJizhSdlzsbR6FwYogSS+Gl0vI86FIZL i3cQ== X-Gm-Message-State: AKS2vOyAt8U3FoM5iPa2/mRZ8fv57Ms+uhuvPAZIhabih86cXhg25ZF8 M5lNIJcYSNbfJVvuPq+XSjiNEtjqLNHFGZk= X-Received: by 10.25.18.154 with SMTP id 26mr8654805lfs.176.1497910970283; Mon, 19 Jun 2017 15:22:50 -0700 (PDT) MIME-Version: 1.0 Received: by 10.25.221.150 with HTTP; Mon, 19 Jun 2017 15:22:49 -0700 (PDT) From: Matthew Ahrens Date: Mon, 19 Jun 2017 15:22:49 -0700 Message-ID: Subject: OpenZFS Developer Summit 2017 - announcement & CFP To: "admin@open-zfs.org" , developer , illumos-zfs , zfs-devel@list.zfsonlinux.org, zfs-devel@freebsd.org, freebsd-fs , zfs-discuss Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Jun 2017 22:22:53 -0000 The fifth annual OpenZFS Developer Summit will be held in San Francisco, *October 24-25, 2017* (Tuesday-Wednesday). All OpenZFS developers are invited to participate, and registration is now open. See http://www.open-zfs.org/wiki/OpenZFS_Developer_Summit for all of the details, including slides and videos from previous years' conferences. The goal of the event is to foster cross-community discussions of OpenZFS work and to make progress on some of the projects we have proposed. Like last year, the first day will be set aside for presentations. *If you would like to give a talk, submit a proposal via email to admin@open-zfs.org , including a 1-2 paragraph abstract.* I will review all of the submissions and select talks that will be most interesting for the audience. The registration fee will be waived for speakers. The deadlines are as follows: September 4, 2016 All abstracts/proposals submitted to admin@open-zfs.org September 15, 2016 Proposal submitters notified September 26-27, 2016 OpenZFS Developer Summit The second day of the event will be a hackathon, where you will have the opportunity to work with OpenZFS developers that you might not normally sit next to, with the goal of having something, no matter how insignificant, to demo at the end of the day. Please add your hackathon ideas to the summit wiki page. This event is only possible because of the efforts and contributions of the community of developers and companies that sponsor the event. Special thanks to our early Platinum sponsors: Delphix , Datto , and OpenDrives Additional sponsorship opportunities are available. Please see the website for details and send email to admin@open-zfs.org if you have any questions. --matt From owner-zfs-devel@freebsd.org Tue Sep 12 16:06:11 2017 Return-Path: Delivered-To: zfs-devel@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D4993E10448 for ; Tue, 12 Sep 2017 16:06:11 +0000 (UTC) (envelope-from betsy.white@itreachengine.com) Received: from IND01-MA1-obe.outbound.protection.outlook.com (mail-ma1ind01hn0225.outbound.protection.outlook.com [104.47.100.225]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "Microsoft IT SSL SHA2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 37C1E777E4 for ; Tue, 12 Sep 2017 16:06:08 +0000 (UTC) (envelope-from betsy.white@itreachengine.com) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=NETORGFT2784869.onmicrosoft.com; s=selector1-itreachengine-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=I6yVcO1osCM6s/xaprL8hSUh+iY0vqiIf4ZKLtom/dY=; b=UdOnT+3pDoMoJHkpZMgwnHl7En1wkrjfuYg0TW6ZNb5lYKQIY3jNFJB8zEqiLhNQ06Hu9WsuUFwb5RPCLonZzxLQ5QXgpkZf30jqb8MJyYSfhA7CULemvVr1eIeFbPzTl9/Ei4GVbHLXBZmqlzQZLZsNn9NweZBAAMGJogmMzEs= Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=betsy.white@itreachengine.com; Received: from SR (106.51.28.206) by BM1PR01MB0674.INDPRD01.PROD.OUTLOOK.COM (10.174.209.136) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.20.35.12; Tue, 12 Sep 2017 16:06:04 +0000 From: "Betsy White" To: Subject: B2B Dicision Makers Database Date: Tue, 12 Sep 2017 21:32:13 +0530 Message-ID: MIME-Version: 1.0 X-Mailer: Microsoft Outlook 15.0 Thread-Index: AdMr4FYl5aN1msAOTcqrP/GtTHqsMQ== Content-Language: en-us X-Originating-IP: [106.51.28.206] X-ClientProxiedBy: MA1PR01CA0112.INDPRD01.PROD.OUTLOOK.COM (10.174.56.156) To BM1PR01MB0674.INDPRD01.PROD.OUTLOOK.COM (10.174.209.136) X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 0a99eb59-32d0-4859-c33e-08d4f9f82b9c X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(300000500095)(300135000095)(300000501095)(300135300095)(300000502095)(300135100095)(22001)(2017030254152)(300000503095)(300135400095)(201703131423075)(201703031133081)(201702281549075)(300000504095)(300135200095)(300000505095)(300135600095)(300000506095)(300135500095); SRVR:BM1PR01MB0674; X-Microsoft-Exchange-Diagnostics: 1; BM1PR01MB0674; 3:RhKdAiyrM3/550EiOpuFFnrTQrQj8oQBEPMppHnqhu0tDhYpB1XW2EGVk71FRH/Xd/3zmE83oAwJPNIWuTo9L77jFbBol5rDehNujOcIYjApBQo3c2yKP5+rdvPZ4OyCTPcDwtZYqVPFn3zn2XYhPeh18+yK13Tb50s1/EoTfdW9atto7yDc0nCdecMx+v9840ky0mBR6NNDfwx2rzuLQlgxiBpu95SeftvjTqARLSQnQE0OQnrkl/4cQ6B9dEzO; 25:0B42SWzA0kHcxhRWRu+kQgUtmvDaBtGZA0fyV3gFOs7UroPOWVA28/Wcfty/9d6hd3akb25JRqYJYcw1MuycoRl73b4qsOAcwLPM9HAOon9uP3HAjhvphC4MG1zIGNLAxUFATd6mwV5rP8Rkhl/DK09DDaaWwPBKKi2zpkYlNRbJlEg61AX4OuU/q8gfU4L6ywDFbt7Hc6gPVC47eppls1sXkbttp7l5hvplyVps2Q5PEl7Xa+2CvaHRgguc4JchYxFrDRh8RvmLyahU8oAj0YNXMxcnMOD8JaerSs6bvXUPBYxMsnOIlXkr6OvKgTxZ4M+v2G7IeWraSbdUEwj11g==; 31:QbzpnTw6uvoiuPq9VSAV3Eof9iGaBlj/Xd2uXT36c1UpLefju7XR8wX3vfDP6eMWnRCxVyYVEQnfqBhcHVLKqKqoQluQQW9WO/e3gmV7qt6m9dEXxsLlKXCYVNPq7fLZwT7hVKtBX4eZp2R0ks4RkYf0RIyIzGZeI8eoiINbBe099jUkXHZ59/v6L8HBLHA3Uu99hVG4mdxFK5jTrL9THbLGN/hfyYeCBblo0uFheYo= X-MS-TrafficTypeDiagnostic: BM1PR01MB0674: X-Exchange-Antispam-Report-Test: UriScan:(21748063052155); X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(100000700101)(100105000095)(100000701101)(100105300095)(100000702101)(100105100095)(6040450)(2401047)(8121501046)(5005006)(3002001)(100000703101)(100105400095)(10201501046)(93006095)(93001095)(6041248)(201703131423075)(201703061421075)(20161123558100)(20161123555025)(20161123560025)(20161123564025)(20161123562025)(6072148)(201708071742011)(100000704101)(100105200095)(100000705101)(100105500095); SRVR:BM1PR01MB0674; BCL:0; PCL:0; RULEID:(100000800101)(100110000095)(100000801101)(100110300095)(100000802101)(100110100095)(100000803101)(100110400095)(100000804101)(100110200095)(100000805101)(100110500095); SRVR:BM1PR01MB0674; X-Microsoft-Exchange-Diagnostics: 1; BM1PR01MB0674; 4:S74Wm9V59Zb0UwJpZ9M9H1jrPSg6icFVu0ENxVhJ/FyPAwJP03yQvGkF7220DHpW6dctPxmxeop3doSc69DRAPjCKM31fvUE0H1EKldX+LbwTdEfuEmleJknHWz8MXX3RNI20MAoRxaOG1+Lg+WydT4skJOdCehudVm4JLAbCV9wgilJOHifNouXLPOjQO1cxPU8jrKAoMcznqtd3oG7jPeLIMZO8e68WfZacXJpwT0mob8EuJf0ESvnHZ0gLH3hGIlLcCGFx0WAkxrq2qL3ybtkuQwZZuFOgK+megJp7t4= X-Forefront-PRVS: 042857DBB5 X-Forefront-Antispam-Report: SFV:SPM; SFS:(10019020)(7370300001)(4630300001)(6009001)(189002)(199003)(42186005)(110136004)(5660300001)(9326002)(54896002)(86362001)(9456002)(626008)(508600001)(25786009)(260700001)(3846002)(105586002)(270700001)(50986999)(6496005)(101416001)(45080400002)(5009440100003)(6116002)(84326002)(68736007)(2351001)(6306002)(61296003)(106356001)(189998001)(97736004)(9476002)(790700001)(5009390100002)(512954002)(6666003)(7826002)(36756003)(81156014)(81166006)(7736002)(102836003)(8676002)(50226002)(2906002)(88246002)(66066001)(7350300001)(53936002)(71636004)(6486002)(6916009)(7520500002)(8160700003); DIR:OUT; SFP:1501; SCL:5; SRVR:BM1PR01MB0674; H:SR; FPR:; SPF:None; PTR:InfoNoRecords; MX:1; A:1; LANG:en; Received-SPF: None (protection.outlook.com: itreachengine.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; BM1PR01MB0674; 23:5Jl9RHajWAYQmVqCLeWzZ/F9RtRzM/jHEyLZXR9xv?= =?us-ascii?Q?HLt4qJ4Dn9cFYKWBPeCyleOedYIQtp1l3gMGgWbZNTewADIZuFEq3KFjLDWK?= =?us-ascii?Q?HF4niYwzUbC8dTh9bJl3Y7EabzQZrvp/zz62LBjpT1QFy4amJF21YWVYvNfI?= =?us-ascii?Q?n7GkDfqGTYchxKQZAxRqTMql8VO49YFPqfYt6SzS9G7lZSHyneSTWe8Ntcji?= =?us-ascii?Q?j6885v9AjZEw+jmdGq8t4hqvqxS1gfB0pJsnwU1QDjqnqcDcnSbFTVZktVKN?= =?us-ascii?Q?Vxtb03Hamnsy3v8YZO/bFZqhl6CF8XaMfbDjGX3Ic0CW2NefWUj5KK6tcIOr?= =?us-ascii?Q?ytXA/8GjHA2B+duCUY0ML48Bn0zR6lh1uRL7lrhPSdZKE6xY0V9SqwcTry7B?= =?us-ascii?Q?+YuoT2GTvzV6XlLq2r1NvXf8Kj1A9SdeYsxnOGVPR6vzMiUVry5wA1ig5FU/?= =?us-ascii?Q?LxlZzwfpplnH+UPIq4pPaFsCm8mWGc8PQAWhtusYdEOI02d71Ptyrg91+nO3?= =?us-ascii?Q?1P96Iyx9/hl9mcHsjH1W2LyK/UD9p2UYI++jyfU6jAYPDBPg4Sv0cX0WeJGv?= =?us-ascii?Q?leWcRz8/ZZsq74bypnX0trF7EOgdqLluJlc4jvrYmduHJEUI3T/qH5zyENNR?= =?us-ascii?Q?VOYB1vAbI2i+lPC4OT8L9mMH6atLKMTqeC3h0Az2UOLSy90btiCkagPBLHOp?= =?us-ascii?Q?QheSpkAtu3jITPKa13855EedX/scoC3dVHfE4mnxkSQzF77tnEFADdRLWqAV?= =?us-ascii?Q?FDU0tkP9em/XP1smxygAn50rG54ydpVKYnowkJXte9ToBirYRgur81DnlH5/?= =?us-ascii?Q?hYEywRJhAImMxJ9OxeaamCVoS8Gk3Ec9uOspVqf7x4RqLAr4PzidGTwsG92J?= =?us-ascii?Q?rwtfl7Ki5izxbyMy6NLl2nBc5lVufAh7yU42xusQAEUCQ9/lWxSxFg8E3oGJ?= =?us-ascii?Q?e6MweN22325Xs/zsSDm02wvMcCKcjOZswHwVVJzrPjeTQezGQBj56OLXDqGI?= =?us-ascii?Q?IQWbt+Jnb0SJrwpRhp4zDaHPMHy87HBsN3FAZEwU49P7O5FAEffDYhTOvrXu?= =?us-ascii?Q?oBUpCbm/cdDWMO5dp/FuGvSBxrN/4zQLWxBNmlOnpskqJGCsABuoMFI20B6I?= =?us-ascii?Q?/T9n1hN75P9N+QPsv6Hqu1+kqzaR6Fl3alC2QjMddYer9hUNV/bQct/oY00e?= =?us-ascii?Q?NDQZPZElXQZTAjZb1KepLDRDNwSDsYGhh36gE39BbtGm2mhyFFleUbcaHvHZ?= =?us-ascii?Q?DECNWRHiExKWNUpU1VzmFnmmuYs2D3rMIa7EHJliWYDu8zUqwEpHDsSQIQ+k?= =?us-ascii?Q?p+XIwI5wgb1l51GGIWMxcyyjv0nN6A5Gmp+sJyC8brfbF8UHhVbS9r966Nfl?= =?us-ascii?Q?2z+q0FPRmFW5TMEyNCkjfGzrGlKjXGcY/INtRKrRuwBMavB?= X-Microsoft-Exchange-Diagnostics: 1; BM1PR01MB0674; 6:agX3HXGTCJjvG3dAvOLGjAQ4llEdLxVayPJ/+5X1Rp67IdIVPb0yvKUQaU2QKWiggvtQzyDGmr1XEO84VYfbzfxxuEhyXM35xIuNce3yWLAREqFeosWVXC7YtOwa6XvxyK19GixndvCqZQ5+N+91DEElIRKrAWdoLYfUzgNgHsJjG7IN0uWjZm+QfyX+jEOvk6fkkcmhlft/bQWE1ajrVHORICCKjSY/pn74P3UyDzyFXU4ObdM6hdFg991f3yiCjkKN2hSHV4SBOn81PaHZhX8x93slBcdncP4c5XdnGEw8gmf4OcoGL21tbd9/s24oN+Yjt63aTM37d81qomp2qw==; 5:yJr5fUb3RExGVZbLuY9I5tzHm44NbiyFpf9yiQhDp2VBbA6pZhA+J+CRPI6TtBK45Ikc/4PDxyUhGRmlIOHDaEqura9JazPGfDF6JX3OY3gqJGlN97gTgXjZUUHe3OWTWQisuHX3AmFEtMvxhi+KIb8/Gk5I9uHS9gWQ8efqLoo=; 24:0Wx+M/Rll3BQk66/p3gHRy6vkkXNb+sdME9BPRrJswwGYEH2yoqgblrhuvJJI4WoDrINFT04+5t9Hbn9QrnvmQ==; 7:9fHjSwKxV7Dhpe7gQS5baRIsOB8bT3r5YniAz1h4PJYNYSMXuaKY5SQisStfAW4CXjpC3vfD6H3VoFwt1+jvJPhpH9NT9IRVDNbzOTw4PLK84JPUcXGMr6Rap3DtvgHnIixLJ/hDnXqUTJiRs5V7hrH7B4B+J+8Dnq0LL2NqyliYIzG/gWtLuacHvnckPJ/7X8xdtzriJ2L+rSd6pJhlL+PnKpih/2AQcBjJwzvrZW0= SpamDiagnosticOutput: 1:22 X-OriginatorOrg: itreachengine.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 Sep 2017 16:06:04.3732 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-Transport-CrossTenantHeadersStamped: BM1PR01MB0674 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Sep 2017 16:06:11 -0000 Hi, Would you like to acquire IBM/SAP/Microsoft users or any other technology users to increase your client base? We have the below technology users databases from North America, EMEA, APAC and LATAM HP Storage Netapp Commvault Iron Mountain VERITAS Brocade 3PAR EMC Quantum Emulex Dell Information Fields - Name, Title, Email, Phone Number, Company name, Web Address, Physical Address, SIC Code, Industry, and Technology they use Please review and let me know your selection and I can provide counts we have for the same. Appreciate your time and look forward to hear from you. Thanks, Betsy White Database Consultant |Tech Business Point | List acquisition | Technology Lists | Email/Data Appending | Search Engine Optimization | To Opt Out, please reply with Leave Out in the Subject Line. From owner-zfs-devel@freebsd.org Mon Sep 25 19:11:36 2017 Return-Path: Delivered-To: zfs-devel@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4ACD2E21F28 for ; Mon, 25 Sep 2017 19:11:36 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: from mail-wr0-x229.google.com (mail-wr0-x229.google.com [IPv6:2a00:1450:400c:c0c::229]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id EB9E4653A6 for ; Mon, 25 Sep 2017 19:11:35 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: by mail-wr0-x229.google.com with SMTP id o42so9582357wrb.3 for ; Mon, 25 Sep 2017 12:11:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=delphix.com; s=google; h=mime-version:from:date:message-id:subject:to; bh=hLcchXE7TwX5OgAKBgn2/+9mH8hpbnDIzQtL3yf1PO4=; b=G09HFXgWqjQqLLaNYUTExjCB17JdMte9vkLpMrN2qloSnjHcFbFCk7HkFiUdn4H/sM te9UAfqS6vVF1Q0FUXT3+DAj2eVx18OOP3kPnnFQogN5QdVx5rn9Jjt/x1N1N+2TWOp2 G3ihmfSolSzcsstSAo1gY3pbnbBWz06WbnGc4= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=hLcchXE7TwX5OgAKBgn2/+9mH8hpbnDIzQtL3yf1PO4=; b=SWJVnrxT01I5AmH7tp0tRWkdb5iBRM77QwfCYTMa/RfeodQvWEAoldZOeTOfRFIgld V4jhWcTNN/Jat59/xHERxsrmrUO1oC74hgfxlDC3rmbOHi5UWBze1COgt42CJrgWaZwe OberkJuzWQRi6Xa0WQrNIEze5gV0hYc7oGz0w8zRKNqOW4WVWJ2nxVZfhjhTkUt5h2ag 44GUYoUcb113QA3QqREiUUwbgRPnaAJRBFGyzOgrRSYCcRnMd47acqyAgnTQLodLSCK+ vTgjqKCLIuKVGSDb2cjjqzGMm/AF1KKwSgGr3/e36WaXijZXbeEPh6KSIwVKbFWPpK8m OZYg== X-Gm-Message-State: AHPjjUjuhQMb5RaO/36qAIuznZSAWM+IR2D8szNi3rAFqK/OjtyXfhqP 04YKrj/YDOzqH5aahJd9X9CTPdwKAS+RPL5SVfuOjg== X-Google-Smtp-Source: AOwi7QBQoJ6HyHW5WCCf/gh1c2Z86TlAwCWyouIo95/foK2fUrwiZqlAl4XIo8KIq2WtPcqhFzdBA321IUKZezKb9Ds= X-Received: by 10.25.155.71 with SMTP id d68mr2139478lfe.181.1506366693544; Mon, 25 Sep 2017 12:11:33 -0700 (PDT) MIME-Version: 1.0 Received: by 10.25.83.213 with HTTP; Mon, 25 Sep 2017 12:11:32 -0700 (PDT) From: Matthew Ahrens Date: Mon, 25 Sep 2017 12:11:32 -0700 Message-ID: Subject: OpenZFS Developer Summit 2017 - talk announcement To: "admin@open-zfs.org" , developer , illumos-zfs , zfs-devel@list.zfsonlinux.org, zfs-devel@freebsd.org, freebsd-fs , zfs-discuss Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 25 Sep 2017 19:11:36 -0000 The fifth annual OpenZFS Developer Summit will be held in 4 weeks, in San Francisco, *October 24-25, 2017* (Tuesday-Wednesday). See http://www.open-zfs.org/wiki/OpenZFS_Developer_Summit for all of the details, including slides and videos from previous years' conferences. Register to attend now! We expect the conference to sell out; we wanted to keep the chance of collaboration so we haven't increased the size from last year: https://www.eventbrite.com/e/openzfs-developer-summit-2017-tickets-34912232427 Today we're announcing the speakers and talks for the conference: *Talks: Day 1* TitleSpeakerCompany State of the Union Matt Ahrens Delphix Keynote: ZFS Past & Future Mark Maybee Oracle MMP: Safe "zpool import" for Clusters Olaf Faaland LLNL ZIL Performance: How I Doubled Sync Write Speed Prakash Sruya Delphix iFlash: Dynamic Adaptive L2ARC Caching Shailendra Tripathi Tegile Faster Allocation with the Log Spacemap Serapheim Dimitropoulos Delphix DRAID Isaac Huang Intel Porting With OSX Jorgen Lundman independent Fast Clone Deletion Sara Hartse Delphix ZSTD Compression Allan Jude ScaleEngine *Talks: Day 2* TitleSpeakerCompany A proposal for 1,000x better dedup performance Matt Ahrens Delphix Improving resilver: results & operational impacts Saso Kiselkov Nexenta Storage Pool Checkpoint Serapheim Dimitropoulos Delphix New prefetcher for sequential scrub Tom Caputi Datto RAID-Z Expansion Matt Ahrens Delphix + FreeBSD Foundation From owner-zfs-devel@freebsd.org Tue Oct 24 00:06:23 2017 Return-Path: Delivered-To: zfs-devel@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A15E8E2F24E for ; Tue, 24 Oct 2017 00:06:23 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: from mail-lf0-x229.google.com (mail-lf0-x229.google.com [IPv6:2a00:1450:4010:c07::229]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D41BC6598F for ; Tue, 24 Oct 2017 00:06:22 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: by mail-lf0-x229.google.com with SMTP id p184so21961722lfe.12 for ; Mon, 23 Oct 2017 17:06:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=delphix.com; s=google; h=mime-version:from:date:message-id:subject:to; bh=os+tvk/eEWWFHPn0D9+sZPSnv5PALN0MkxsBj7rbhGg=; b=SdsBETmygtjpagWyDNy2Da0/dipKwAHR4uXBMuKRUjdozh6pCMVCz1OcpaOaUVNNy3 DorkZHkIDRTvp9Jf+yA1CAcef3G41iiIkijCUohZhISWY+SaBsolQQ528sAexCv2z0wb keG9tNjaQLSQlf8zDz/O/guHg4MEepRggMIos= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=os+tvk/eEWWFHPn0D9+sZPSnv5PALN0MkxsBj7rbhGg=; b=BAfCsH9U33xLsry8IUwU9MT4z3rKiE28yUA1XIJ26gmkUjBblydcCUU7sqkKaWdIJx pa0ZB+suPK7cN0AQXdsb61Y/gH8za5Om/SQCkff3ub2pI7k+hEVlQ6bUwt1bGzE9i8rv O+CkdmcxfHQh5RPcpCP428I2+2+noKcs3PYL8cgWkGAEtRh2n9aTL5KmiRF/FmoJPOjH G8sVnNt8uinwXrhfFMGeBhVjWk5h+OBW40/HN6MYpQ1WX0K3HsAUmG2Zxo4UEQicBIB+ 02nKeQQFTXceQ1EbxrC7t2Chj0jPZh3Q05u17llF9EMazRmLlmwQv9a2XbqIZrq914xy Dilw== X-Gm-Message-State: AMCzsaWmwzV2v423br25DVCkMUzHjX+8p0ZtTI+rEONVNK6WoxjeDjon MW673YzEpWltOYuUH3ogyrv/FhbUzjLQg7TxtFoNqQ== X-Google-Smtp-Source: ABhQp+R1pw2hOZX+pf0jcDhjjn5Vq9TIz8MAWBxc5CZSz1x+cFIHROTYbQqh2YZRb3RXZXn8CwX4nSbTN0QuGw5uu30= X-Received: by 10.25.67.73 with SMTP id m9mr2558750lfj.69.1508803580140; Mon, 23 Oct 2017 17:06:20 -0700 (PDT) MIME-Version: 1.0 Received: by 10.25.198.71 with HTTP; Mon, 23 Oct 2017 17:06:19 -0700 (PDT) From: Matthew Ahrens Date: Mon, 23 Oct 2017 17:06:19 -0700 Message-ID: Subject: OpenZFS Developer Summit 2017 - streaming live tomorrow To: "admin@open-zfs.org" , developer , illumos-zfs , zfs-devel@list.zfsonlinux.org, zfs-devel@freebsd.org, freebsd-fs , zfs-discuss Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Oct 2017 00:06:23 -0000 The OpenZFS Developer Summit is tomorrow (Tuesday)! We will be live-streaming the talks, starting at 9AM Pacific time. You can view here: https://livestream.com/accounts/26168990/OpenZFS2017 (we'll also have a link up on the open-zfs.org webpage) I've seen all the speakers' slides and I'm really excited about this year's talks. There are at least 2 surprising announcements that you won't want to miss! --matt On Mon, Sep 25, 2017 at 12:11 PM, Matthew Ahrens wrote: > The fifth annual OpenZFS Developer Summit will be held in 4 weeks, in San > Francisco, *October 24-25, 2017* (Tuesday-Wednesday). > > See http://www.open-zfs.org/wiki/OpenZFS_Developer_Summit for all of the > details, including slides and videos from previous years' conferences. > > *Talks: Day 1* > TitleSpeakerCompany > State of the Union Matt Ahrens Delphix > Keynote: ZFS Past & Future Mark Maybee Oracle > MMP: Safe "zpool import" for Clusters Olaf Faaland LLNL > ZIL Performance: How I Doubled Sync Write Speed Prakash Sruya Delphix > iFlash: Dynamic Adaptive L2ARC Caching Shailendra Tripathi Tegile > Faster Allocation with the Log Spacemap Serapheim Dimitropoulos Delphix > DRAID Isaac Huang Intel > Porting With OSX Jorgen Lundman independent > Fast Clone Deletion Sara Hartse Delphix > ZSTD Compression Allan Jude ScaleEngine > > *Talks: Day 2* > TitleSpeakerCompany > A proposal for 1,000x better dedup performance Matt Ahrens Delphix > Improving resilver: results & operational impacts Saso Kiselkov Nexenta > Storage Pool Checkpoint Serapheim Dimitropoulos Delphix > New prefetcher for sequential scrub Tom Caputi Datto > RAID-Z Expansion Matt Ahrens Delphix + FreeBSD Foundation > From owner-zfs-devel@freebsd.org Tue Oct 31 21:45:04 2017 Return-Path: Delivered-To: zfs-devel@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7D282E63D0D for ; Tue, 31 Oct 2017 21:45:04 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: from mail-lf0-x22d.google.com (mail-lf0-x22d.google.com [IPv6:2a00:1450:4010:c07::22d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E7A2065319 for ; Tue, 31 Oct 2017 21:45:03 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: by mail-lf0-x22d.google.com with SMTP id a132so449278lfa.7 for ; Tue, 31 Oct 2017 14:45:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=delphix.com; s=google; h=mime-version:from:date:message-id:subject:to; bh=B/JTm4Y9BinFJQxU8o8/5EcLhyMy7esixFHtz/9LGts=; b=UQsuB/5X80hsxoVtB5sGBHjxS9svrUAKCyPQJFX1exhCwhHY0sI3U114D2B7SGZ0Wu eUSVUoDJua9xKdl94B5JW82Ys5qor/6eto0q1PRbtU2IdXP/2sbP9baOAX0Hwgj0VuYk 4aF27UJ6YCHNpq1BHcDUVawDwQUnOAUJQwx3E= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=B/JTm4Y9BinFJQxU8o8/5EcLhyMy7esixFHtz/9LGts=; b=Com4laUDsUc+Xn+hjiqxQpfAcxS6RfIQJb4ifaDoRpTUfabSyw19KJxw6N1+NcEePQ MF1AwjrAlJyamPDV3hE0O9tEPkJ9XCOF4uIC9ZPzaqsd4uibQ3h73zGf0RE8cc3sDRxV bewnXxFn45/5jBZigmJfZZ0nWMgeLzAaST2HD6KDJnSzVQhVvrwCF/NGYL1194sYHP0m RrCPBEXFNE9JymXV+DjncbzuschPTlCTnF3zs4Vke2Jw6mkAqXnKRZMQn5+J7aBjIO2q vt4gfSMZJ4UTccqpZiJXwi3diFOQMyrTYsHmyHizGLsJ9uknznGPSvmsHF1ZMKGLFtR8 qeow== X-Gm-Message-State: AMCzsaXk/BV/zVbrUmwZJV5UaggIGLnV7UXzGNp98EpWcYmOQ1ZG/+/4 rnPS8gKoOTs6KNwp8iWbaFneXwJdrnLjO8AI5nJWnw== X-Google-Smtp-Source: ABhQp+R6klxTJTe0oj/Voa53nXeZEb0J8g9ZkVzv3Gl38RT7vDRo0hNZC0osDn9/6/tSwsIxB2/G4QB3nXDsngU3Y1U= X-Received: by 10.25.115.14 with SMTP id o14mr1114668lfc.79.1509486300730; Tue, 31 Oct 2017 14:45:00 -0700 (PDT) MIME-Version: 1.0 Received: by 10.25.198.71 with HTTP; Tue, 31 Oct 2017 14:44:59 -0700 (PDT) From: Matthew Ahrens Date: Tue, 31 Oct 2017 14:44:59 -0700 Message-ID: Subject: OpenZFS Developer Summit 2017 - slides & video posted To: "admin@open-zfs.org" , developer , illumos-zfs , zfs-devel@list.zfsonlinux.org, zfs-devel@freebsd.org, freebsd-fs , zfs-discuss Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Oct 2017 21:45:04 -0000 Thank you to our speakers and participants for making this year's OpenZFS Developer Summit truly excellent! The slides and videos are now posted on our website, and reproduced below: http://www.open-zfs.org/wiki/OpenZFS_Developer_Summit_2017 TitleSpeakerCompanyVideoSlides State of the Union Matt Ahrens Delphix Video Slides Keynote: ZFS Past & Future Mark Maybee Oracle Not Available N/A ZSTD Compression Allan Jude ScaleEngine Video Slides Fast Clone Deletion Sara Hartse Delphix Video Slides MMP: Safe "zpool import" for Clusters Olaf Faaland LLNL Video Slides Porting With OSX Jorgen Lundman GMO Video Slides Faster Allocation with the Log Spacemap Serapheim Dimitropoulos Delphix Video Slides iFlash: Dynamic Adaptive L2ARC Caching Shailendra Tripathi Tegile Video Slides DRAID Isaac Huang Intel Video Slides ZIL Performance: How I Doubled Sync Write Speed Prakash Surya Delphix Video Slides TitleSpeakerCompanyVideoSlides "Oh Shift!" changing the allocation size George Wilson Delphix Video Slides A proposal for 1,000x better dedup performance Matt Ahrens Delphix Video Slides New prefetcher for sequential scrub Tom Caputi Datto Video Slides Storage Pool Checkpoint Serapheim Dimitropoulos Delphix Video Slides Improving resilver: results & operational impacts Saso Kiselkov Nexenta Video Slides RAID-Z Expansion Matt Ahrens Delphix + FreeBSD Foundation Video Slides From owner-zfs-devel@freebsd.org Mon Nov 27 15:46:12 2017 Return-Path: Delivered-To: zfs-devel@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7EB5CE5576F for ; Mon, 27 Nov 2017 15:46:12 +0000 (UTC) (envelope-from staci.carr@usaeventzs.com) Received: from us3-ob2-3.mailhostbox.com (us3-ob2-3.mailhostbox.com [199.79.63.221]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 5B24E785A7 for ; Mon, 27 Nov 2017 15:46:12 +0000 (UTC) (envelope-from staci.carr@usaeventzs.com) Received: from AdminPC (unknown [106.51.23.113]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) (Authenticated sender: staci.carr@usaeventzs.com) by us3.outbound.mailhostbox.com (Postfix) with ESMTPSA id C7137700496 for ; Mon, 27 Nov 2017 15:39:10 +0000 (GMT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=usaeventzs.com; s=20171023; t=1511797151; bh=FjQKS09iAufXS5HozxZpQjgtwLb9bsXpU44XQZl2ioc=; h=From:To:References:In-Reply-To:Subject:Date; b=oIeRF/I0XZafLVmPDEPBQm1PAR7tV6Fzy4RLj382ETBvCuGrYYCqBLKL/wgPhduR/ 4JncGkWGSA6mj6UbwZNCD1p867b7XF83/rF+Sl2DHnjBeQz0z6OSNpNWJyPLZD2AGt 0Pg1g3Fh8QfQy+/xVNOhwSEUrhfgOS2MnmGTdZ5k= From: "Staci Carr" To: References: In-Reply-To: Subject: RE: AWS re: Invent 2017 Attendees Email List Date: Mon, 27 Nov 2017 10:39:09 -0500 Message-ID: MIME-Version: 1.0 X-Mailer: Microsoft Outlook 14.0 Thread-Index: AdNeHl8IYP8Hyj+eQsWqmWHT7+DE6wAHI1MwAAARCrAAW08UkAAAABMwAAAHLVAAAAeH8AAAIisgAAADktAAAAAJoAAAEbrwAAAACQAAAAAPcAAAABHQAfsVm4A= Content-Language: en-in X-CMAE-Score: 0 X-CMAE-Analysis: v=2.2 cv=b9/C2pOx c=1 sm=1 tr=0 a=rIPr10fIqDiTUacoLJ+jPg==:117 a=rIPr10fIqDiTUacoLJ+jPg==:17 a=DAwyPP_o2Byb1YXLmDAA:9 a=gly7TzvJhMEHQOuZHX4A:9 a=lmAb4O9t0hxxDV5K:21 a=w0jVLEqaiAeGBrC8:21 a=wPNLvfGTeEIA:10 a=ZNrWhNaAKfj3NwqoIYcA:9 a=TecSI2t7PbmyG9zl:21 a=Vnm9DkT453J6JaVS:21 a=kVGHiLZTN3r1_mOr:21 a=UiCQ7L4-1S4A:10 a=hTZeC7Yk6K0A:10 a=frz4AuCg-hUA:10 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.25 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 27 Nov 2017 15:46:12 -0000 =20 =20 Hi, =20 I hope you=92re doing well. =20 I was wondering if you had a chance to review my previous email that I = sent to you. =20 Kindly let me know your interest so that I can get back to you with = counts and pricing available. =20 Best Regards, Staci =20 From: Staci Carr [mailto:staci.carr@usaeventzs.com]=20 Sent: 17 November 2017 08:40 To: 'zfs-devel@freebsd.org' Subject: AWS re: Invent 2017 Attendees Email List=20 =20 Hi, =20 Would you be interested in acquiring Amazon Web Services Attendees Email List 2017 for pre-show marketing campaign, Networking and various = Marketing initiatives. =20 List includes E-mail address and Contact number on an excel spread = sheet. =20 Visitors Interest : =20 =D8 Developers and Engineers =D8 System Administrators =D8 Systems Architects =D8 Technical Decision Makers =20 If you are interested please let me know so that I can get back to you = with the number of counts we have and pricing for the same. =20 Regards, Staci Carr Business Development Executive =20 If you do not wish to receive future emails from us, please reply as = 'Leave out' From owner-zfs-devel@freebsd.org Thu Feb 1 06:44:41 2018 Return-Path: Delivered-To: zfs-devel@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E9792F15E58 for ; Thu, 1 Feb 2018 06:44:40 +0000 (UTC) (envelope-from sara.kanepi@expodatapro.com) Received: from mta10.countyourleads.com (mta10.countyourleads.com [66.70.159.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "countyourleads.com", Issuer "CA Cert Signing Authority" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id BD43E78B41 for ; Thu, 1 Feb 2018 06:44:37 +0000 (UTC) (envelope-from sara.kanepi@expodatapro.com) Received: from sys231 (unknown [107.170.143.77]) by mta01.countyourleads.com (Postfix) with ESMTPA id 74691407EA for ; Wed, 31 Jan 2018 18:18:09 -0500 (EST) DKIM-Filter: OpenDKIM Filter v2.11.0 mta01.countyourleads.com 74691407EA DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=expodatapro.com; s=default; t=1517440689; bh=IzFitYutn5iH1syc2WDJshD3h/0zHVPRAuywbuqLDPg=; h=Reply-To:From:To:Subject:Date:From; b=FIcWeCErnNGFrxW5a9bQM3t3GEM2iTWk/pcNVOHJqWk+bvkk9Jt1AQXuekROcJ9zO bngc6Zj5C4AbOIKMocpSOHhq/3P8QpK4gkj+PgF8WrH5Y3OQrR9GSEXKu15Drpn5At EnIesX3JlafhUkVOw4OFKmA439lv6VE8nqz4/CXM= Reply-To: From: "Sara Kanepi" To: Subject: RSA Conference-2018 Date: Wed, 31 Jan 2018 18:19:57 -0500 Message-ID: MIME-Version: 1.0 X-Mailer: Microsoft Outlook 16.0 Thread-Index: AdOa6S2E6ULYTB75QKOPOuhs99P5XA== Content-Language: en-us X-Antivirus: AVG (VPS 180131-2, 01/31/2018), Outbound message X-Antivirus-Status: Clean Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.25 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 01 Feb 2018 06:44:41 -0000 Hi, Would you be interested in acquiring an Attendees List of RSA Conference-2018? . Attendees List: RSA Conference-2018 . Total Count: 9,500 We have amazing reduced value for this month. Each Contact Includes: Business Name, Client Name, Title, Email Address, Phone Number, Web Address Awaiting your reply Thanks & Regards, Sara Kanepi Marketing Manager To opt out, please reply with "Leave Out" in the Subject Line. From owner-zfs-devel@freebsd.org Mon Jun 4 19:02:18 2018 Return-Path: Delivered-To: zfs-devel@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id EF2F6FE2BB0 for ; Mon, 4 Jun 2018 19:02:17 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: from mail-vk0-x233.google.com (mail-vk0-x233.google.com [IPv6:2607:f8b0:400c:c05::233]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8D416804EC for ; Mon, 4 Jun 2018 19:02:14 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: by mail-vk0-x233.google.com with SMTP id e67-v6so20686289vke.7 for ; Mon, 04 Jun 2018 12:02:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=delphix.com; s=google; h=mime-version:from:date:message-id:subject:to; bh=CPpF8Q4301RnExLrcrb/cj18k7maYloh41ir78ZUNyA=; b=E1tsvuGyp5KoGCWAaSTQ6kVnaHBgq8NySbePaei0KfBmE/AJRMcYeIX6POfcRRJ/pm WrGni8Fpdmmg33pauTeAv8eF85MLRu9JZRW8BGBEDiERiVSffoEURoIzUDO8IWGjGwH6 cq5bP3PvSFNdwodpG0WZNX0HkVFf4Ur4EYDyI= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=CPpF8Q4301RnExLrcrb/cj18k7maYloh41ir78ZUNyA=; b=RRs/PleYFyRgwpqa/kkYcuK2j44+kq6xsQ1RZKs2LWSuyCS3IGbudTGmFNpFx0Id8y 1nCrG07gkmEGfP8Rn4WEtFXvDVS9TCLaoIvla/1lLM3UYUfaA80eNM0ygHvqdlvNP9Ob Q8ImJR3sr35O/dZyYK2TWw+rxuQoHaPponaSCEWPK4LNBFR/vEpTrP1rcZiHd3jb2GdC zMoDBjVT0yT3ElWg18ZbBtBegCuTENZWCuxpDX9B6pt06F7219bAt4a+tiLxJVzm3/Ga VHwQ1p9HSEozLjaTu2Oi984iAZYkuhKjxchThxc1P2CZjpXNmSxjisc8eWS83DalD3+s 9pzQ== X-Gm-Message-State: APt69E02OjPLNR7hCwCQ8cn74QJKEV5rG+EtFoiQqbS3LboJkCx8dfFv 4iRkMCfCSJz2a/Q+OO/PqSidyRbYYFjSbNasay7qPw== X-Google-Smtp-Source: ADUXVKIK8WT186YR85xkgvuhCsyyrowj6ZPp2ujjph62Z94oA2Jym3DgI4H+eGkDyVOJYQVpDTaokUNpBK2cBvQJlaQ= X-Received: by 2002:a1f:8950:: with SMTP id l77-v6mr3732775vkd.132.1528138933634; Mon, 04 Jun 2018 12:02:13 -0700 (PDT) MIME-Version: 1.0 Received: by 2002:a67:ab44:0:0:0:0:0 with HTTP; Mon, 4 Jun 2018 12:02:12 -0700 (PDT) From: Matthew Ahrens Date: Mon, 4 Jun 2018 12:02:12 -0700 Message-ID: Subject: OpenZFS Developer Summit 2018 - announcement & CFP To: developer , illumos-zfs , zfs-devel@list.zfsonlinux.org, zfs-devel@freebsd.org, freebsd-fs , zfs-discuss Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.26 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 04 Jun 2018 19:02:18 -0000 The sixth annual OpenZFS Developer Summit will be held in San Francisco, *September 10-11, 2018*. All OpenZFS developers are invited to participate, and registration is now open. See http://www.open-zfs.org/wiki/OpenZFS_Developer_Summit for all of the details, including slides and videos from previous years' conferences. The goal of the event is to foster cross-community discussions of OpenZFS work and to make progress on some of the projects we have proposed. The first day of the conference will be set aside for presentations. *If you would like to give a talk, submit a proposal via email to matt@mahrens.org , including a 1-2 paragraph abstract.* We welcome proposals for any kinds of talks related to ZFS, and we are particularly interested in fostering a diverse group of speakers. In the past, the most popular talks have featured internal details of new or upcoming features in OpenZFS. If you're new to the OpenZFS community or just new to giving talks at conferences, mentorship is available to help make your presentation the best it can be. The registration fee will be waived for speakers. The deadlines are as follows: *July 16, 2018 All abstracts/proposals submitted* to matt@mahrens.org July 27, 2018 Proposal submitters notified September 10, 2018 OpenZFS Developer Summit The second day of the event will be a hackathon, where you will have the opportunity to work with OpenZFS developers that you might not normally sit next to, with the goal of having something, no matter how insignificant, to demo at the end of the day. This event is only possible because of the efforts and contributions of the community of developers and companies that sponsor the event. Special thanks to our early Diamond sponsors: Delphix , Datto , and OsNexus . Additional sponsorship opportunities are available. Please see the website for details and send email to Karyn.Ritter@delphix.com if you have any questions. --matt From owner-zfs-devel@freebsd.org Mon Jul 30 21:49:45 2018 Return-Path: Delivered-To: zfs-devel@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9FEE41061EDD for ; Mon, 30 Jul 2018 21:49:45 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: from mail-ua0-x22b.google.com (mail-ua0-x22b.google.com [IPv6:2607:f8b0:400c:c08::22b]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 11ECC8639B for ; Mon, 30 Jul 2018 21:49:44 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: by mail-ua0-x22b.google.com with SMTP id g18-v6so8901240uam.6 for ; Mon, 30 Jul 2018 14:49:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=delphix.com; s=google; h=mime-version:from:date:message-id:subject:to; bh=hlYta6B3r0Azt5Yf1m64h9Cq6OoDCJfuTM0Piwqani8=; b=LnA7XuKiyIOKki99eO19qMZZOdDBuMRXg9bKOc0DQRTUNAVbvLt6i8DQRyOADRM5Y1 4dgmCIyIlUymMT0R0N+qzyR6H9QVTH8BMWett0Rsfd/IsLkmDlyKgScXyIqiar4AvOkB sfnDBqnolp7bwZZLGM/ABatZLYJzcRsTNrUZw= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=hlYta6B3r0Azt5Yf1m64h9Cq6OoDCJfuTM0Piwqani8=; b=ToJIx315wqoM1VuwYKfJUK1+viZpLdmLE0qkqMr3aXCXkfW53c51WEqRV6oiQjCPrp DSstr22imLtlY+RgUBmJBez75O+6E7o58RFhW2D/VPUJbQ9RbgLNjQBe5nu0dWCYR/4j crvKX0NPuOLkZhHBTHlDQzZ/fCKzrA0or5PyG7xdS7gAp9RnwBB755bbRY8JsZOs4hCU DPDiAs+ArXH8U2V8nzx4JkZcXJzcsQu9S4AYeAJ4YVe/sJCdWW/6DjJPDo8ZD+pI1d0T H5D1ZPAqJvVoQ2SW8C1svVXRiKYcG/OnvFySJsHlgkSYjHLNuoDBHUFWhlSwUmsubw17 d1Zg== X-Gm-Message-State: AOUpUlE3+HRYVu4BcEunwLGRzz2iv9NxbMCwDK0N6vbI4hWjFUzbwCHI 7kv5kWiPCOoUPS11xd5GLsnGJ4wlODwzeB8rSN45RNPARBw= X-Google-Smtp-Source: AAOMgpd9F9oW9M1HrPbAgJa8Ykuj0HV+sivp8en/cebpfXxA/jJqEsiL7cm7CCl7+iSYD+YD6wbP2zenEs9uvI2GeNw= X-Received: by 2002:ab0:4505:: with SMTP id r5-v6mr13650684uar.77.1532987384263; Mon, 30 Jul 2018 14:49:44 -0700 (PDT) MIME-Version: 1.0 From: Matthew Ahrens Date: Mon, 30 Jul 2018 14:49:33 -0700 Message-ID: Subject: OpenZFS Developer Summit 2018 - Speakers announced To: developer , illumos-zfs , zfs-devel@list.zfsonlinux.org, zfs-devel@freebsd.org, freebsd-fs , zfs-discuss Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.27 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Jul 2018 21:49:45 -0000 The sixth annual OpenZFS Developer Summit will be held in San Francisco, *September 10-11, 2018*. Everyone interested in OpenZFS is invited to participate, and registration is now open. See http://www.open-zfs.org/wiki/OpenZFS_Developer_Summit for all of the details, including slides and videos from previous years' conferences. The goals of the event are 1. to foster cross-community discussions of OpenZFS work and 2. to make progress on some of the projects we have proposed, at a 1-day hackathon. We have a great lineup of talks this year: TitleSpeakerCompany State of the Union Matt Ahrens Delphix ZIO Pipeline George Wilson Delphix zrepl , a one-stop ZFS replication solution Christian Schwartz Student Observing and Monitoring ZFS Metrics Using Open Source Tools Richard Elling Newisys Managing ZVOLs with vzvol Samantha Smith Moogsoft ZFS Hardware Acceleration with QAT Weigang Li Intel ZoL Releases Tony Hutter LLNL Device Removal Matt Ahrens Delphix iRAID Shailendra Tripathi Tegile, a Western Digital Brand TitleSpeakerCompany Log Spacemap: Flushing algorithm and performance Serapheim Dimitropoulos Delphix DRAID Rebuild Performance Carles Mateo Newisys Send Dedup Paul Dagnelie Delphix Allocation Classes Don Brady Delphix Vdev Properties Allan Jude Klara Systems This event is only possible because of the efforts and contributions of the community of developers and companies that sponsor the event. Special thanks to our early Diamond sponsors: Delphix , Datto , and OsNexus . Additional sponsorship opportunities are available. Please see the website for details and send email to Karyn.Ritter@delphix.com if you have any questions. --matt From owner-zfs-devel@freebsd.org Tue Oct 2 21:32:34 2018 Return-Path: Delivered-To: zfs-devel@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DAAF810ADBA2 for ; Tue, 2 Oct 2018 21:32:33 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: from mail-vs1-xe30.google.com (mail-vs1-xe30.google.com [IPv6:2607:f8b0:4864:20::e30]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7E93A7D863 for ; Tue, 2 Oct 2018 21:32:33 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: by mail-vs1-xe30.google.com with SMTP id e206so2018018vsd.0 for ; Tue, 02 Oct 2018 14:32:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=delphix.com; s=google; h=mime-version:from:date:message-id:subject:to; bh=PEaQqL2IG6c8uzfNBRXUOvLGpDAISvz3V4oQjHL8Qhs=; b=DzoNmwMcYh9HhLA/HMF7gE4scjBzzzib3bqXw49/mM8sP8EehIyAeEzzHKvopB4EyI /AlTksww47lodIZVTF7pu3FeU4UUhPE/hkYL53vX+5VetlKSEZIp30Nkj4EyrEjFjBSu Xda9K9czj40d5XgcELXDMol0RIiasUMA9TAvo= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=PEaQqL2IG6c8uzfNBRXUOvLGpDAISvz3V4oQjHL8Qhs=; b=NUkWvdhFOKEqua0pdr6hh1Ht7EcZHTY3V1zgAEG1f2/c7u4n98t9PFw3KzFBpMZwb9 ROdJdlvAK5AWfLTg09vz/3WHPGzsNYkwcL4FCXz6uFScOzOTj2OP7OAdn4C31E8SgyQ+ DXaNtnkXtlWk/sl/4kyGBSkQFMhxd9NWbz1gR0x8Xfb5g+1j5Qt/QkSFOJP0TlUIl2lS k/VpJb04XkV+daXFbAyx+cHiZkDAIo5aUv68Vyv4iy1RH8pAUojqAUxwkO25keM2Fyxx VZbaALHIY7ormas/g7rpKOGqhw+iJNmdpsfobs9ddVbQSUadvpwsZPLRtlkg+fLgT/C+ lzJg== X-Gm-Message-State: ABuFfohIqdGd+QUOrNRh6Xb/wVesxa9yjcqNqbEisBC0a8T0GAebaK8b h7DmFpsW5PwnSUZrW3p9nUkVxDXx5bx4u1XP4va5HA== X-Google-Smtp-Source: ACcGV628IVpZ9boYMLcklU/68Wb2Gl7bGUHSTe94Ytwh2qzJaKHVqYA5p9DMzvBTOls/fEWbJA5+P58LKbAbTtH5v88= X-Received: by 2002:a67:c982:: with SMTP id y2-v6mr6350959vsk.94.1538515952601; Tue, 02 Oct 2018 14:32:32 -0700 (PDT) MIME-Version: 1.0 From: Matthew Ahrens Date: Tue, 2 Oct 2018 14:32:21 -0700 Message-ID: Subject: First OpenZFS Leadership Meeting To: developer , illumos-zfs , zfs-devel@list.zfsonlinux.org, zfs-devel@freebsd.org, freebsd-fs , zfs-discuss Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.27 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Oct 2018 21:32:34 -0000 At this year's OpenZFS Developer Conference, we decided we should have more regular meetings to coordinate activities across all the platforms. The first meeting will be held *October 9th, 1:00-1:30pm Pacific time*, and we plan to hold this every 4 weeks thereafter. Everyone is welcome to attend and participate, and we will try to keep the meeting on agenda and on time. The meetings will be held online via Zoom , and recorded and posted to the website and YouTube after the meeting. The agenda for the first meeting will be a discussion of *feature availability on each platform*, with a goal of finding owners for everything in the "integrated to at least one platform" section. For more information and details on how to attend, please see the agenda document: https://docs.google.com/document/d/1w2jv2XVYFmBVvG1EGf-9A5HBVsjAYoLIFZAnWHhV-BM/edit --matt p.s. Videos from the OpenZFS Developer Summit are now available: http://www.open-zfs.org/wiki/OpenZFS_Developer_Summit_2018 From owner-zfs-devel@freebsd.org Mon Nov 5 18:12:14 2018 Return-Path: Delivered-To: zfs-devel@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A109C1107ED6 for ; Mon, 5 Nov 2018 18:12:14 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: from mail-vk1-xa31.google.com (mail-vk1-xa31.google.com [IPv6:2607:f8b0:4864:20::a31]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id CD4DB6BFB6 for ; Mon, 5 Nov 2018 18:12:13 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: by mail-vk1-xa31.google.com with SMTP id o10-v6so2232667vki.6 for ; Mon, 05 Nov 2018 10:12:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=delphix.com; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to; bh=e0nuVUTA47/SnDAz3bClnr9Oc0tPtKetRKxdQBlmXlQ=; b=ElpWyrhspKNpHAOzM3/yEzKq6/JENrlAOEMP8qhuygifnOef5GZoRjIwjmpGcmZ81/ UtKfOVXrYe96ytGRKuFSmEtzSEL+6md0XB8EmDN8shwHIYRAXlnyx7qFs4L0uUM6MpWB Qkfo0hUruIhstHwuDE/uOOZkZvHNaKVEc/dNU= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=e0nuVUTA47/SnDAz3bClnr9Oc0tPtKetRKxdQBlmXlQ=; b=hiKSNO/tcVLRX+gCAzAY/5s3s3Zg7b/hLGQDZTkDAp7f4isT1OSfCTIX15ecq0fKua eNNWq7YHLcAb7AZ+nfZwszGYeGEAtPufE3BFaNMnAHaWDazfL9tvPPWQbAKjGuzL7laY //mu0cszaEylkKEQbpoOfeterbgfdLiDVeO09pxMiVhsF3nnuAY9I0VBf8Rhn6MSw15o B994HwdtB4dwqboAXcgkHyDGWEWAfgWJR1eanyK4z8dPntyEDtmVyBr63b/YNIzXwyYp WOCl9JUgq/m+WQagMiras3eOe+/UHfKJnVGk6d3e/hs4poAFc9LrMP9yrUWsA73hrddb S3Mw== X-Gm-Message-State: AGRZ1gJN26EOM7WQXVEl4Y78ze1P0yG5dRvnRP/AnuSHxbTXi8xTEJYf zIwidMi0L8OiJesWKSR5E4kPCsp+kXhzqtbqrYX6KA== X-Google-Smtp-Source: AJdET5daoZoOcLyMEOXbFVAdGava4emgqP08LQ19d4HmC+ZUfyumczKOraE8Os5iwcHCFBsnXu12j8tGF2HYxVIz7C8= X-Received: by 2002:a1f:6b14:: with SMTP id g20mr3335543vkc.47.1541441172870; Mon, 05 Nov 2018 10:06:12 -0800 (PST) MIME-Version: 1.0 References: In-Reply-To: From: Matthew Ahrens Date: Mon, 5 Nov 2018 10:06:01 -0800 Message-ID: Subject: Second OpenZFS Leadership Meeting To: developer , illumos-zfs , zfs-devel@list.zfsonlinux.org, zfs-devel@freebsd.org, freebsd-fs , zfs-discuss X-Rspamd-Queue-Id: CD4DB6BFB6 X-Spamd-Result: default: False [3.34 / 200.00]; RSPAMD_URIBL(4.50)[zoom.us]; ARC_NA(0.00)[]; R_DKIM_ALLOW(0.00)[delphix.com]; NEURAL_HAM_MEDIUM(-0.87)[-0.871,0]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(0.00)[+ip6:2607:f8b0:4000::/36]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[zfs-devel@freebsd.org]; RCPT_COUNT_FIVE(0.00)[6]; BAD_REP_POLICIES(0.10)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[delphix.com:+]; MX_GOOD(-0.01)[cached: alt1.aspmx.l.google.com]; DMARC_POLICY_ALLOW(0.00)[delphix.com,none]; RCVD_IN_DNSWL_NONE(0.00)[1.3.a.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; NEURAL_HAM_SHORT(-0.16)[-0.160,0]; FORGED_SENDER(0.30)[mahrens@delphix.com,matthew.ahrens@delphix.com]; RCVD_TLS_LAST(0.00)[]; IP_SCORE(-0.42)[ipnet: 2607:f8b0::/32(-0.61), asn: 15169(-1.44), country: US(-0.06)]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[mahrens@delphix.com,matthew.ahrens@delphix.com]; GREYLIST(0.00)[pass,body]; RCVD_COUNT_TWO(0.00)[2] X-Rspamd-Server: mx1.freebsd.org Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 05 Nov 2018 18:12:15 -0000 The second OpenZFS Leadership meeting will be held *November 6th, 1:00-1:30pm Pacific time*, and we plan to hold this every 4 weeks thereafter. Everyone is welcome to attend and participate, and we will try to keep the meeting on agenda and on time. The meetings will be held online via Zoom , and recorded and posted to the website and YouTube after the meeting. The agenda for the meeting will be a discussion of *feature availability on each platform*, with a goal of finding owners for everything in the "integrated to at least one platform" section. For more information and details on how to attend, as well as notes and video from the previous meeting, please see the agenda document: https://docs.google.com/document/d/1w2jv2XVYFmBVvG1EGf-9A5HBVsjAYoLIFZAnWHhV-BM/edit --matt From owner-zfs-devel@freebsd.org Mon Dec 3 19:50:11 2018 Return-Path: Delivered-To: zfs-devel@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B770F1330B46 for ; Mon, 3 Dec 2018 19:50:11 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: from mail-ua1-x92a.google.com (mail-ua1-x92a.google.com [IPv6:2607:f8b0:4864:20::92a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 232A66B054 for ; Mon, 3 Dec 2018 19:50:11 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: by mail-ua1-x92a.google.com with SMTP id j3so4885826uap.3 for ; Mon, 03 Dec 2018 11:50:11 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=delphix.com; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to; bh=vJ/bHEoYK/Sl90Fw6HnqR8KpHrjrb4ni6gvvA9cJZDs=; b=FN1D0TZIkG68pDUAzZSbHSK4ZZGTnScSSkuhe+ja6K7mCcJmxstGrTYATMDVCGyO5K 9Px/VeQQLybmAoZhOBOVtd+w+INkWAofjE2AwLoGPluwmlRk/kzE1NiEN6lMStqc7B+G HdseNh3FCd9cv8pLBY5YMbyjeZUxgTtriqjpk= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=vJ/bHEoYK/Sl90Fw6HnqR8KpHrjrb4ni6gvvA9cJZDs=; b=GnvUHOXI+V3XDlOirsTbbKLYMzy24uUIXmQqDfpBk6oTlEje3mb3hlqxg5nzyKfpJX 7ecu/5MLrRA0ugBiZRFaQ0TD+Fxgw30J5KyU5agoyrR9jj761Cafe5QXWIcu/MfGo7hQ r5VDyXhj2zyEJAFNCFCrVBzcBdPHR8yY1Qih6kM9hmGXnOkXuMZPLMhQhqh7OIROZuR4 fE9C5+vtw44X92jRwluoajuw2t+ZtzklhjPVdTQrUhK5pYDXo3MmL8Pm7rytEvD2n0zU ZuaWfkD4+KTkWQjkMn2UcGSYoNUeltRqmYwRzu9o+x6JydGW+c/u8crYKa10bSggoSin VBcw== X-Gm-Message-State: AA+aEWaZDBAqOownlY4xisYCt2uMVo5ZlXedWidM9kPwcUOcJbw/GMDA VvHBTFsJOeI4LXk4xZiU5gUpuXj45SUE/6WNqB+B8g== X-Google-Smtp-Source: AFSGD/WEge9Qi6bLPqRwKlfPRAsRHcNR1yckSrOcVmytwhRyTDGpfuA4gDT4KQ25J9pMU7/NrKC0LS0caa/51tJffHk= X-Received: by 2002:ab0:7251:: with SMTP id d17mr8085298uap.0.1543866609714; Mon, 03 Dec 2018 11:50:09 -0800 (PST) MIME-Version: 1.0 References: In-Reply-To: From: Matthew Ahrens Date: Mon, 3 Dec 2018 11:49:58 -0800 Message-ID: Subject: Monthly OpenZFS Leadership Meeting To: developer , illumos-zfs , zfs-devel@list.zfsonlinux.org, zfs-devel@freebsd.org, freebsd-fs , zfs-discuss X-Rspamd-Queue-Id: 232A66B054 X-Spamd-Result: default: False [-4.28 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; R_DKIM_ALLOW(-0.20)[delphix.com]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; NEURAL_HAM_LONG(-1.00)[-0.999,0]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[zfs-devel@freebsd.org]; RCPT_COUNT_FIVE(0.00)[6]; NEURAL_HAM_SHORT(-0.97)[-0.973,0]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[delphix.com:+]; MX_GOOD(-0.01)[cached: alt1.aspmx.l.google.com]; DMARC_POLICY_ALLOW(-0.50)[delphix.com,none]; RCVD_IN_DNSWL_NONE(0.00)[a.2.9.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; FORGED_SENDER(0.30)[mahrens@delphix.com,matthew.ahrens@delphix.com]; RCVD_TLS_LAST(0.00)[]; IP_SCORE(-0.59)[ipnet: 2607:f8b0::/32(-1.60), asn: 15169(-1.28), country: US(-0.09)]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[mahrens@delphix.com,matthew.ahrens@delphix.com]; RCVD_COUNT_TWO(0.00)[2] X-Rspamd-Server: mx1.freebsd.org Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Dec 2018 19:50:12 -0000 The next OpenZFS Leadership meeting will be held tomorrow, December 4th, 1:00-1:30pm Pacific time, and we plan to hold this every 4 weeks thereafter. Everyone is welcome to attend and participate, and we will try to keep the meeting on agenda and on time. The meetings will be held online via Zoom, and recorded and posted to the website and YouTube after the meeting. The agenda for the meeting will be a discussion of feature availability on each platform, with a goal of finding owners for everything in the "integrated to at least one platform" section. For more information and details on how to attend, as well as notes and video from the previous meeting, please see the agenda document: https://docs.google.com/document/d/1w2jv2XVYFmBVvG1EGf-9A5HBVsjAYoLIFZAnWHhV-BM/edit --matt From owner-zfs-devel@freebsd.org Thu Dec 6 20:57:41 2018 Return-Path: Delivered-To: zfs-devel@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E539D13132FA for ; Thu, 6 Dec 2018 20:57:40 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: from mail-vk1-xa2c.google.com (mail-vk1-xa2c.google.com [IPv6:2607:f8b0:4864:20::a2c]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id F07B68149A for ; Thu, 6 Dec 2018 20:57:39 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: by mail-vk1-xa2c.google.com with SMTP id n126so446441vke.12 for ; Thu, 06 Dec 2018 12:57:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=delphix.com; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to; bh=jV6XHj+lnKeG5MQ8nzfc+QLmb54asy7xEayRD8MlnzE=; b=QyDny6NroJbcOyOy7Y+mS89JlkHoI0Bm79uDYqI16ljZ585rqAhD7ey6SO0C1IMve1 Lcr6bmboam2IX/nHCz6EAfCdb8VEwaXBo7MzrM771CSIQtjCcp9sYqsK/UqMCKeISxtR 1k7ZLve/MTjjNvnEjsVZlq2pAAxXFH/FYWvVw= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=jV6XHj+lnKeG5MQ8nzfc+QLmb54asy7xEayRD8MlnzE=; b=l1wI+HgHD3tiHDTYWo3YYegkmKGZmwaIHBrS7wenm1A8R542lXXXH3HGrzydNw3XRC J412gQuNq4oC0S2oAeL7S1cMey00UtNkTg0dRYy5fc0f1WOk64OB6Sd16T+0yHic3Tsy KuubZNUM6L2q+roHmXOGXIQczn3VhizyKym6/pS/g/F3l4U7mxtjQY9sf8vbKaUFHOf7 Xs8CRQ8EMv7G0jxXDt0JnL6oyPvkZs9GkDJ4QQ39/zOonw03/gMTPKnzzWYlAPL0u/xZ 7LQR14KPQ3PVPxeT5WP3ne96ZoRAuLJkGopYm1C1YYFpZHUesBhxtjqgCIyoC7c60Tz5 iYvQ== X-Gm-Message-State: AA+aEWbD78dJn2agmPW36q2Sc+b7UCwpIp9sJa8IK8OUcj3uzvxf8XEN HzL1P6n3FM39ZeRyVJG2IDTG+ipBRuaeBSKU0Km8CTAA X-Google-Smtp-Source: AFSGD/X8jjGO+FGuo8KE4i47WpZk5BWe0/VkwGBZHge59G7xXC9wzvTMlPRSM68MbHIkVFY52Bk63XDbNW9oSdLuG0g= X-Received: by 2002:a1f:aa44:: with SMTP id t65mr13198459vke.66.1544129859109; Thu, 06 Dec 2018 12:57:39 -0800 (PST) MIME-Version: 1.0 References: In-Reply-To: From: Matthew Ahrens Date: Thu, 6 Dec 2018 12:57:27 -0800 Message-ID: Subject: Re: Monthly OpenZFS Leadership Meeting To: developer , illumos-zfs , zfs-devel@list.zfsonlinux.org, zfs-devel@freebsd.org, freebsd-fs , zfs-discuss X-Rspamd-Queue-Id: F07B68149A X-Spamd-Result: default: False [-4.27 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; R_DKIM_ALLOW(-0.20)[delphix.com]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; NEURAL_HAM_LONG(-1.00)[-0.999,0]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[zfs-devel@freebsd.org]; RCPT_COUNT_FIVE(0.00)[6]; NEURAL_HAM_SHORT(-0.98)[-0.977,0]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[delphix.com:+]; DMARC_POLICY_ALLOW(-0.50)[delphix.com,none]; RCVD_IN_DNSWL_NONE(0.00)[c.2.a.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; MX_GOOD(-0.01)[alt1.aspmx.l.google.com,aspmx.l.google.com,aspmx2.googlemail.com,alt2.aspmx.l.google.com,aspmx3.googlemail.com]; FORGED_SENDER(0.30)[mahrens@delphix.com,matthew.ahrens@delphix.com]; RCVD_TLS_LAST(0.00)[]; IP_SCORE(-0.59)[ipnet: 2607:f8b0::/32(-1.53), asn: 15169(-1.31), country: US(-0.09)]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[mahrens@delphix.com,matthew.ahrens@delphix.com]; RCVD_COUNT_TWO(0.00)[2] X-Rspamd-Server: mx1.freebsd.org Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Dec 2018 20:57:41 -0000 Thanks to everyone who participated in the meeting on Tuesday. The meeting notes have been updated and the video is posted: https://docs.google.com/document/d/1w2jv2XVYFmBVvG1EGf-9A5HBVsjAYoLIFZAnWHhV-BM/edit https://www.youtube.com/watch?v=TwCc9zjhRw4&feature=youtu.be This month's meeting featured a substantial discussion about the goals and merits of the OpenZFS umbrella project and its relationship with the illumos, FreeBSD, and ZFSonLinux projects. I'm glad to have the opportunity to help everyone understand the goals of each of these projects. These goals are largely aligned: we all want ZFS to be high quality storage software that works well and similarly on lots of operating systems. The leaders in all of the operating-system-specific communities come from similar backgrounds of engineering rigor and agree that code changes should be undertaken deliberately, with evidence of sufficient testing, and with consideration of the impact on other operating systems. I'm happy to continue this discussion individually, or on the developer@open-zfs.org mailing list. The next meeting will be January 8, 1pm-2pm pacific time. We have several agenda items that we weren't able to get to this month (or last month!), so at the January meeting I'll be keeping us strictly to the agenda. If there are topics you'd like to discuss, please add them to the agenda. --matt On Mon, Dec 3, 2018 at 11:49 AM Matthew Ahrens wrote: > The next OpenZFS Leadership meeting will be held tomorrow, December 4th, > 1:00-1:30pm Pacific time, and we plan to hold this every 4 weeks > thereafter. Everyone is welcome to attend and participate, and we will try > to keep the meeting on agenda and on time. The meetings will be held > online via Zoom, and recorded and posted to the website and YouTube after > the meeting. > > The agenda for the meeting will be a discussion of feature availability on > each platform, with a goal of finding owners for everything in the > "integrated to at least one platform" section. > > For more information and details on how to attend, as well as notes and > video from the previous meeting, please see the agenda document: > > > https://docs.google.com/document/d/1w2jv2XVYFmBVvG1EGf-9A5HBVsjAYoLIFZAnWHhV-BM/edit > > --matt > From owner-zfs-devel@freebsd.org Mon Jan 7 19:14:04 2019 Return-Path: Delivered-To: zfs-devel@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 541EB149C8FB for ; Mon, 7 Jan 2019 19:14:04 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: from mail-vk1-xa2d.google.com (mail-vk1-xa2d.google.com [IPv6:2607:f8b0:4864:20::a2d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8E28A74B11 for ; Mon, 7 Jan 2019 19:14:00 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: by mail-vk1-xa2d.google.com with SMTP id v70so334111vkv.7 for ; Mon, 07 Jan 2019 11:14:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=delphix.com; s=google; h=mime-version:from:date:message-id:subject:to; bh=pQDI/XlueStA29B9vb6LvkUIAXx3zyYgtQnMe4HHUJU=; b=CUVcGM48IXprJwcU3QIsfvRlcZUiCcp8wzLsP/XVwjpg+1YUKOQnKSX+lDDggcWjXK SA/Zjr10xR2pBrNCwunscqi0hLJMvRjSruM0tCMu9oaciwjRKFyQJtlRZlprs0ZhjZEx x+DJxvX8Ov2iye4+lrCsr1siqVNe4MCcJuH1k= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=pQDI/XlueStA29B9vb6LvkUIAXx3zyYgtQnMe4HHUJU=; b=rVIghgD3WLIluWm0qE+Bd4tE4XR8xdk4MFXf+ClsqpsIEY2m+9ajJflKSMXji487uE 5QIefc4lykb+HUtlynAHazL/f58Wu+4Dqjrlu3vkOPuXgjsDX/HF7pqK6YWU7TU0CUjl OZdC3gIYWnz4MzY1Wm1YB7sSnay0na9F1a5AW1s83yh67rqDrXUDQZDoxfeBMvbhlPEe 40EljgWTf4gqHoxu4pYDsCZ4LcMYkZwP3YMNDRVBUpoitGkXPkr8XtXKjUl/CnxuYYV+ wnPEaDSO/wHGvky77s1q1RG07tK8vyszkeLRhJne7TN4H6S4R6TaluvByn/Gpc58J4tz SxFA== X-Gm-Message-State: AJcUukeAvULv8P602iKjU2R9CK6HEZ0s7QgM1JjwMaQE3+7uh/ULwNko 2KGbN/r/P0KgAAjhPs85D2gCPvFTqHsIi6Qxvo91og== X-Google-Smtp-Source: ALg8bN5s1f8K7obXTi7s6ZP4Uhl/oaa7GM+CSR6O9ftTNjC9RMuqUsYYe7GfpxIBJsG6pqg+lmCw4tDZkmK4ywhmcDY= X-Received: by 2002:a1f:e7c5:: with SMTP id e188mr21257270vkh.92.1546888439641; Mon, 07 Jan 2019 11:13:59 -0800 (PST) MIME-Version: 1.0 From: Matthew Ahrens Date: Mon, 7 Jan 2019 11:13:48 -0800 Message-ID: Subject: January OpenZFS Leadership Meeting To: developer , illumos-zfs , zfs-devel@list.zfsonlinux.org, zfs-devel@freebsd.org, freebsd-fs , zfs-discuss X-Rspamd-Queue-Id: 8E28A74B11 X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=delphix.com header.s=google header.b=CUVcGM48; dmarc=pass (policy=none) header.from=delphix.com; spf=pass (mx1.freebsd.org: domain of matthew.ahrens@delphix.com designates 2607:f8b0:4864:20::a2d as permitted sender) smtp.mailfrom=matthew.ahrens@delphix.com X-Spamd-Result: default: False [-4.29 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[delphix.com:s=google]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[zfs-devel@freebsd.org]; RCPT_COUNT_FIVE(0.00)[6]; NEURAL_HAM_SHORT(-0.79)[-0.790,0]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[delphix.com:+]; DMARC_POLICY_ALLOW(-0.50)[delphix.com,none]; RCVD_IN_DNSWL_NONE(0.00)[d.2.a.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; MX_GOOD(-0.01)[cached: alt1.aspmx.l.google.com]; IP_SCORE(-0.79)[ipnet: 2607:f8b0::/32(-2.21), asn: 15169(-1.68), country: US(-0.08)]; FORGED_SENDER(0.30)[mahrens@delphix.com,matthew.ahrens@delphix.com]; MIME_TRACE(0.00)[0:+,1:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[mahrens@delphix.com,matthew.ahrens@delphix.com]; RCVD_COUNT_TWO(0.00)[2] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Jan 2019 19:14:04 -0000 The next OpenZFS Leadership meeting will be held tomorrow, January 8th, 1:00-2:00pm Pacific time, and we plan to hold this every 4 weeks thereafter. Everyone is welcome to attend and participate, and we will try to keep the meeting on agenda and on time. The meetings will be held online via Zoom, and recorded and posted to the website and YouTube after the meeting. The agenda for the meeting will be a discussion of the projects listed in the agenda doc. For more information and details on how to attend, as well as notes and video from the previous meeting, please see the agenda document: https://docs.google.com/document/d/1w2jv2XVYFmBVvG1EGf-9A5HBVsjAYoLIFZAnWHhV-BM/edit --matt From owner-zfs-devel@freebsd.org Wed Jan 9 19:28:31 2019 Return-Path: Delivered-To: zfs-devel@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1DD6E148BF85 for ; Wed, 9 Jan 2019 19:28:31 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: from mail-ua1-x934.google.com (mail-ua1-x934.google.com [IPv6:2607:f8b0:4864:20::934]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B84B470E35 for ; Wed, 9 Jan 2019 19:28:29 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: by mail-ua1-x934.google.com with SMTP id c12so2804394uas.6 for ; Wed, 09 Jan 2019 11:28:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=delphix.com; s=google; h=mime-version:from:date:message-id:subject:to; bh=fGFSDSVdcVdUVd0PEllqFokPY5Mi/3QIJFZH/3NVxt8=; b=dMjmY6tIA+UDQ87tNv7I08sshtKd6ZuLlyPIRoBs8ORjKAXCEg/FmWCAh9WrtGdvHr 6s1D6dlXLgNbX9yfTa3bL3jORhGWmcSVVYcsV0i3FYkUHhcpjCAaMCd67NN8okeoPwed /HuyWQ90ONh5u++FMN/jxj847zIOu41N4MUGc= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=fGFSDSVdcVdUVd0PEllqFokPY5Mi/3QIJFZH/3NVxt8=; b=BARRVsU8LjEI5hL0VKhRujE/YwElZPrSU35yCYRJ38mwStxqCVeK+ry7oKY7chdhPA Lwa7yUQUggYb1VkqTOTg11135YoTT4EO78+KSUqIpwGx4gEH6wifmG6Q1JyESZ7mYuVR INIL/0Gi8bNC2Pf1I5ROlEKDUmsYoedTGFYo0GLxsApiPKmGRRDjE/zUFmiRu62Td7v3 OqcrqqiJb8IpmCQTPvd3ebH/CPm/y40RAiBreoJyknmcFWuVk/nifijvbjk9FDs26dKM mVqI1i3NGSgZO2AyXaP/cKjoeGx7I0O4sjDP7TanNEO0OAOu5sSWXw84FZeah6YJkPoA L79Q== X-Gm-Message-State: AJcUuke2nqVR6rUaGBqcsn4M2crsBdWuQ7yb7N/Z6e+5RtFnwmIVN3jL hkz+6oPQJYxuhg1Bj//ndHkKB6DxP4dbTrznGaczhg== X-Google-Smtp-Source: ALg8bN7I4yQM0faaffynzbt4B3r1v+z/8ddr2PbPITTS+1b+oypyg5E6C2574adrYqEX7PzukhqPgYxxu1i9Oq/5WJE= X-Received: by 2002:a9f:3f41:: with SMTP id i1mr2542194uaj.42.1547062108496; Wed, 09 Jan 2019 11:28:28 -0800 (PST) MIME-Version: 1.0 From: Matthew Ahrens Date: Wed, 9 Jan 2019 11:28:17 -0800 Message-ID: Subject: ZFS feature notifications To: developer , illumos-zfs , zfs-devel@list.zfsonlinux.org, zfs-devel@freebsd.org, freebsd-fs , zfs-discuss X-Rspamd-Queue-Id: B84B470E35 X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=delphix.com header.s=google header.b=dMjmY6tI; dmarc=pass (policy=none) header.from=delphix.com; spf=pass (mx1.freebsd.org: domain of matthew.ahrens@delphix.com designates 2607:f8b0:4864:20::934 as permitted sender) smtp.mailfrom=matthew.ahrens@delphix.com X-Spamd-Result: default: False [-4.36 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[delphix.com:s=google]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[zfs-devel@freebsd.org]; RCPT_COUNT_FIVE(0.00)[6]; NEURAL_HAM_SHORT(-0.84)[-0.841,0]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[delphix.com:+]; DMARC_POLICY_ALLOW(-0.50)[delphix.com,none]; RCVD_IN_DNSWL_NONE(0.00)[4.3.9.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; MX_GOOD(-0.01)[alt1.aspmx.l.google.com,aspmx.l.google.com,aspmx2.googlemail.com,alt2.aspmx.l.google.com,aspmx3.googlemail.com]; IP_SCORE(-0.81)[ipnet: 2607:f8b0::/32(-2.27), asn: 15169(-1.72), country: US(-0.08)]; FORGED_SENDER(0.30)[mahrens@delphix.com,matthew.ahrens@delphix.com]; MIME_TRACE(0.00)[0:+,1:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[mahrens@delphix.com,matthew.ahrens@delphix.com]; RCVD_COUNT_TWO(0.00)[2] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Jan 2019 19:28:31 -0000 As discussed at the December OpenZFS Leadership Meeting[1], we would like to make sure that all stakeholders are informed of important proposed changes, regardless of what platform they originate on (Linux, illumos, FreeBSD, etc). This will help us get input from the best experts at code review time. "Important" changes include: - adding (or removing!) a new user-visible feature, no matter how small (e.g. adding a new CLI flag) - changing the user-visible behavior of existing features (e.g. now X will automatically do Y for you) - changing the on-disk format (e.g. to improve performance) - changing the zfs send stream format As a rule of thumb, if you are changing a manpage (other than to clarify existing functionality), it's probably an "important" change. The notification will occur via email to the developer@open-zfs.org mailing list, so please join that list if you'd like to be informed about upcoming important changes to OpenZFS. If you have feedback on these changes, please let the author know within 7 days. This guideline applies to newly-proposed changes, not retroactively to open PR's. However, here are some examples of open PR's that would be classified as "important": https://github.com/openzfs/openzfs/pull/690 ("zfs allow" can take numeric id) https://github.com/openzfs/openzfs/pull/664 (new kstats) https://github.com/openzfs/openzfs/pull/668 (SIGINT/SIGKILL can cancel channel program) https://github.com/openzfs/openzfs/pull/649 (add new functions to channel programs) https://github.com/openzfs/openzfs/pull/638 ("zfs send" handles SIGINFO) https://github.com/zfsonlinux/zfs/pull/8238 (remove new subcommand) https://github.com/zfsonlinux/zfs/pull/8044 (new compression algorithm) https://github.com/zfsonlinux/zfs/pull/8255 (new property, subcommand) https://github.com/zfsonlinux/zfs/pull/7958 (new on-disk format, send stream format, subcommand) --matt [1] https://docs.google.com/document/d/1w2jv2XVYFmBVvG1EGf-9A5HBVsjAYoLIFZAnWHhV-BM/edit From owner-zfs-devel@freebsd.org Wed Jan 9 19:45:29 2019 Return-Path: Delivered-To: zfs-devel@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0BC04148C846 for ; Wed, 9 Jan 2019 19:45:29 +0000 (UTC) (envelope-from kate.hudson@blueskymediazone.com) Received: from n1nlsmtp03.shr.prod.ams1.secureserver.net (n1nlsmtp03.shr.prod.ams1.secureserver.net [188.121.43.193]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "relay-hosting.secureserver.net", Issuer "Starfield Secure Certificate Authority - G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4D8A571C48 for ; Wed, 9 Jan 2019 19:45:27 +0000 (UTC) (envelope-from kate.hudson@blueskymediazone.com) Received: from n3plcpnl0244.prod.ams3.secureserver.net ([160.153.157.141]) by : HOSTING RELAY : with ESMTP id hJTdgk7WyG7JRhJTdguDc6; Wed, 09 Jan 2019 12:25:21 -0700 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=blueskymediazone.com; s=default; h=Content-Type:MIME-Version:Message-ID: Date:Subject:To:From:Sender:Reply-To:Cc:Content-Transfer-Encoding:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:In-Reply-To:References:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=D+1qlT1wzQlZB6i/ZRxMRc+mhU8GhP9OjwmtAJGz0M4=; b=vRR0dtw37/EgDJmZuU+pANzy77 +hzhZWHtT5370wQzrbnN6796DsRLTaHe7xH+TGXPjnX0ATxbTYj+oGMF0RwWq+mWfFhGoKBQ2QVQG gb5NCcQv7I8EgS/88gyo0bhETN7YLQ6bCF0qre9eiE0/DF6lENJvXrvZdn2e9ZnkQpQODTzwYTZt/ IaOkTGvsnD8xiRGTDcVzFIyw/KD9V8idJeUeCSfe89t5GPeMaTed7F74pyKS+LDF2FYMb8Rl9imt9 iM1TWMOU5zgpdv2JWnurgMlxHhcqVjx0x+xzhHtP6F5Zrzhwvi/quCrHvaYw32c43PdplfOzTDYBc nxCDueew==; Received: from [191.96.145.34] (port=54997 helo=HIPL3005) by n3plcpnl0244.prod.ams3.secureserver.net with esmtpsa (TLSv1:ECDHE-RSA-AES256-SHA:256) (Exim 4.91) (envelope-from ) id 1ghJTY-001mWy-Hk for zfs-devel@freebsd.org; Wed, 09 Jan 2019 12:25:21 -0700 From: "Kate Hudson" To: Subject: Solaris Business Potential Contacts Date: Thu, 10 Jan 2019 00:54:54 +0530 Message-ID: MIME-Version: 1.0 X-Mailer: Microsoft Outlook 15.0 Thread-Index: AdSoUPUnb0FEPCK3QpaV9UyzBhlS1g== Content-Language: en-us X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - n3plcpnl0244.prod.ams3.secureserver.net X-AntiAbuse: Original Domain - freebsd.org X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - blueskymediazone.com X-Get-Message-Sender-Via: n3plcpnl0244.prod.ams3.secureserver.net: authenticated_id: kate.hudson@blueskymediazone.com X-Authenticated-Sender: n3plcpnl0244.prod.ams3.secureserver.net: kate.hudson@blueskymediazone.com X-Source: X-Source-Args: X-Source-Dir: X-CMAE-Envelope: MS4wfBOW3DQ02APyAL2wtBKPdKnvLeAFltGbU53WcZTX24vCMqQLTOhnThVi7ZXZYlmFmFAK85j0Szn9JXjdLZCBdgWLuvfuQpUtNx5mKPtPl9LsUyyH6ZVZ BPU887Rf8cchIMez9He/TlnKatYV49NrH7hOLVNHybIdPZJFwaz2wK97M8fqHXsS5VDYWTlpjKuLTM0TZ1Cy8DGO9DrdBAxOVsOX8DslVhfR3QjH0nrsItUK X-Rspamd-Queue-Id: 4D8A571C48 X-Spamd-Bar: +++++++ Authentication-Results: mx1.freebsd.org; dkim=none (invalid DKIM record) header.d=blueskymediazone.com header.s=default header.b=vRR0dtw3; spf=pass (mx1.freebsd.org: domain of kate.hudson@blueskymediazone.com designates 188.121.43.193 as permitted sender) smtp.mailfrom=kate.hudson@blueskymediazone.com X-Spamd-Result: default: False [7.06 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; MX_INVALID(0.50)[cached]; R_SPF_ALLOW(0.00)[+ip4:188.121.43.0/24]; HAS_X_SOURCE(0.00)[]; TO_DN_NONE(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; DKIM_TRACE(0.00)[blueskymediazone.com:~]; HAS_X_ANTIABUSE(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+]; IP_SCORE(0.63)[ip: (2.42), ipnet: 188.121.40.0/22(0.68), asn: 26496(0.14), country: US(-0.08)]; ASN(0.00)[asn:26496, ipnet:188.121.40.0/22, country:US]; MID_RHS_MATCH_FROM(0.00)[]; HAS_X_AS(0.00)[kate.hudson@blueskymediazone.com]; ARC_NA(0.00)[]; RECEIVED_SPAMHAUS_XBL(3.00)[34.145.96.191.zen.spamhaus.org : 127.0.0.4]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_SPAM_SHORT(0.93)[0.926,0]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; DMARC_NA(0.00)[blueskymediazone.com]; NEURAL_SPAM_MEDIUM(1.00)[1.000,0]; RCPT_COUNT_ONE(0.00)[1]; BAD_REP_POLICIES(0.10)[]; NEURAL_SPAM_LONG(1.00)[1.000,0]; RCVD_IN_DNSWL_NONE(0.00)[193.43.121.188.list.dnswl.org : 127.0.5.0]; R_DKIM_PERMFAIL(0.00)[blueskymediazone.com:s=default]; HAS_X_GMSV(0.00)[kate.hudson@blueskymediazone.com]; RCVD_TLS_ALL(0.00)[]; GREYLIST(0.00)[pass,body] X-Spam: Yes Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Jan 2019 19:45:29 -0000 Hi, I'm curious to know if you would be interested in acquiring our updated Contact list of Solaris Users. Each Contact contains:- Names, Title, Email, Phone, Company Name, Company URL, Company physical address, SIC Code, Industry, Company Size (Revenue and Employee). Let me know if you are interested and I will get back to you with Counts and Pricing. It would be GREAT if you could reply at the earliest! Regards, Kate Hudson Sr. Marketing Manager Santa Clara | CA To stop receiving future emails, please reply with "UNSUBSCRIBE "or "Leave Out "in the Subject Line. From owner-zfs-devel@freebsd.org Mon Jan 14 18:56:54 2019 Return-Path: Delivered-To: zfs-devel@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7461B1483FD0 for ; Mon, 14 Jan 2019 18:56:54 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: from mail-ua1-x932.google.com (mail-ua1-x932.google.com [IPv6:2607:f8b0:4864:20::932]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 2E9EA709FF for ; Mon, 14 Jan 2019 18:56:53 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: by mail-ua1-x932.google.com with SMTP id n7so28392uao.7 for ; Mon, 14 Jan 2019 10:56:53 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=delphix.com; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to; bh=KULufWgLJR51+HtB9oZN9XD+CufRM+VC+UW+GVssFLM=; b=YSohpUZdG44UwyT9CZPeYv/WMWxgRyf/j3Dm/tKAbpO63ZvaRO9N/6hzltPnb0dS92 mt/W4hXZhGMZR/TK6Am7PMyqKk+OFQMh5dJCfWdaQVvx34mRMJ++p0Y66mFiEhPKjZVm ZU6jiAZt5a7LMlkK8uQS5mDpsH/y6KcJD3A3g= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=KULufWgLJR51+HtB9oZN9XD+CufRM+VC+UW+GVssFLM=; b=sxdQEF0qYPf4xgOe8EQhHBYk/4bRdv/brHLOoDT9c9llLJbqLEftAFnJ2btyntf3L4 wEgS+SPQnRBTpNDB5ATcumt89Y9sLvBiutVO0rxSZBuoRGFTAgEs8WEHwnTZ7Tr1gEiP rDvLtDbuqWr/ACEK1ODBlRPi+2ogJUod1NW/Hj9nBdUpvg+SVG8NG7Jcow/TzFoD36TL iISHTYSbw5vOxbKy8TLI1SYs9x9C+YE4kkrl0aa+06e2fsPAtaFyYCd5/MwV4syLYiPp KyghBH+GiFeoBczpp7vIO0U/xS89eDr99+HPC9/rBcBrzJUyAQ8YeVNUlVnCzT1hfwY1 R6uQ== X-Gm-Message-State: AJcUukfXzIEh/0y8tzOoe4sXVvwZWo5Eet/QWgrkW9fPE1oF6Qmue1ps X315pcs29xjzYq40DKmPoKBNpJClkaeMuU2ynVMndg== X-Google-Smtp-Source: ALg8bN59DFUkCMMbt6Fk4T4CcTSGY4FaTQmNLpVyuEESMjcovCdS20aroTyIGG53sfIHLlp9gUcHn7Tv+o54Qsen/3c= X-Received: by 2002:ab0:7251:: with SMTP id d17mr10322749uap.0.1547492211376; Mon, 14 Jan 2019 10:56:51 -0800 (PST) MIME-Version: 1.0 References: In-Reply-To: From: Matthew Ahrens Date: Mon, 14 Jan 2019 10:56:38 -0800 Message-ID: Subject: Re: January OpenZFS Leadership Meeting To: developer , illumos-zfs , zfs-devel@list.zfsonlinux.org, zfs-devel@freebsd.org, freebsd-fs , zfs-discuss X-Rspamd-Queue-Id: 2E9EA709FF X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=delphix.com header.s=google header.b=YSohpUZd; dmarc=pass (policy=none) header.from=delphix.com; spf=pass (mx1.freebsd.org: domain of matthew.ahrens@delphix.com designates 2607:f8b0:4864:20::932 as permitted sender) smtp.mailfrom=matthew.ahrens@delphix.com X-Spamd-Result: default: False [-4.51 / 15.00]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; RCPT_COUNT_FIVE(0.00)[6]; DKIM_TRACE(0.00)[delphix.com:+]; DMARC_POLICY_ALLOW(-0.50)[delphix.com,none]; MX_GOOD(-0.01)[alt1.aspmx.l.google.com,aspmx.l.google.com,aspmx2.googlemail.com,alt2.aspmx.l.google.com,aspmx3.googlemail.com]; FORGED_SENDER(0.30)[mahrens@delphix.com,matthew.ahrens@delphix.com]; MIME_TRACE(0.00)[0:+,1:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[mahrens@delphix.com,matthew.ahrens@delphix.com]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[delphix.com:s=google]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_SHORT(-0.96)[-0.965,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[zfs-devel@freebsd.org]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[2.3.9.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; IP_SCORE(-0.83)[ipnet: 2607:f8b0::/32(-2.31), asn: 15169(-1.77), country: US(-0.08)]; RCVD_COUNT_TWO(0.00)[2] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 14 Jan 2019 18:56:54 -0000 The meeting notes and recording are now available: https://www.youtube.com/watch?v=Li8Z08KZvg8 https://docs.google.com/document/d/1w2jv2XVYFmBVvG1EGf-9A5HBVsjAYoLIFZAnWHhV-BM/edit?ts=5bb3b66c The next meeting will be January 29 at 11AM Pacific time. Note that this is 1 week and 2 hours earlier than the regular schedule. --matt On Mon, Jan 7, 2019 at 11:13 AM Matthew Ahrens wrote: > The next OpenZFS Leadership meeting will be held tomorrow, January 8th, > 1:00-2:00pm Pacific time, and we plan to hold this every 4 weeks > thereafter. Everyone is welcome to attend and participate, and we will try > to keep the meeting on agenda and on time. The meetings will be held > online via Zoom, and recorded and posted to the website and YouTube after > the meeting. > > The agenda for the meeting will be a discussion of the projects listed in > the agenda doc. > > For more information and details on how to attend, as well as notes and > video from the previous meeting, please see the agenda document: > > > https://docs.google.com/document/d/1w2jv2XVYFmBVvG1EGf-9A5HBVsjAYoLIFZAnWHhV-BM/edit > > --matt > > From owner-zfs-devel@freebsd.org Mon Jan 28 17:18:00 2019 Return-Path: Delivered-To: zfs-devel@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B2DFE14B977D for ; Mon, 28 Jan 2019 17:18:00 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: from mail-vs1-xe2a.google.com (mail-vs1-xe2a.google.com [IPv6:2607:f8b0:4864:20::e2a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7F5E76E886 for ; Mon, 28 Jan 2019 17:17:59 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: by mail-vs1-xe2a.google.com with SMTP id p74so10226024vsc.0 for ; Mon, 28 Jan 2019 09:17:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=delphix.com; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to; bh=7ppkC/BFJZ2t1hoYnfRp5Fk6bGlG68RhF3R3Kp7dvek=; b=XY69Pwg1ZP7FQ8ZMJhYzdm4kP+mBcUF5+4fW/Xp76Uid7quzPNjlpGDqw1R4PsMdSO A3KNocksCCKP+T66NEvszzqrnU8kywTO1vaTMPNsneRZYO9pucMdGKypAEh8GoXvb1gl mlRSBI1Gjin55UwXCQBdtOT9hszW+RkRTx9VQ= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=7ppkC/BFJZ2t1hoYnfRp5Fk6bGlG68RhF3R3Kp7dvek=; b=RaOCWFra3hLXeUI/+4bcw39P/gowQfYpVwa0mbLpq/uyo4XT1qmP3E6Xb/xfgyJyIz guY7/zPHetZaxpfibuBsGbUcwc2ikcPfTQhfpQ9LhVrPeo6iCeEpEM2E7yjqMg5rbozC XRiXXBPi/a3FVA3MzqnXBiC8/VUdbQNAHQLVoT5FY4XORo3cqDS0KmoIGFBGQ/mbatIv /rE9LehungvJa3jvwcBYvupkAqhoDFMrjnF4QPEtwuOwaI3Bp3t1RsEBybcO0Yd0zcGP 3OMeMcn0V1ILORu82RIIf1oYD+Pja7v6bwDxVe3mzW5/0Q91xIgqoHKaKDUuYtNWWSU+ VdqQ== X-Gm-Message-State: AJcUukePZe8ZMFK471xY0JoP8P3yo+McfzWY0u+Q6cJlyRVJaqaC3P+r pVTiFbeiOSSV2zkgyJqiJwZFyNm3gWKe45l1EhaLZg== X-Google-Smtp-Source: ALg8bN623CAA/rv/ljNcIcueQEE+FzlaM/edf4XfP3VIrRs1nBzyKXwhcvW/p86HURDFX/8bdmHb21QzMEovS/P3ldo= X-Received: by 2002:a67:eed8:: with SMTP id o24mr8180095vsp.237.1548695878595; Mon, 28 Jan 2019 09:17:58 -0800 (PST) MIME-Version: 1.0 References: In-Reply-To: From: Matthew Ahrens Date: Mon, 28 Jan 2019 09:17:47 -0800 Message-ID: Subject: Next OpenZFS Leadership Meeting To: developer , illumos-zfs , zfs-devel@list.zfsonlinux.org, zfs-devel@freebsd.org, freebsd-fs , zfs-discuss X-Rspamd-Queue-Id: 7F5E76E886 X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org; dkim=pass header.d=delphix.com header.s=google header.b=XY69Pwg1; dmarc=pass (policy=none) header.from=delphix.com; spf=pass (mx1.freebsd.org: domain of matthew.ahrens@delphix.com designates 2607:f8b0:4864:20::e2a as permitted sender) smtp.mailfrom=matthew.ahrens@delphix.com X-Spamd-Result: default: False [-6.49 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[delphix.com:s=google]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[zfs-devel@freebsd.org]; RCPT_COUNT_FIVE(0.00)[6]; NEURAL_HAM_SHORT(-0.95)[-0.946,0]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[delphix.com:+]; DMARC_POLICY_ALLOW(-0.50)[delphix.com,none]; RCVD_IN_DNSWL_NONE(0.00)[a.2.e.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; MX_GOOD(-0.01)[alt1.aspmx.l.google.com,aspmx.l.google.com,aspmx2.googlemail.com,alt2.aspmx.l.google.com,aspmx3.googlemail.com]; IP_SCORE(-2.84)[ip: (-9.73), ipnet: 2607:f8b0::/32(-2.46), asn: 15169(-1.92), country: US(-0.08)]; FORGED_SENDER(0.30)[mahrens@delphix.com,matthew.ahrens@delphix.com]; MIME_TRACE(0.00)[0:+,1:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[mahrens@delphix.com,matthew.ahrens@delphix.com]; RCVD_COUNT_TWO(0.00)[2] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 28 Jan 2019 17:18:00 -0000 The next OpenZFS Leadership meeting will be held tomorrow, January 29th, 11:00am-12noon Pacific time. *Note the earlier time for this meeting!* Everyone is welcome to attend and participate, and we will try to keep the meeting on agenda and on time. The meetings will be held online via Zoom, and recorded and posted to the website and YouTube after the meeting. The agenda for the meeting will be a discussion of the projects listed in the agenda doc. For more information and details on how to attend, as well as notes and video from the previous meeting, please see the agenda document: https://docs.google.com/document/d/1w2jv2XVYFmBVvG1EGf-9A5HBVsjAYoLIFZAnWHhV-BM/edit --matt From owner-zfs-devel@freebsd.org Tue Jan 29 21:48:57 2019 Return-Path: Delivered-To: zfs-devel@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1907614A92C4 for ; Tue, 29 Jan 2019 21:48:57 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: from mail-ua1-x930.google.com (mail-ua1-x930.google.com [IPv6:2607:f8b0:4864:20::930]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5F8996CE69 for ; Tue, 29 Jan 2019 21:48:55 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: by mail-ua1-x930.google.com with SMTP id d21so7388735uap.9 for ; Tue, 29 Jan 2019 13:48:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=delphix.com; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to; bh=ys5AT3DphPoB7JKRRZUWV+cBin23vT00KPGz5TEsN8s=; b=Go1O0mtKKvXEyGNGvfaswn8QjfGo55zeb+IJ5mWKn7hxVoYpiAujr7I4dcrQrIwvoi ZJRnKtGhk31LJuTb7z7LwDnZdvbzzCLABAg48hgcz6E60tKqcphiIGdZPOlwPzwb3TI6 6S8bjA00pbc+34Qvz0DHBLEQ+t1rcy2uQX4k8= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=ys5AT3DphPoB7JKRRZUWV+cBin23vT00KPGz5TEsN8s=; b=FWGkSUaJsPHi3n8hSkndqNZhPH+GjIMLw4kXaK7vCowcgpsnr27cR4BRw1ox7TbAMz DvNCC7X7Bi/vkVAPKmnErlFR3FrR1DP95nFbQv9IiSLpAYN4YaHuOukrlR6kE53OWNrH OIoLeEy1yi6dgUXKKulE1Y2l/C8D8hkWAf3VDXTR+o82wTaC/q2AD1v3FckATbjZkQiG uf9nKPoZM9wo81w5JIPdB6ecURoxC2ckKxP0jYCH7m1oFwxk0ygvCe2SdlwIoOMP4SlQ 1jakfJmzoD5vgG4gnmQGGdwKdmk4b3qVTuh1Eo2s8tI76XMYaXII3vfYQ6iqkMIg8+x5 S8Qw== X-Gm-Message-State: AJcUukeMy8yoawocHv7mLK8I35FZvBqSUlg8jTMgDxPu+CZGHINZEarN SHV+9bXpzlMyx3OS3od+u/60hsu+jGhNDLt9qN1atyXeBPg= X-Google-Smtp-Source: ALg8bN5UqoQpJ4UOLxH9/+O1bDyrQmPS+NxOP+bJHy4TA5NWqlwKpKlmqp/OJ62TV6wNUjUiLoxE6uO1Q/NjDtaWA04= X-Received: by 2002:ab0:164a:: with SMTP id l10mr11951622uae.48.1548798534339; Tue, 29 Jan 2019 13:48:54 -0800 (PST) MIME-Version: 1.0 References: In-Reply-To: From: Matthew Ahrens Date: Tue, 29 Jan 2019 13:48:42 -0800 Message-ID: Subject: Re: Next OpenZFS Leadership Meeting To: developer , illumos-zfs , zfs-devel@list.zfsonlinux.org, zfs-devel@freebsd.org, freebsd-fs , zfs-discuss X-Rspamd-Queue-Id: 5F8996CE69 X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org; dkim=pass header.d=delphix.com header.s=google header.b=Go1O0mtK; dmarc=pass (policy=none) header.from=delphix.com; spf=pass (mx1.freebsd.org: domain of matthew.ahrens@delphix.com designates 2607:f8b0:4864:20::930 as permitted sender) smtp.mailfrom=matthew.ahrens@delphix.com X-Spamd-Result: default: False [-6.55 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[delphix.com:s=google]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[zfs-devel@freebsd.org]; RCPT_COUNT_FIVE(0.00)[6]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[delphix.com:+]; MX_GOOD(-0.01)[alt1.aspmx.l.google.com,aspmx.l.google.com,aspmx2.googlemail.com,alt2.aspmx.l.google.com,aspmx3.googlemail.com]; RCVD_IN_DNSWL_NONE(0.00)[0.3.9.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; NEURAL_HAM_SHORT(-0.99)[-0.992,0]; DMARC_POLICY_ALLOW(-0.50)[delphix.com,none]; IP_SCORE(-2.85)[ip: (-9.71), ipnet: 2607:f8b0::/32(-2.52), asn: 15169(-1.95), country: US(-0.08)]; FORGED_SENDER(0.30)[mahrens@delphix.com,matthew.ahrens@delphix.com]; MIME_TRACE(0.00)[0:+,1:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[mahrens@delphix.com,matthew.ahrens@delphix.com]; RCVD_COUNT_TWO(0.00)[2] Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 29 Jan 2019 21:48:57 -0000 The meeting notes and recording are now available: https://www.youtube.com/watch?v=3DE6lvNOxCxOw&index=3D6&t=3D0s&list=3DPLaUV= vul17xSeHoMLiE_cp68mPvowtYteN https://docs.google.com/document/d/1w2jv2XVYFmBVvG1EGf-9A5HBVsjAYoLIFZAnWHh= V-BM/edit?ts=3D5bb3b66c We discussed status updates on several features, and also the future of the OpenZFS github account. The next meeting will be February 26 at 1PM Pacific time. Here are the notes from this meeting: - Code review progress on redacted send/recv (Paul D) - Still need people to review chunks to minimize code churn - BrianB: What is the schedule for 0.8? This wouldn=E2=80=99t make .8 b= ecause they are trying to get this out in Feb/March (as quickly as possible)= . - TRIM update (Brian) - User visible change is to align CLI with vdev =E2=80=A6 Everything el= se is under the covers - Looking for reviewers this week! Please review it if you have time. - Update on default pool features to enable (Josh P) - Bootpool hasn=E2=80=99t been considered. Proposal to include thoughts= about this to spark the broader discussion. - Josh P will send email with a consensus proposal. Please send feedback / questions. As needed, discuss at the next meeting - Remove dedup-ditto (Matt) - This is very little used and not very useful. The option still exists, and the system can deal with existing pools. But users won=E2= =80=99t be able to use this functionality going forward. - Silence and encouragement for removing this. - Remove zpool remap (Matt/Brian) - It is possible to get value out of this in certain circumstances, but there are lots of caveats. Also, the cost to maintain it is significa= nt since there is a bug that causes a panic if you=E2=80=99re removing a= device (remap) and a receive hits at the wrong time. - Seems like this functionality also isn=E2=80=99t much use, and isn=E2= =80=99t helpful. This has already been removed from Linux since there were no tagged releases (and it was new to Linux). Command line options were removed= in Linux, but the code is there and disabled. At some point in the not-too-distant future, the code would be removed. - Propose removing it from illumos & FreeBSD (shipped in 12.0 in Nov/Dec). - It is possible to get value out of this in certain circumstances, but there are lots of caveats. Also, the cost to maintain it is significa= nt since there is a bug that causes a panic if you=E2=80=99re removing a= device (remap) and a receive hits at the wrong time. - Josh C: Seems fine to remove the mechanism. For illumos, they will probably want to think more about what to do with the command (deprecation warning, etc) - Allan Jude: Seems to make sense to neuter it as soon as possible so that people don=E2=80=99t start to rely on it. Similar to Linux, ther= e should be and exit 0 and error. - Log spacemap (Serapheim) - Upstreaming of this feature, which improves random writes. It is an on-disk format change - Opened multiple PRs in ZoL to break down the changes. - Aiming to have a pull request for the feature end of this week / beginning of next week. - There are a lot of metaslab changes. - Igor: What about upstreaming to OZFS and illumos? - Serapheim: Planning to do this work after ZoL, and will do it as a separate change. Mostly the code is the same. Will give an update at the next meeting. - Fast clone deletion (Sara) - Originally developed on illumos as well. Pull request for OpenZFS and ZoL. - Performance enhancement for doing clone deletion, and tying it back to the number of changes on the clone. - PR has a lot of information about the design and implementation. - Igor: Please share performance details about this change. - Sara: Perf results are in the PR, will add test code for reproducing the results - How many event/fault management frameworks exist? Can they be unified? (Michael Dexter) - On illumos is system-wide, not just ZFS. Not sure about other platforms. - Igor: Should we port FMA to other platforms? - Don B: In Linux, they emulated enough of FMA to get things working, and there is a README in that code that provides some details. In Lin= ux, they had ZFS Event Daemon (ZED), and that allowed them to share the s= ame modules as illumos. - Josh C: FMA provides details about faults and their resolution. It goes through sysevent to a daemon. Events are collected across the system, and then you can correlate errors from across the system and ties those to issues in the vdev layer. It would likely be problematic to put all of this into ZFS because the broader context of the system and source of the error would be missing. - Josh P: FreeBSD also has some emulation. There was a lot of effort put into getting things working, and he would prefer not to touch it = at this point. - OpenZFS/openzfs github repo (Matt) - Primary motivator to create this repo: Make it easier to get changes into illumos for non-illumos developers (e.g., porting ZFS changes from one platform to another). - Secondary motivatory: Gives visibility to the project and community rather than having changes and activity distributed across platform-specific repositories. - The primary motivator is still valid and valuable. It is also a lot of work to maintain it. - Delphix is spending the most time on this, but we=E2=80=99re going to= be focusing on Linux going forward. - Do people find it useful? Are others willing to take this on / help? - Josh C: Joyent is looking to set up and maintain infrastructure this year (especially around testing), and provide facilities and drive engagement. You=E2=80=99re welcome to rename it to =E2=80=9Cillumo= s=E2=80=9D-something. - Allan J: Does it make sense to also use it to help coordinate changes across platforms? Like changes to on-disk format. - Mixing raw/non-raw send streams (Tom) - Last(?) encryption issue (that we know of!) - Tom is working on the PR Attendees - Linux: Brian Behlendorf, Andreas Dilger - illumos: Joshua M. Clulow, Kody Kantor, Sriram Narayanan, Jerry Jelinek, Sanjay Nadkarni, Joyce McIntosh, Roman - Linux & illumos: Matt Ahrens, Paul Dagnelie, Serapheim Dimitropoulos, Sara Hartse, Prakash Surya, Don Brady, John Kennedy, George Wilson, Alek Pinchuk - FreeBSD: Rainbow, Josh Paetzel, Pawel Dawidek, Allan Jude, Kirk McKusick - Cross-platform: Michael Dexter - Project management: Karyn Ritter Other attendees:, zfslurker, sef, Igor Kozhukhov, Dan Langille, Richard Elling, ram, Rich Teer,, Ryan Moeller, Tony Hutter, Alexander Motin, wombat --matt On Mon, Jan 28, 2019 at 9:17 AM Matthew Ahrens wrote: > The next OpenZFS Leadership meeting will be held tomorrow, January 29th, > 11:00am-12noon Pacific time. *Note the earlier time for this meeting!* > > Everyone is welcome to attend and participate, and we will try to keep th= e > meeting on agenda and on time. The meetings will be held online via Zoom= , > and recorded and posted to the website and YouTube after the meeting. > > The agenda for the meeting will be a discussion of the projects listed in > the agenda doc. > > For more information and details on how to attend, as well as notes and > video from the previous meeting, please see the agenda document: > > > https://docs.google.com/document/d/1w2jv2XVYFmBVvG1EGf-9A5HBVsjAYoLIFZAnW= HhV-BM/edit > > --matt > From owner-zfs-devel@freebsd.org Mon Feb 4 10:27:21 2019 Return-Path: Delivered-To: zfs-devel@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2406914C65D8 for ; Mon, 4 Feb 2019 10:27:21 +0000 (UTC) (envelope-from jack@gandi.net) Received: from gandi.net (mail12.gandi.net [IPv6:2001:4b98:dc4:5:ae1f:6bff:fe2d:9fdc]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 582E477A61 for ; Mon, 4 Feb 2019 10:27:20 +0000 (UTC) (envelope-from jack@gandi.net) Received: from thinkvoid (tgordon.gandi.net [217.70.181.24]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by gandi.net (Postfix) with ESMTPS id 4045B16033B for ; Mon, 4 Feb 2019 10:27:10 +0000 (UTC) Date: Mon, 4 Feb 2019 11:27:10 +0100 From: Jack Halford To: zfs-devel@freebsd.org Subject: parallel zfs mount Message-ID: <20190204102710.hpia7w46xqowofy4@thinkvoid> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Disposition: inline User-Agent: NeoMutt/20180716 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 04 Feb 2019 10:27:21 -0000 Hello all, Gandi has a need for parallel mounting of zfs datasets, apparently this has been implemented on illumos [1] and then was ported to ZOL [2]. I'm wondering if anybody has started any work in this direction for FreeBSD and maybe just hasn't upstreamed or doesn't have the time for the review, I'd be open to do that work. Otherwise I'll probably start porting this patch for FreeBSD myself. [1] https://www.illumos.org/issues/8115 [2] https://github.com/zfsonlinux/zfs/pull/8092 --jack From owner-zfs-devel@freebsd.org Mon Feb 4 20:16:34 2019 Return-Path: Delivered-To: zfs-devel@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7B4CB14B0EF6 for ; Mon, 4 Feb 2019 20:16:34 +0000 (UTC) (envelope-from mavbsd@gmail.com) Received: from mail-yb1-xb44.google.com (mail-yb1-xb44.google.com [IPv6:2607:f8b0:4864:20::b44]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 0F858708EE for ; Mon, 4 Feb 2019 20:16:34 +0000 (UTC) (envelope-from mavbsd@gmail.com) Received: by mail-yb1-xb44.google.com with SMTP id s76so125465ybs.10 for ; Mon, 04 Feb 2019 12:16:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=subject:to:references:from:openpgp:autocrypt:message-id:date :user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=gURhl72pM78DMgrSf7UCmcFjRuDHke9f8eZWdiQZ3wE=; b=c9UkFvqUT4Bmw9hZbks7rOZBtevxTaFaf4Z0/7BwFGcP4IrSU+geOvyVApoOjZK5KA DjVUqCdxNCyrqv5nUmT4uLXgcoCs7TD0hax/h8MeZ+UhMzLhIV916GXSG2RBWvaVS0IY ci0Ut59Rq//RMfq0wyZhFTqUbWuCJsbrnDvNO/t/UGa37/zAbc7/QaS8d/5nqzr3qaip ONUgNbGNmcnWIIpFaHrjLXn66UmnLArLBXncqs0nc6YKMDQeKX//7hR4GDZUkCwkkvSK YGvbJPxqvUCu6rY6gGMW+W1xl4u9eLuJOI08E8sgc7IU9ayTLvAhNfikmdB1C3mRv+/7 22TQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:references:from:openpgp:autocrypt :message-id:date:user-agent:mime-version:in-reply-to :content-language:content-transfer-encoding; bh=gURhl72pM78DMgrSf7UCmcFjRuDHke9f8eZWdiQZ3wE=; b=rDkXxi8dP8t1wqq123bA6mysFvQbTiRTnNESXZzhdNDLzdRZhxSlqOZqjkf6wzz/Dz s4ARv1SBAKDjBtEkJOeomMSccWfxk0D8NHyMz6+nt4XmE8Qw9GArSDesz84FwbI0rpyF VmUGrQfaJ7BCMZhf2Y98HimP16Gi2M33eSY3BLHQNT1gwxEfDZn2Y8nizyE7DVKXa43Z pSPwWU4763nTDm0ws+y3Lv+MImUkKPVe3OlyofkwbW+ni4AttKDjK3fgksk52zXBEdRZ 3MwAWBMfuiNOaqoIacHQ1ayFX2gp3yF6OjhH5Ps4WDme2hSD+uXFFF13aLwbCfm4wXtK Qp/A== X-Gm-Message-State: AHQUAuY5Qp/nqydM0MKZLPZ4jL+8v6BK7BRN3BpAf3tKsnvCW/UcLmqa 6zIQ8nF+FIKfnM0c5JGIZghuAo8r X-Google-Smtp-Source: AHgI3Iah5cgMuq5z1VYDsozLwtSI9Ji6F1vanFE7uAEpT9q9l1wdWPfMCxSkmZRfFebHvRUSdTb1vg== X-Received: by 2002:a25:ba8d:: with SMTP id s13mr1069420ybg.332.1549311392532; Mon, 04 Feb 2019 12:16:32 -0800 (PST) Received: from mavoffice.ixsystems.com ([12.189.233.129]) by smtp.gmail.com with ESMTPSA id c68sm694811ywd.52.2019.02.04.12.16.31 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 04 Feb 2019 12:16:32 -0800 (PST) Subject: Re: parallel zfs mount To: Jack Halford , zfs-devel@freebsd.org References: <20190204102710.hpia7w46xqowofy4@thinkvoid> From: Alexander Motin Openpgp: preference=signencrypt Autocrypt: addr=mavbsd@gmail.com; prefer-encrypt=mutual; keydata= xsBNBFOzxAwBCADkPrax0pI2W/ig0CK9nRJJwsHitAGEZ2HZiFEuti+6/4UVxj81yr4ak/4g 9bKUyC7rMEAp/ZHNhd+MFCPAAcHPvtovnfykqE/vuosCS3wlSLloix2iKVLks0CwbLHGAyne 46lTQW74Xl/33c3W1Z6d8jD9gVFT/xaVzZ0U9xdzOmsYAZaAj4ki0tuxO9F7L+ct9grRe7iP g8t9hai7BL4ee3VRwk2JXnKb7UvBiVITKYWKz1jRvZIrjPokgEcCLOSlv7x/1kjuFnj3xWZU 7HSFFT8J93epBbrSSCsYsppIk2fZH41kaaFXsMQfTPH8wkeM6qwrvOh4HiQM08R+9tThABEB AAHNIUFsZXhhbmRlciBNb3RpbiA8bWF2QEZyZWVCU0Qub3JnPsLAlwQTAQoAQQIbAwULCQgH AwUVCgkICwUWAwIBAAIeAQIXgAIZARYhBOmM88TmnMPNDledVYMYw5VbqyJ/BQJZYMKuBQkN McyiAAoJEIMYw5VbqyJ/tuUIAOG3ONOSNYqjK4eTZ1TVh9jdUBAhWk5nhDFnODN49Wj0AbYm 7aIqy8O1hnCDSZG5LttjSAo3UfXJZDKQM0BLb0gpRMBnAYqO6tdolLNqAbPGJBnGoPjsh24y 6KcbDaNnis+lD4GwPXwQM+92wZGhCUFElPV9NciZGVS65TNIgk7X+yEjjhD1MSWKKijZ1r9Z zIt4OzUTxxNOvzdlABZS88nNRdJkatOQJPmFdd1mpP6UzTNCiLUo1pIqOEtJgvVVDYq5WHY6 tciWWYdmZG/tIBexJmv2mV2OLVjXR6ZeKmntVH14H72/wRHJuYHQC+r5SVRcWWayrThsY6jZ Yr4+raTOwE0EU7PEDAEIAOZgWf2cJIu+58IzP2dkXE/urj3tr4OqrB/yHGWUf71Lz6D0Fi6Z AXgDtmcFLGPfMyWuLAvSM+xmoguk7zC4hRBYvQycmIhuqBq1jO1Wp/Z+lpoPM/1cDYLn8Flv mI/c40MhUZh345DA4jYWWaZNjQHUWVQ1fPf595vdVVMPT/abE8E5DaF6fSkRmqFTmfYRkfbt 3ytU8NdUapDcJVY7cEP2nJBVNZPnOIObR/ZIgSxjjrG5o34yXoqeup8JvwEv+/NylzzuyXEZ R1EdEIzQ/a1nh/0j4NXtzZEqKW4aTWlmSqb6wN8jh1OSOOqkYsfnE3nfxcZbxi4IRoNQYlm5 9R8AEQEAAcLAZQQYAQoADwUCU7PEDAIbDAUJBaOagAAKCRCDGMOVW6sif7FRB/4k9y/GaGqU fcJiXdQHRAKHCUvbKMFgeEDHOg33qx+POS2Ah85/PXVa2jYBldCZDmYc+zl48aEMd163a7s3 0gJaB7CYElwxlKUk6c+5gwoYIJuJJzSzW0JzSD5ch7RIRxbfxrKdsiHrUW8AeduZWzlK6VaW RmWILgLmxfLdhEVFWxbr99GSeVFZaZwn6tl/8CvBcgYoARvJvl0V5zS1akQfEISYkwL9EfUI W44EOHranL5qUXkedXBYp6fRsooGrIimfwYxaC8FbXhk3FMgMjDMRiVq4POHo1iGeYETsUrL NM6184E25gPVtX2fb3RhM8Xh6BkwCZ6ZYbQ+AcD4F/cKwsB8BBgBCgAmAhsMFiEE6YzzxOac w80OV51VgxjDlVurIn8FAllgwtgFCQ0xzMwACgkQgxjDlVurIn9OqAf9FAcKWS95wTTbraXA qg/+bQyHgjlMtGCgkmfxLsbUGeqiFgmSIuoDrF7q6sYPs6p00CXXZRuuNZt0lX7O95re8mgz gxm5iJisZpdbHMVepYlw/AxT2wCHwxGCEe64Lm+A9vjlOd+3D3/6fSLwZ9WFCE6p6lQZ1CDg 09xe+JKSgC+KDqmn0tzGKyfSCuhRAq3XkZyxL1hxBaDeP0eeKlzoy7jXodf3wVvXXc0cmpza B5McuRHK4EU6jIioHo30YqPM4AjPHGxV2X1N6/Aayungzj9EXNZtKCxs6dsTvjniWa5VkZ9F 4SOdSbxEen1DZRYpeWnd7GVmO86n+5USkKCXPg== Message-ID: Date: Mon, 4 Feb 2019 15:16:31 -0500 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101 Thunderbird/52.7.0 MIME-Version: 1.0 In-Reply-To: <20190204102710.hpia7w46xqowofy4@thinkvoid> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 0F858708EE X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-6.99 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; NEURAL_HAM_SHORT(-0.99)[-0.985,0]; REPLY(-4.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0] X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 04 Feb 2019 20:16:34 -0000 Hi Jack, I've tried to merge respective Illumos patches to FreeBSD when they got to Illumos, but that appeared to be problematic due to total mess of kernel and user-space locking primitives and types in ZFS headers, additionally complicated by FreeBSD shims. After spending there several days I gave up. I haven't checked myself, but I heard that Linux took different approach and just reimplemented it completely in different primitives, possibly to avoid the same kind of problems. If I had time for second try, I'd try this way. On 04.02.2019 05:27, Jack Halford wrote: > Hello all, > > Gandi has a need for parallel mounting of zfs datasets, apparently this > has been implemented on illumos [1] and then was ported to ZOL [2]. I'm > wondering if anybody has started any work in this direction for FreeBSD > and maybe just hasn't upstreamed or doesn't have the time for the > review, I'd be open to do that work. > > Otherwise I'll probably start porting this patch for FreeBSD myself. > > [1] https://www.illumos.org/issues/8115 > [2] https://github.com/zfsonlinux/zfs/pull/8092 > > --jack > _______________________________________________ > zfs-devel@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/zfs-devel > To unsubscribe, send any mail to "zfs-devel-unsubscribe@freebsd.org" -- Alexander Motin From owner-zfs-devel@freebsd.org Mon Feb 25 18:53:39 2019 Return-Path: Delivered-To: zfs-devel@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3D94415124BA for ; Mon, 25 Feb 2019 18:53:39 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: from mail-ua1-x929.google.com (mail-ua1-x929.google.com [IPv6:2607:f8b0:4864:20::929]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5A85073339 for ; Mon, 25 Feb 2019 18:53:38 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: by mail-ua1-x929.google.com with SMTP id r21so2222216uan.11 for ; Mon, 25 Feb 2019 10:53:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=delphix.com; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to; bh=p7gniPdyOIWgfldZ8hteAI2xgH+LMXZZIZfzQSSAFIs=; b=H4fcQsSJwW1WmSmcBa0bB5hKEiIaJcNGy24MlA6G5ie3QUXM46mWOOXosctSbULGeH xea9e/rMuPzqCIb78lQqFheLKtdbw5dEwBgMyoSM2EbvETT1y0YjnUbLzcJkwDGKn9Jz 7q7aaoweHXjR+ZD33eoFeJRXOYrh0a9Jc1qPg= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=p7gniPdyOIWgfldZ8hteAI2xgH+LMXZZIZfzQSSAFIs=; b=GWSL+A2UKMxOfFrCvsYtZurmUobKhqDyOSwCzZab/GjqlSdAEWxe/c+9gxO+hnKbCc 3nmzHKKE34PlAgKEbZidJQdyuDdNkA2Ay3iwirYIl4Cict8jy8XZ1Cgk8t7bvg/3bkBv vcRkrld4BJwXjqc/11bUwS3t7ILYUzW84h0Zu1gAsFe7S7ev62o6386HP+TGf1FedIwQ eCNfoFDqAz8iPgG7ef7gvm3qPBqpkvr3RoIyYhi6ms1I3UOErf8Js/3oXDvJee0+AxgX DC+LmHse1rbMw+M8YlQMuIvbTt0Iy1t/rCXRDZnNMKj1Go4vS6J8ZBGa4y5aH2sG7g9V bQMw== X-Gm-Message-State: AHQUAuZfW9ilcd1FWPBtC3R9/JxpYGB5EGFAKHBR0EEzWm053TDpdAZ3 EpQT1k7fpXRAHZhCgSCTRUpPrR35mv3bdjkjjCpnsw== X-Google-Smtp-Source: AHgI3IafeuLS5/+2W+wVwpVxl0ByioXBCsh9XPQgnO9GKnnQTuPXoqnNvSNW2hT1+ogpG8UgxZ4FRWuEaJmHBEcjB2Q= X-Received: by 2002:a67:eed8:: with SMTP id o24mr9112383vsp.237.1551120817403; Mon, 25 Feb 2019 10:53:37 -0800 (PST) MIME-Version: 1.0 References: In-Reply-To: From: Matthew Ahrens Date: Mon, 25 Feb 2019 10:53:26 -0800 Message-ID: Subject: February OpenZFS Leadership Meeting To: developer , illumos-zfs , zfs-devel@list.zfsonlinux.org, zfs-devel@freebsd.org, freebsd-fs , zfs-discuss X-Rspamd-Queue-Id: 5A85073339 X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=delphix.com header.s=google header.b=H4fcQsSJ; dmarc=pass (policy=none) header.from=delphix.com; spf=pass (mx1.freebsd.org: domain of matthew.ahrens@delphix.com designates 2607:f8b0:4864:20::929 as permitted sender) smtp.mailfrom=matthew.ahrens@delphix.com X-Spamd-Result: default: False [-4.34 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; R_DKIM_ALLOW(-0.20)[delphix.com:s=google]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[zfs-devel@freebsd.org]; RCPT_COUNT_FIVE(0.00)[6]; NEURAL_HAM_SHORT(-0.68)[-0.684,0]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[delphix.com:+]; DMARC_POLICY_ALLOW(-0.50)[delphix.com,none]; RCVD_IN_DNSWL_NONE(0.00)[9.2.9.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; MX_GOOD(-0.01)[cached: alt1.aspmx.l.google.com]; IP_SCORE(-0.95)[ipnet: 2607:f8b0::/32(-2.66), asn: 15169(-2.00), country: US(-0.07)]; FORGED_SENDER(0.30)[mahrens@delphix.com,matthew.ahrens@delphix.com]; MIME_TRACE(0.00)[0:+,1:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[mahrens@delphix.com,matthew.ahrens@delphix.com]; RCVD_COUNT_TWO(0.00)[2] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 25 Feb 2019 18:53:39 -0000 The next OpenZFS Leadership meeting will be held tomorrow, February 26, 1pm-2pm Pacific time. *Note we are back to the usual time for this meeting!* Everyone is welcome to attend and participate, and we will try to keep the meeting on agenda and on time. The meetings will be held online via Zoom, and recorded and posted to the website and YouTube after the meeting. The agenda for the meeting will be a discussion of the projects listed in the agenda doc. For more information and details on how to attend, as well as notes and video from the previous meeting, please see the agenda document: https://docs.google.com/document/d/1w2jv2XVYFmBVvG1EGf-9A5HBVsjAYoLIFZAnWHhV-BM/edit --matt From owner-zfs-devel@freebsd.org Wed Feb 27 17:04:42 2019 Return-Path: Delivered-To: zfs-devel@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5456915225F2 for ; Wed, 27 Feb 2019 17:04:42 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: from mail-vk1-xa44.google.com (mail-vk1-xa44.google.com [IPv6:2607:f8b0:4864:20::a44]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 2F3B986AD2 for ; Wed, 27 Feb 2019 17:04:41 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: by mail-vk1-xa44.google.com with SMTP id 17so4018620vkf.4 for ; Wed, 27 Feb 2019 09:04:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=delphix.com; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to; bh=dzhYxylufNQJRJm2kpmXPBpKmouEU15RK8I0LYAsYtQ=; b=EVrsFOv0EmrwE0kngmtuuPUo3HabxH4TVZGr/ZAE+kTWlvyN8t7hIPkjmYna49CIk6 5XGkFaMSmv/1hzqo993Ie/sMrOseoD4UhwvaBgRNDwUvGdimjBLlqsOydIMdH3hKQuFU CHsb6dQIUSP0FNVJ2a9sSMpZF9Rf8o1VIIRus= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=dzhYxylufNQJRJm2kpmXPBpKmouEU15RK8I0LYAsYtQ=; b=a3O0gyiNN0G/Sk4MPozRPClqsxyem+5tnHz9FRi+AAtaaRGneL8ABgPvhPeHLaIutP AiN9yN3817+4HJiIpuOL4ul3GTeJJcjI9Ch8B2ctLJvhu8J1mtXCH2OfDnzDz8WUKCGh hgHyq3fL3qtX7nULjfJqCNzV8V1z6F5pf5epaVIpB4+CdsAr6J0TDZk9jzrIBAVMLY0B kvCLtW6EdS/vpc4T1GnpK/Rrd6XysSqb0C+frsuVgyVH3OYyw+qHFMbbQyCsNSjXoLJX vyeQnyX9RNzJutDNd6ftjOfuhssXNpLUzBdz3mA4xn6mGq/mwOOj9sTwrrrFME4FZCCo 9ltA== X-Gm-Message-State: AHQUAuZZiWvDtXVgVFVKc9ySHOAAiMAIVAt2mfJApAGMTjewI4E7QPaq PXAVKZqghxjioqij0R/ZJJXdLUeM5mSs0h1lmhmUaA== X-Google-Smtp-Source: AHgI3IYkbhh2CMAZy7uk+TwExm5H7oeLstVDGaGH2Lju1ccZ5IzJR7OSQS9WgQYPyOjYm3sk6gS4Z29nV49/5Iaxxu4= X-Received: by 2002:a1f:84c4:: with SMTP id g187mr1225557vkd.47.1551287080115; Wed, 27 Feb 2019 09:04:40 -0800 (PST) MIME-Version: 1.0 References: In-Reply-To: From: Matthew Ahrens Date: Wed, 27 Feb 2019 09:04:27 -0800 Message-ID: Subject: Re: February OpenZFS Leadership Meeting To: developer , illumos-zfs , zfs-devel@list.zfsonlinux.org, zfs-devel@freebsd.org, freebsd-fs , zfs-discuss X-Rspamd-Queue-Id: 2F3B986AD2 X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=delphix.com header.s=google header.b=EVrsFOv0; dmarc=pass (policy=none) header.from=delphix.com; spf=pass (mx1.freebsd.org: domain of matthew.ahrens@delphix.com designates 2607:f8b0:4864:20::a44 as permitted sender) smtp.mailfrom=matthew.ahrens@delphix.com X-Spamd-Result: default: False [-3.32 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; R_DKIM_ALLOW(-0.20)[delphix.com:s=google]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[zfs-devel@freebsd.org]; RCPT_COUNT_FIVE(0.00)[6]; NEURAL_HAM_SHORT(-0.08)[-0.078,0]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[delphix.com:+]; DMARC_POLICY_ALLOW(-0.50)[delphix.com,none]; RCVD_IN_DNSWL_NONE(0.00)[4.4.a.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; MX_GOOD(-0.01)[cached: alt1.aspmx.l.google.com]; IP_SCORE(-0.53)[ip: (2.11), ipnet: 2607:f8b0::/32(-2.68), asn: 15169(-2.01), country: US(-0.07)]; FORGED_SENDER(0.30)[mahrens@delphix.com,matthew.ahrens@delphix.com]; MIME_TRACE(0.00)[0:+,1:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[mahrens@delphix.com,matthew.ahrens@delphix.com]; RCVD_COUNT_TWO(0.00)[2] Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 27 Feb 2019 17:04:42 -0000 At this meeting we discussed: - who will review Fast Clone Deletion - FIPS 140-2 certification - making compressed ARC mandatory - platform-specific 'sharenfs' property video recording available: https://www.youtube.com/watch?v=3DEXstK9ckcZQ meeting notes (thanks Karyn!): - Reviewers for fast clone deletion (ZoL PR ; illumos PR ) (Sara) - There is a feature flag change that Sara sent email out about. - Sara is seeking reviewers. - Brian volunteered to review, initial pass looks great - How much review is needed? - Will have some conflicts with bpobj iteration work. - BB: Wasn=E2=80=99t planning to get this in before 0.8, but if it woul= d be useful it is possible. - Let=E2=80=99s get it in right after 0.8 (which is due out in March= ) - There are a few fixes that are pending for 0.8 to go out. Matt will look at them! - FIPS 140-2 certification (Luke) - Defense contractors could use ZFS for many things, but require FIPS. Other industries (e.g., healthcare) also have this requirement. - JC: Can you get certification for a source or is it a specific binary build? - Rainbow: It is for specific binary builds, and she does a lot of compliance and can help here. You can do source code level certification. - sef: It is really expensive and time consuming. Level of configuration for testing and certification is super specific. - BB: We do have binaries from companies like RHEL, but they aren=E2= =80=99t official builds.=E2=80=9CFIPS verified=E2=80=9D rather than certif= ied? We=E2=80=99re already using the appropriate crypto algorithm. - PD: Certifying at the source level makes it easier for a vendor to get certified. There are some additional components that would probably need to be looked at (like hash algorithms). - MA: Certification applies to the crypto algorithm. Does that help us since it is a separate module. - AJ: Different Linux distros will have different binaries that would need to be certified separately. - Luke can connect with Rainbow and sef to see what would need to be done to see if it is viable. - Should compressed ARC be mandatory? (Issue ) (Allan J) - Does anyone turn off compressed ARC? If not, we can avoid special cases for this. - Please let Allan know right away if you do turn off compressed ARC. Else that functionality may just be taken out. - MA: Seems like there are some cases on Linux where we can be confident that people aren=E2=80=99t using this combination (i.e.,= it doesn=E2=80=99t work), but that doesn=E2=80=99t cover all cases. - AJ: The crypto changes definitely made it different than what was in FreeBSD. - JC: FWIW, compressed ARC makes ARC better in many different ways in Postgres (at least). They haven=E2=80=99t noticed any latency i= ncreases or memory overhead that has been called out. - AM: It is pretty pointless to disable compressed ARC. The difference when you disable it and use other mechanisms (e.g., bcopy), is negligible and there are other benefits to keep it on. - Please comment in github! - There was no significant negative feedback, so we plan to move forward with making compressed ARC mandatory. - Platform-specific sharenfs (George) - Sent this out the proposal last night. - Create platform-specific properties. These platform-specific properties won=E2=80=99t take effect when importing the pool on a dif= ferent platform. - MA: =E2=80=98sharenfs=E2=80=99 is a system property because ZFS ta= kes action based on it (e.g. share/unshare when you do =E2=80=98zfs rename=E2=80=99= ). It should be platform-specific because the value of the property isn=E2=80=99t verified/interpreted by ZFS - it=E2=80=99s passed to the OS-specif= ic share utility without modification. - AJ: Cross-OS import is a feature I=E2=80=99d like to keep as a 1st= class citizen. - MA: JC provided feedback on the proposal. Please talk about your counter-proposal. - JC: Biggest difference was to keep the functionality the same if people really want it, but there would be a =E2=80=9Cveneer=E2=80= =9D interface that would allow platform-specific properties. Generally people would just us= e whatever properties on their OS. Garrett=E2=80=99s comments are an appliance-centric view. The key bit of the proposal is to ensure that it isn=E2=80=99t dangerous to import onto a different platform. The better eventual solution is to actually do something good in this scenario. - AJ: What about namespacing the property like we do for the user properties: sharenfs:illumos. Maybe make it more feature flag. Do it in a way that is consistent with what is already done. - CS: bikeshed: org.openzfs.illumos:sharenfs - RE: There could be many different iSCSI servers, and would need to figure out how to get to the right one. Some sharing is cross-OS: samba, nfs-ganesha - AJ: share@samba, share@nfs-ganesha, share@illumos=E2=80=A6 -- Tie = this to the server rather than OS? - JC/GW: Maybe a =E2=80=9C.=E2=80=9D instead of =E2=80=9C@=E2=80=9D? - sef: Who is going to decide if this is an OS-specific property? - CS: Have some hooks into a layer (e.g., via lua) rather than having this built into zfs. - MA: Seems like George=E2=80=99s proposal is better than what we have = now, but we should get feedback from various platform vendors. On Mon, Feb 25, 2019 at 10:53 AM Matthew Ahrens wrote= : > The next OpenZFS Leadership meeting will be held tomorrow, February 26, > 1pm-2pm Pacific time. *Note we are back to the usual time for this > meeting!* > > Everyone is welcome to attend and participate, and we will try to keep th= e > meeting on agenda and on time. The meetings will be held online via Zoom= , > and recorded and posted to the website and YouTube after the meeting. > > The agenda for the meeting will be a discussion of the projects listed in > the agenda doc. > > For more information and details on how to attend, as well as notes and > video from the previous meeting, please see the agenda document: > > > https://docs.google.com/document/d/1w2jv2XVYFmBVvG1EGf-9A5HBVsjAYoLIFZAnW= HhV-BM/edit > > --matt > From owner-zfs-devel@freebsd.org Tue Mar 26 16:30:02 2019 Return-Path: Delivered-To: zfs-devel@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 73B821557DBF for ; Tue, 26 Mar 2019 16:30:02 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: from mail-vk1-xa29.google.com (mail-vk1-xa29.google.com [IPv6:2607:f8b0:4864:20::a29]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 62A9C72D92 for ; Tue, 26 Mar 2019 16:30:01 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: by mail-vk1-xa29.google.com with SMTP id q189so2943843vkq.11 for ; Tue, 26 Mar 2019 09:30:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=delphix.com; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to; bh=4ZlPAWu1GIeDuTo4Eyin8d0uEL4K46nBpBf5d0rrd1k=; b=RGSyxpWy04i3BulnrdiP3kjip9LPahrQEcFqC924pIFffVaPpv/OWhF1i0azBTLtFE uikPVaDs/5Ltho5oI/SAN2pRJ6qk4xXRsw4cyumFpyMmQBTGNUsxtfZMfKksd9UtWmtP UoKF8WpTYpgpSplUjW3Ll6Lwdx37PXa9DS9x8= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=4ZlPAWu1GIeDuTo4Eyin8d0uEL4K46nBpBf5d0rrd1k=; b=Yqw6N1L4rFxTVZhjx66jBEU9XSFdZFLtMXCIbtMDQmLd2lbJCOl92s3xJtnbljW0x2 5B9BLMMjJZkH+263Wr4rBUWgnYjV+TUldBqQCvWMC00UgVWL/5el5KWbZgfQ9mDd0Z+U 3/NSV+4TKvrPXkq63TQWDJU+um7I1FlC9mcK3/ELJRBiRxpgAPnfZ9V9l4Ev1WpTZC3i ODviBIUJX4W0euqBkZkRGaQ1oljo98eMyUcPiCNX4tZLO+tb5AjcITq8+q+TCAjnCcbu gf4q2g0SIA24E73xCs3hMs47WsISK2HYgxSoGSZcLDvunslGjvs0dZgluu49zBM/8PjF nnew== X-Gm-Message-State: APjAAAWJgWWHEAx9kkthHMfxk1rJFHaCusgHuJ82K+YiTqVgPRzxKUQ+ /KanE1/7DT4mQoRZSfp8VmgIRGsSx3u5k6QPUKZK1A== X-Google-Smtp-Source: APXvYqwBFFL6973Igl7e4KtHvJU0IbsTlqmFxbYhyehGgtpMrvAcBvgxZnnSARejsgQZCYwfAnduMLv4h2sHNejutGs= X-Received: by 2002:a1f:1714:: with SMTP id 20mr18496303vkx.47.1553617800363; Tue, 26 Mar 2019 09:30:00 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Matthew Ahrens Date: Tue, 26 Mar 2019 09:29:49 -0700 Message-ID: Subject: March OpenZFS Leadership Meeting To: developer , illumos-zfs , zfs-devel@list.zfsonlinux.org, zfs-devel@freebsd.org, freebsd-fs , zfs-discuss X-Rspamd-Queue-Id: 62A9C72D92 X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org; dkim=pass header.d=delphix.com header.s=google header.b=RGSyxpWy; dmarc=pass (policy=none) header.from=delphix.com; spf=pass (mx1.freebsd.org: domain of matthew.ahrens@delphix.com designates 2607:f8b0:4864:20::a29 as permitted sender) smtp.mailfrom=matthew.ahrens@delphix.com X-Spamd-Result: default: False [-6.59 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[delphix.com:s=google]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[zfs-devel@freebsd.org]; RCPT_COUNT_FIVE(0.00)[6]; NEURAL_HAM_SHORT(-0.99)[-0.989,0]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[delphix.com:+]; DMARC_POLICY_ALLOW(-0.50)[delphix.com,none]; RCVD_IN_DNSWL_NONE(0.00)[9.2.a.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; MX_GOOD(-0.01)[alt1.aspmx.l.google.com,aspmx.l.google.com,aspmx2.googlemail.com,alt2.aspmx.l.google.com,aspmx3.googlemail.com]; IP_SCORE(-2.89)[ip: (-9.34), ipnet: 2607:f8b0::/32(-2.88), asn: 15169(-2.15), country: US(-0.07)]; FORGED_SENDER(0.30)[mahrens@delphix.com,matthew.ahrens@delphix.com]; MIME_TRACE(0.00)[0:+,1:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[mahrens@delphix.com,matthew.ahrens@delphix.com]; RCVD_COUNT_TWO(0.00)[2] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 26 Mar 2019 16:30:02 -0000 The next OpenZFS Leadership meeting will be held today, March 26, 1pm-2pm Pacific time. Everyone is welcome to attend and participate, and we will try to keep the meeting on agenda and on time. The meetings will be held online via Zoom, and recorded and posted to the website and YouTube after the meeting. The agenda for the meeting will be a discussion of the projects listed in the agenda doc. For more information and details on how to attend, as well as notes and video from the previous meeting, please see the agenda document: https://docs.google.com/document/d/1w2jv2XVYFmBVvG1EGf-9A5HBVsjAYoLIFZAnWHhV-BM/edit --matt From owner-zfs-devel@freebsd.org Wed Mar 27 16:06:29 2019 Return-Path: Delivered-To: zfs-devel@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 994E51553D1E for ; Wed, 27 Mar 2019 16:06:28 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: from mail-vs1-xe43.google.com (mail-vs1-xe43.google.com [IPv6:2607:f8b0:4864:20::e43]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 3435E82627 for ; Wed, 27 Mar 2019 16:06:28 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: by mail-vs1-xe43.google.com with SMTP id g127so10244451vsd.6 for ; Wed, 27 Mar 2019 09:06:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=delphix.com; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to; bh=OvaVcE7PjXmeRhEWZrYBwxgsAlMgfGKbLjjK5/DMtIE=; b=d7UaS4XS4QePX88XAOKkvB4eqjawEkOxcvYgNVsY3P0Jtrxlrl1DkxTY3GVWfow9fv 07M7V4cq+rPmTeuOwHndWqs3KQKFWqdfhpgFYzTVbHXLtEw2LruQKqZHOJxGVAAuWNZf smfy7maJTfrxlMnUlsXA6uk/XgHJcAnbl4pL4= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=OvaVcE7PjXmeRhEWZrYBwxgsAlMgfGKbLjjK5/DMtIE=; b=Zs9USEMyivNXhqNM6wVkTYCqOVgwedojRj2YdK8nOX9MsoAXIHGyDWPLaGlBrYKBUb KlJ2oSl8akVb2yE3HeWsm3bmKrUIr0miVkxYKug2FHz3z4wZMOfXJnC5qVwfDZK/Mkn5 +YyLTHtEn4QqtIUhiY6G8pl9vor5d3CcL5vHLUPkB7wyA22iJc/LE8wF7wFsXscA2TT1 clOOWcXhUCSByZOf9KIdCS3k0jZawlqBXyQ6waWvckDS6dcYLAGTjZyrxXkwmBptfPHD WtUGuQzNhd05aWbFd5WbBPs3wwZCeds2X8jvUPBrTOnEIvXpmwHzzHNrUJBZAmlgN6eH jXjA== X-Gm-Message-State: APjAAAVSoIOH5naO2vpyS6u2YEloV2Wgv62fddcvAxKhplCIAJGjpUAi WlZC///keyU3ml8yCSWi5lVIIQd6Z1kwoMUe1xZ0cRlfVHj3mg== X-Google-Smtp-Source: APXvYqxydGHy4u3Ev3ngv4invBzORcRR5OoXlQr8fNUm7xs1YTcg+2nAUdnaUVvvbkaYzAEwBfeu2H8He9dW4dxvFV0= X-Received: by 2002:a67:ee91:: with SMTP id n17mr22848607vsp.56.1553702787350; Wed, 27 Mar 2019 09:06:27 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Matthew Ahrens Date: Wed, 27 Mar 2019 09:06:15 -0700 Message-ID: Subject: Re: March OpenZFS Leadership Meeting To: developer , illumos-zfs , zfs-devel@list.zfsonlinux.org, zfs-devel@freebsd.org, freebsd-fs , zfs-discuss X-Rspamd-Queue-Id: 3435E82627 X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-6.98 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; NEURAL_HAM_SHORT(-0.98)[-0.978,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; REPLY(-4.00)[] Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 27 Mar 2019 16:06:29 -0000 At yesterday's meeting we discussed: - log spacemap review - DRAID summit - default pool features - filesystem GUIDs. Video recording available: https://www.youtube.com/watch?v=3DtVhf4ZxfB1Y The next meeting will be April 23rd at 11AM Pacific time (2 hours earlier than usual). Detailed notes, thanks to Karyn: - Log spacemap is out for review (Serapheim) - Open PR for this work, including performance results - For people who want to port this feature to another platform, Serapheim also has a list of other changes required - DRAID summit (Richard Elling) - This work has been progressing, and it is at the point where it would be great to get people together in a conference room (physically and virtually) for a day to hash it all out. We would do this in the May/= June timeframe. - Please raise your hand (click yes in the participants window) if you want to participate. - Working on finding the venue. - Yes votes: Joyent, Mark Maybee, BrianB, Don Brady, Matt Ahrens, Tom Caputti. Please reach out to Richard if you are interested in participating. - Default pool features (Josh P) - Summary of the proposal - Portable, current, and what was available in 20XX + Tier 1 platforms - Tier 1 platforms: FreeBSD (11.X), ZoL (0.7.X), illumos-gate (from 1 year ago) - zpool status or upgrade would use the previously selected portability setting - Questions raised: - Ability to define the feature sets at runtime, so you can add new definitions on your own - Sef: Disagrees with the editable file. My own suggested addition would be the ability to query an existing pool against portability list. e.g., "How portable is this pool?" I don=E2=80=99t know = if that=E2=80=99s useful outside my head. - Tier 1 platform: specific ZoL version rather than a specific OS platform - Root pools: grub doesn=E2=80=99t support the same features as the = Tier 1 platforms. - Matt=E2=80=99s take: - User-configurable features may or may not be useful, but we can hold off and decide on that later. - ZoL: Which distros are shipping their own versions and how important are they? - Grub: We discussed this before, and we decided it was too hard and we were going to defer the conversation until later - Do we want to bias toward portability or the latest feature set? - Each distro ships different versions of ZoL: - RHEL has a separate repo for ZoL (0.7) - Debian ships 0.6.x - Matt: If you aren=E2=80=99t shipping your own version of ZoL= , you aren=E2=80=99t a Tier 1 platform? - Since portability isn=E2=80=99t turned on by default, maybe it=E2= =80=99s fine - You could say that only RHEL & SLES (+Ubuntu) are the platforms that matter since they are large, supported releases. Focus on main distros and release versions - Ubuntu LTS will be Tier 1 - Christian: Heard a rumor that Canonical is working on an installer that will include ZoL. Is that happening? What will that mean for portability? - Canonical are working on it, but we don=E2=80=99t have all the = details. - Jorgen: Considering that "upgrading" is one-way - you need to be conservative. At the moment, users want to dual boot Windows and Linux, and don't care they are missing "userobj quota" as it's not exactly going to make the pool faster. - JoshP: Boot pool is not nearly as important as data pools - No volunteers to lead the implementation effort. - Suggestion to have the discussion on the mailing list to make sure people are included. - Matt pointed out that the person leading the implementation gets to decide the details beyond what=E2=80=99s in the core requiremen= ts. (subject to code review) - Feature idea: GUIDs for filesystems that are invariant to zfs send | recv (Christian) - `guid` is publicly documented since https://github.com/zfsonlinux/zfs/pull/6102 - The `guid` property for snapshots is invariant to zfs send | recv - I use `guid` in zrepl to build diffs of the snapshot lists between =E2=80=9Cthe same=E2=80=9D filesystem on the sending & receiving side= . - However, filesystem identity is currently derived from the dataset path. The `guid` for a sent-recvd filesystem is different on the receiving side, hence not invariant. - Filesystem identity across pools / machines is required to reliably track renames & destroys of filesystems for purposes of replication. - Question: Do we want such a cross-pool `guid` for filesystems _in_ ZFS? - Previous work & ideas in that area? - How does a receiving pool handle an (unlikely) collision of GUIDs? - Do we have this problem for snapshots already? - Alternative: implement it as a user-property. - JoshC: How does -R work with this? - Matt: It is in userland, and it does work. When you send, it has the from guid and you send it to the new guid. You don=E2=80=99t necessarily know the name if it has been renamed, but you can find the snapshot that has the same guid as the from guid that the send stream is describing. You= can receive the same snapshot in one pool. The problem is solvable with the solution today. - Christian: Exhaustive search is good, but it can get tricky if there is divergence and deleted ... Wants reliable detection mecha= nism. - No clear conclusion on this, and Christian also doesn=E2=80=99t have = time to implement this on his own at this point. - Matt is open to someone implementing a guid for zrepl / other ZFS-specific functions. - For pjd=E2=80=99s need: Exclude these sub-filesystems or metada= ta. Use redacted send/receive (+ more stuff) On Tue, Mar 26, 2019 at 9:29 AM Matthew Ahrens wrote: > The next OpenZFS Leadership meeting will be held today, March 26, 1pm-2pm > Pacific time. > > Everyone is welcome to attend and participate, and we will try to keep th= e > meeting on agenda and on time. The meetings will be held online via Zoom= , > and recorded and posted to the website and YouTube after the meeting. > > The agenda for the meeting will be a discussion of the projects listed in > the agenda doc. > > For more information and details on how to attend, as well as notes and > video from the previous meeting, please see the agenda document: > > > https://docs.google.com/document/d/1w2jv2XVYFmBVvG1EGf-9A5HBVsjAYoLIFZAnW= HhV-BM/edit > > --matt > From owner-zfs-devel@freebsd.org Mon Apr 22 16:01:03 2019 Return-Path: Delivered-To: zfs-devel@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 81429159DFB5 for ; Mon, 22 Apr 2019 16:01:03 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: from mail-vs1-xe36.google.com (mail-vs1-xe36.google.com [IPv6:2607:f8b0:4864:20::e36]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 69A496AD83 for ; Mon, 22 Apr 2019 16:01:02 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: by mail-vs1-xe36.google.com with SMTP id f15so6482440vsk.9 for ; Mon, 22 Apr 2019 09:01:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=delphix.com; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to; bh=PwCPgH5ZMx2MP+Fp6eHFgeCpb28JsByjwbuf2BmhUv0=; b=YArPoDmCxdGGZHH3H+2AINEsdae9wpbczE5dvcwVSJrug4STq9NRoVqRwjvVEzjpfZ h1Ca+xe0IBdmPJlffeUGmqg4k3C3MUSGnQuzt7XielfZI5Ln+rF8Rc3FfOWX48sho097 Sf32CauU4vp8GN9dlQsc0wSNHj9bZeRTnNK3U= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=PwCPgH5ZMx2MP+Fp6eHFgeCpb28JsByjwbuf2BmhUv0=; b=Adhvujk9eSz2QIzt2inkxkf7ZQXlzRi0jcJca69cnaLRmMYgCHE3EEErgUeplCi7Zz H06vyn27d9wDEzttIeYo+fqtFBD56S8VZABM71rgV3gEAnFI0YhwupDNy+/yiMlW+6j/ iHk2LGsNoTo6misyEsVW+dS5A7GlUbOuA/PIYJ/3XFjBY0F6JBALydR7YXIKHNS2yr+f 8cOeCBr5cU30Q1SC2yJpkGvorxb8pFd7UK/6i7Qlnp35DWUwBwxLtaR2WOGFsDraY1Cj vsNVQSfIwHQvT4PzDHILRzgRtY85yC8VhZCwX5FNTz0rN9D3/yVhL7WGYs+1kgsydCv5 rjtQ== X-Gm-Message-State: APjAAAW1iaihdz868MBXlt8RUvImPjJWALLaupoQx7irUsOyXP6Bws3R NkozvP5iWG2uUTomzUZoHiSUnhowyeiWKRdZhb0QOA== X-Google-Smtp-Source: APXvYqzvWncdSonziFZuQ/F53mmd1dAFvd9UKtuYhJ3/vmCPQViRWPYWhiw//lMKwbd810bMmgDfXneF9qEU8b8JgOg= X-Received: by 2002:a67:f414:: with SMTP id p20mr10222198vsn.94.1555948861541; Mon, 22 Apr 2019 09:01:01 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Matthew Ahrens Date: Mon, 22 Apr 2019 09:00:50 -0700 Message-ID: Subject: April OpenZFS Leadership Meeting To: developer , illumos-zfs , zfs-devel@list.zfsonlinux.org, zfs-devel@freebsd.org, freebsd-fs , zfs-discuss X-Rspamd-Queue-Id: 69A496AD83 X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org; dkim=pass header.d=delphix.com header.s=google header.b=YArPoDmC; dmarc=pass (policy=none) header.from=delphix.com; spf=pass (mx1.freebsd.org: domain of matthew.ahrens@delphix.com designates 2607:f8b0:4864:20::e36 as permitted sender) smtp.mailfrom=matthew.ahrens@delphix.com X-Spamd-Result: default: False [-6.74 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[delphix.com:s=google]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[zfs-devel@freebsd.org]; RCPT_COUNT_FIVE(0.00)[6]; NEURAL_HAM_SHORT(-0.99)[-0.991,0]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[delphix.com:+]; DMARC_POLICY_ALLOW(-0.50)[delphix.com,none]; RCVD_IN_DNSWL_NONE(0.00)[6.3.e.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; MX_GOOD(-0.01)[alt1.aspmx.l.google.com,aspmx.l.google.com,aspmx2.googlemail.com,alt2.aspmx.l.google.com,aspmx3.googlemail.com]; IP_SCORE(-3.04)[ip: (-9.74), ipnet: 2607:f8b0::/32(-3.12), asn: 15169(-2.26), country: US(-0.06)]; FORGED_SENDER(0.30)[mahrens@delphix.com,matthew.ahrens@delphix.com]; MIME_TRACE(0.00)[0:+,1:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[mahrens@delphix.com,matthew.ahrens@delphix.com]; RCVD_COUNT_TWO(0.00)[2] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Apr 2019 16:01:03 -0000 The next OpenZFS Leadership meeting will be held tomorrow, April 23, 11am-12noon Pacific time (note the earlier than usual time). We don't have much on the agenda for this month, so if there are things you'd like to discuss, this is a good opportunity - send me an email :) Everyone is welcome to attend and participate, and we will try to keep the meeting on agenda and on time. The meetings will be held online via Zoom, and recorded and posted to the website and YouTube after the meeting. The agenda for the meeting will be a discussion of the projects listed in the agenda doc. For more information and details on how to attend, as well as notes and video from the previous meeting, please see the agenda document: https://docs.google.com/document/d/1w2jv2XVYFmBVvG1EGf-9A5HBVsjAYoLIFZAnWHhV-BM/edit --matt From owner-zfs-devel@freebsd.org Tue Apr 23 20:46:46 2019 Return-Path: Delivered-To: zfs-devel@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id CA3041582319 for ; Tue, 23 Apr 2019 20:46:45 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: from mail-vs1-xe44.google.com (mail-vs1-xe44.google.com [IPv6:2607:f8b0:4864:20::e44]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id DA9608A7F9 for ; Tue, 23 Apr 2019 20:46:44 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: by mail-vs1-xe44.google.com with SMTP id f22so9113958vso.7 for ; Tue, 23 Apr 2019 13:46:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=delphix.com; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to; bh=6kMzXauN1vB+STFulB++en3cGsmC90IjeO3Cg+GlucM=; b=RuPT/l8LbcWms1dRZuK3G7nwwR1dLEgnOoeV7bkbu2U96SRTejJklOwFcvSA2prilT fUUTznS844LuuAYJsj1ElQqOKd7lbY4owhbCRZctmmSkqzWsEBaUFbwOz8hCSZRDXxni WYWQVFCU+vq7qP4fVWR5/1QAEvyezsWlHNMxo= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=6kMzXauN1vB+STFulB++en3cGsmC90IjeO3Cg+GlucM=; b=P6Gklunez9R1vIGOZh2dFhhYO3YylrqRJUS8w8hGJjYqYQRUXbBUdjLyiHRluuatGo Ugscm3tNMUNq1iGgESg0PsbIP1F/yrbQ7683K87+8ZLSGTA/CRUrmilbHMrLUYexNosP tmKNnjyvrKRWXExPBYJeTy9R52bO1ecgcH58AZEhfXswgO3dWy1ALsVCZIdQhFSPym86 qVJgOailbdmHiST6zDwbMF8c76K2ljKOgeerTJjKOnyZ8PJf5ohsnZnPNjuPFjiJCCXl zqyXEA7cHT6xGelIaSdA3BDt+j95x67SKxTM80gS1mnNv6SQA1l8k83wza2xpDZVI4it SlWg== X-Gm-Message-State: APjAAAVI9pcl9xP3VU3xxiRW9l0hYO+SGy9duuBfp37vdgNoD/HPvvX7 QQyAbJwstWHeb9mIWvHQOroTSRb3dwV1MuWQB8lRaw== X-Google-Smtp-Source: APXvYqx0P8jSghzU1EmD6YRWZ13Ue4U+dBJCySQos9u+6jPDSl+w47EytxgAzVMDjeqGgHl8x6I3GrOwPPjB2q/JhqY= X-Received: by 2002:a67:b343:: with SMTP id b3mr14059848vsm.237.1556052404107; Tue, 23 Apr 2019 13:46:44 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Matthew Ahrens Date: Tue, 23 Apr 2019 13:46:32 -0700 Message-ID: Subject: Re: April OpenZFS Leadership Meeting To: developer , illumos-zfs , zfs-devel@list.zfsonlinux.org, zfs-devel@freebsd.org, freebsd-fs , zfs-discuss X-Rspamd-Queue-Id: DA9608A7F9 X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=delphix.com header.s=google header.b=RuPT/l8L; dmarc=pass (policy=none) header.from=delphix.com; spf=pass (mx1.freebsd.org: domain of matthew.ahrens@delphix.com designates 2607:f8b0:4864:20::e44 as permitted sender) smtp.mailfrom=matthew.ahrens@delphix.com X-Spamd-Result: default: False [-3.94 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[delphix.com:s=google]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[zfs-devel@freebsd.org]; RCPT_COUNT_FIVE(0.00)[6]; NEURAL_HAM_SHORT(-0.69)[-0.692,0]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[delphix.com:+]; MX_GOOD(-0.01)[cached: alt1.aspmx.l.google.com]; DMARC_POLICY_ALLOW(-0.50)[delphix.com,none]; RCVD_IN_DNSWL_NONE(0.00)[4.4.e.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; IP_SCORE(-0.54)[ip: (2.77), ipnet: 2607:f8b0::/32(-3.14), asn: 15169(-2.27), country: US(-0.06)]; FORGED_SENDER(0.30)[mahrens@delphix.com,matthew.ahrens@delphix.com]; MIME_TRACE(0.00)[0:+,1:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[mahrens@delphix.com,matthew.ahrens@delphix.com]; RCVD_COUNT_TWO(0.00)[2] Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Apr 2019 20:46:46 -0000 At today's meeting we covered: Discussions: - Making it easier to port commits from ZoL -> other platforms - Enabling compression by default - Updating LZ4 implementation Announcements: - DRAID meetup - ZoL 0.8 status - ZoF (ZoL on FreeBSD) test images - Code of Conduct Video recording available: https://www.youtube.com/watch?v=3DdIVGGJaZ7zk The next meeting will be May 28th at 1PM Pacific time (5 weeks from today). Detailed notes, thanks to Karyn and Serapheim: - DRAID meetup (Richard Elling) - This is an all-day workshop for people interested in using and developing the feature - The goal is to finalize the design and start working on it - May 3 at the Delphix SF office (343 Sansome St, Suite 900, San Francisco) at 9am - Please RSVP so we are sure to have enough space. Reach out to anyone on the team over email, Slack (#draid) - Karyn to set up Zoom - Richard will pull together an agenda document: remote folks (in particular) can add comments and items to the agenda - ZoL 0.8 status (Brian B) - Tagged final release candidate last week. FC and soaking since then. - Please test it as much as possible! - There are a few remaining bugs they are trying to finish up, but there shouldn=E2=80=99t be major changes. - Also working on documentation - Hoping to release in 2-3 weeks - ZoL =3D> FreeBSD: test images available (based on Dec ZoL)! See this announcement for details. Please test it out. - Allan will cross-post the ZoL on FreeBSD announcement to the openzfs list. - JoshP: The current ZoL-based implementation doesn't have TRIM enabled, are we dropping it? - Darth is waiting for ZoL to reach 0.8 and Delphix to upstream their latest features to ZoL before doing a final rebase for relea= se. Hopefully in 4-6 weeks. - This rebase should also include the TRIM changes. - Any feedback on Porting ZoL commits to other platforms? (Brian B) Are there things we can do to make this easier? Problems found during porting? - Igor: Identify pre-platform features, functions, etc. where things have diverged between platforms. How to report issues in a way that provides visibility to all of the platforms? - BB: They=E2=80=99ve tried to keep things consistent with illumos, = but have likely diverged. Getting some documentation from illumos and other folks would help. Source code? Wiki? Mailing list? - Jerry J: Has been documenting divergence in the SPL interfaces (that he noticed as he was porting from ZoL to illumos) in the and can send that out to the mailing list and we can figure out a better place to put i= t. - Enable compression by default at pool creation (Issue ) (kpande) The idea is to give a better out of the box experience. The downsides are: potential CPU cost (and write performance due to compressing) and new users misunderstanding how the space is used in certain cases. - Igor: There have been issues when root pools are expecting compression to be disabled (on SPARC?) - Allan: This has been the default in FreeBSD installer for ~2 years - Sef: Boot pools need to be different for each platform, but grub doesn=E2=80=99t seem like it needs to be different. Sef supports it b= eing the default. - Compression=3Don has been the default in FreeNAS for several years; n= o complaints - *** Someone to take an action item to investigate benefits and issues, and write it up? (Matt will put out a call for someone to tak= e ownership of this) - Next meeting: push back 1 week (to May 28)? - Done! - Code of Conduct - Send feedback by Apr 29! - Allan: Proposal to upgrade the LZ4? - In general people are in favor of this - The first step is to just update decompression so people get the performance improvements - Next is the exploration of enabling the compressor potentially with a tunable that can fall back to the old decompressor - The fear is that enabling the new compressor could increase the space usage if using nopwrite (e.g. double it for existing snapsho= ts) - Igor: How stable is LZ4+compression? - Allan: No known issues that he is aware of. On Mon, Apr 22, 2019 at 9:00 AM Matthew Ahrens wrote: > The next OpenZFS Leadership meeting will be held tomorrow, April 23, > 11am-12noon Pacific time (note the earlier than usual time). We don't ha= ve > much on the agenda for this month, so if there are things you'd like to > discuss, this is a good opportunity - send me an email :) > > Everyone is welcome to attend and participate, and we will try to keep th= e > meeting on agenda and on time. The meetings will be held online via Zoom= , > and recorded and posted to the website and YouTube after the meeting. > > The agenda for the meeting will be a discussion of the projects listed in > the agenda doc. > > For more information and details on how to attend, as well as notes and > video from the previous meeting, please see the agenda document: > > > https://docs.google.com/document/d/1w2jv2XVYFmBVvG1EGf-9A5HBVsjAYoLIFZAnW= HhV-BM/edit > > --matt > From owner-zfs-devel@freebsd.org Wed Apr 24 14:04:58 2019 Return-Path: Delivered-To: zfs-devel@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E23041599D19; Wed, 24 Apr 2019 14:04:57 +0000 (UTC) (envelope-from jimklimov@cos.ru) Received: from relay-mta.cos.ru (relay-mta.cos.ru [93.175.31.8]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 632718DFBC; Wed, 24 Apr 2019 14:04:56 +0000 (UTC) (envelope-from jimklimov@cos.ru) Received: from sunmail.cos.ru (mail [81.5.113.73]) by relay-mta.cos.ru (8.15.2+Sun/8.15.2) with ESMTP id x3OE5ibJ021835; Wed, 24 Apr 2019 17:05:50 +0300 (MSK) MIME-version: 1.0 Content-type: text/plain; charset=utf-8 Received: from [192.168.2.167] ([31.7.243.238]) by sunmail.cos.ru (Sun Java(tm) System Messaging Server 6.3-7.04 (built Sep 26 2008; 64bit)) with ESMTPSA id <0PQG007MRXYUID00@sunmail.cos.ru>; Wed, 24 Apr 2019 17:09:19 +0300 (MSK) Date: Wed, 24 Apr 2019 14:03:29 +0000 User-Agent: K-9 Mail for Android In-reply-to: References: Content-transfer-encoding: quoted-printable Subject: Re: [zfs] Re: April OpenZFS Leadership Meeting To: illumos-zfs , Matthew Ahrens , developer , zfs-devel@list.zfsonlinux.org, zfs-devel@freebsd.org, freebsd-fs , zfs-discuss From: Jim Klimov Message-id: X-Greylist-Inspected: inspected by milter-greylist-4.5.12-COS (relay-mta.cos.ru [93.175.31.8]); Wed, 24 Apr 2019 17:05:50 +0300 (MSK) for IP:'81.5.113.73' DOMAIN:'mail' HELO:'sunmail.cos.ru' FROM:'jimklimov@cos.ru' X-Greylist: Sender IP whitelisted, ACL 393 matched, not delayed by milter-greylist-4.5.12-COS (relay-mta.cos.ru [93.175.31.8]); Wed, 24 Apr 2019 17:05:50 +0300 (MSK) X-Rspamd-Queue-Id: 632718DFBC X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-6.98 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; NEURAL_HAM_SHORT(-0.98)[-0.984,0]; REPLY(-4.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0] X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Apr 2019 14:04:58 -0000 On April 23, 2019 8:46:32 PM UTC, Matthew Ahrens wr= ote: >At today's meeting we covered: > >Discussions: > > - Making it easier to port commits from ZoL -> other platforms > - Enabling compression by default > - Updating LZ4 implementation > >Announcements: > > - DRAID meetup > - ZoL 0=2E8 status > - ZoF (ZoL on FreeBSD) test images > - Code of Conduct > >Video recording available: https://www=2Eyoutube=2Ecom/watch?v=3DdIVGGJaZ= 7zk > >The next meeting will be May 28th at 1PM Pacific time (5 weeks from >today)=2E > >Detailed notes, thanks to Karyn and Serapheim: > > - > > DRAID meetup (Richard Elling) > - > > This is an all-day workshop for people interested in using and > developing the feature > - > > The goal is to finalize the design and start working on it > - > > May 3 at the Delphix SF office (343 Sansome St, Suite 900, San > Francisco) at 9am > - > > Please RSVP so we are sure to have enough space=2E Reach out to anyone > on the team over email, Slack (#draid) > - > > Karyn to set up Zoom > - > > Richard will pull together an agenda document: remote folks (in > particular) can add comments and items to the agenda > - > > ZoL 0=2E8 status (Brian B) > - > > Tagged final release candidate last week=2E FC and soaking since then= =2E > - > > Please test it as much as possible! > - > > There are a few remaining bugs they are trying to finish up, but > there shouldn=E2=80=99t be major changes=2E > - > > Also working on documentation > - > > Hoping to release in 2-3 weeks > - > > ZoL =3D> FreeBSD: test images available (based on Dec ZoL)! See this > announcement > > for details=2E Please test it out=2E > - > > Allan will cross-post the ZoL on FreeBSD announcement to the openzfs > list=2E > - > > JoshP: The current ZoL-based implementation doesn't have TRIM > enabled, are we dropping it? > - > > Darth is waiting for ZoL to reach 0=2E8 and Delphix to upstream > their latest features to ZoL before doing a final rebase for release=2E > Hopefully in 4-6 weeks=2E > - > > This rebase should also include the TRIM changes=2E > - > > Any feedback on Porting ZoL commits to other platforms? (Brian B) > >Are there things we can do to make this easier? Problems found during >porting? > > - > > Igor: Identify pre-platform features, functions, etc=2E where things > have diverged between platforms=2E How to report issues in a way that > provides visibility to all of the platforms? > - > > BB: They=E2=80=99ve tried to keep things consistent with illumos, bu= t have > likely diverged=2E Getting some documentation from illumos and >other folks > would help=2E Source code? Wiki? Mailing list? > - > > Jerry J: Has been documenting divergence in the SPL interfaces (that > he noticed as he was porting from ZoL to illumos) in the and >can send that >out to the mailing list and we can figure out a better place to put it=2E > - > > Enable compression by default at pool creation (Issue > ) (kpande) > >The idea is to give a better out of the box experience=2E The downsides >are: >potential CPU cost (and write performance due to compressing) and new >users >misunderstanding how the space is used in certain cases=2E > > - > > Igor: There have been issues when root pools are expecting > compression to be disabled (on SPARC?) > - > > Allan: This has been the default in FreeBSD installer for ~2 years > - > > Sef: Boot pools need to be different for each platform, but grub > doesn=E2=80=99t seem like it needs to be different=2E Sef supports it b= eing the > default=2E > - > > Compression=3Don has been the default in FreeNAS for several years; no > complaints > - > > *** Someone to take an action item to investigate benefits and > issues, and write it up? (Matt will put out a call for someone to take > ownership of this) > - > > Next meeting: push back 1 week (to May 28)? > - > > Done! > - > > Code of Conduct > - > > Send feedback by Apr 29! > - > > Allan: Proposal to upgrade the LZ4? > - > > In general people are in favor of this > - > > The first step is to just update decompression so people get the > performance improvements > - > > Next is the exploration of enabling the compressor potentially with a > tunable that can fall back to the old decompressor > - > > The fear is that enabling the new compressor could increase the > space usage if using nopwrite (e=2Eg=2E double it for existing snapshot= s) > - > > Igor: How stable is LZ4+compression? > - > > Allan: No known issues that he is aware of=2E > > >On Mon, Apr 22, 2019 at 9:00 AM Matthew Ahrens >wrote: > >> The next OpenZFS Leadership meeting will be held tomorrow, April 23, >> 11am-12noon Pacific time (note the earlier than usual time)=2E We >don't have >> much on the agenda for this month, so if there are things you'd like >to >> discuss, this is a good opportunity - send me an email :) >> >> Everyone is welcome to attend and participate, and we will try to >keep the >> meeting on agenda and on time=2E The meetings will be held online via >Zoom, >> and recorded and posted to the website and YouTube after the meeting=2E >> >> The agenda for the meeting will be a discussion of the projects >listed in >> the agenda doc=2E >> >> For more information and details on how to attend, as well as notes >and >> video from the previous meeting, please see the agenda document: >> >> >> >https://docs=2Egoogle=2Ecom/document/d/1w2jv2XVYFmBVvG1EGf-9A5HBVsjAYoLIF= ZAnWHhV-BM/edit >> >> --matt >> > >------------------------------------------ >illumos: illumos-zfs >Permalink: >https://illumos=2Etopicbox=2Ecom/groups/zfs/T70b8c262d29daaa6-Mfca8e62f8c= b12f220806d5cc >Delivery options: https://illumos=2Etopicbox=2Ecom/groups/zfs/subscriptio= n Hello all, For anecdotal experience regarding root pool compression, the issue was th= ere with older GRUB loader of OpenSolaris and contemporary Solaris 10; I be= lieve it was fixed in the illumos branch after the fork (can't find and poi= nt to a commit now, possibly soon after lz4 introduction; no idea about sta= te of later Solaris releases)=2E The problem manifested by GRUB refusing to read datasets (rpool, rpool/ROO= T, rpool/ROOT/$BENAME) that had the compression enabled (even if no blocks = were in fact compressed) or disabled at the moment but compressed blocks en= countered during a read=2E This was one of main reasons early on for my split-root support scripts, s= o /usr could be separated into a gzip-9'ed dataset and shrink from ~3gb to = ~1gb on a default install=2E On my current laptop I see rpool has compression=3Doff, rpool/ROOT has lz4= and rootfs(es) inherit that=2E Works well WRT stability and performance=2E Hope this helps, Jim -- Typos courtesy of K-9 Mail on my Android From owner-zfs-devel@freebsd.org Wed Apr 24 16:19:10 2019 Return-Path: Delivered-To: zfs-devel@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BE3AB159D103 for ; Wed, 24 Apr 2019 16:19:10 +0000 (UTC) (envelope-from igor@dilos.org) Received: from mail-lj1-x241.google.com (mail-lj1-x241.google.com [IPv6:2a00:1450:4864:20::241]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 3C1FC6D050 for ; Wed, 24 Apr 2019 16:19:10 +0000 (UTC) (envelope-from igor@dilos.org) Received: by mail-lj1-x241.google.com with SMTP id h16so4900687ljg.11 for ; Wed, 24 Apr 2019 09:19:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=dilos-org.20150623.gappssmtp.com; s=20150623; h=from:message-id:mime-version:subject:date:in-reply-to:cc:to :references; bh=YSnt6KM69tKHHnp7abX4olqHib9L22iEjbhCao4h9nQ=; b=r8cw2J8eFMtgfMw0F2HoVi0b1vRN5u28VvS8NvPto5S2fn5E4ft1OCznvbh3hbSBwn tzMnLn+esvZOD5AxPfu43vsGS3oZnq/Tu2MyPrqOA9bydmwedkW5A7O5cu1DEjqgog4+ fenJH8Zx4d9txIcbK9wS/8/i6NFzD2y9SRYQHyHaOevWDxt4K6K4sPU01AsE1y975Kk+ C0x50YIW+V/7YGRchu2hecRnlSXafnVZWE8Pagq5SnKCvx6gCOKE0gfjjWYLhOAL1VYB Im2iH9jKFOt9Xgsm053B2yDOu2+NfnXzm0989Kx0pUOn9Ydap3F/7plDW8PTUmlV2SkI Tr2A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:message-id:mime-version:subject:date :in-reply-to:cc:to:references; bh=YSnt6KM69tKHHnp7abX4olqHib9L22iEjbhCao4h9nQ=; b=ldYdnJmBzu5R2mG6tedMVJs4SSigXMzdmmbBZu/9E79vZD9MszWJB/LOkGavJCI9q9 pnULSZPhg8bxYDOOMMKP8pCVIxiBjeHW3Cc1wRtaDQ+zy+l48Zp6gVpYbLaG6QLbDGEd TM/dCEMAChiKKgSU9msRQleLAqx5rbSGmcRJ0ahcKCmD18Xd/3yGvxrL05l5VgUXML3C 4/gBQhNZJVO3yahypkYVfIkyQ4R/1r7b41Fhx5yaUPTNc2dagVd/R2nGbBP4IOk1AEJx bKZGqQ/ASQ8nxJMCL0a6S/+7zbUimRNJNDUSAoWb01mIKIhx0hwVEWxB3tbmoC47FV58 QxGg== X-Gm-Message-State: APjAAAX4vpF/oh3Z25sClRh6lU2JxUWyWtoefAnVIX+QoTHl+Q9Xj5uU NnJyUWA9XbevMAIDn6qLMJwEqw== X-Google-Smtp-Source: APXvYqy+f49XGRzVKGdUCqY/vd+ctJIkuEMnimIqAXYqBX2E9OJHk5R1owQg9imFWYz1ALfhqgO4tQ== X-Received: by 2002:a2e:8854:: with SMTP id z20mr7601510ljj.25.1556122748762; Wed, 24 Apr 2019 09:19:08 -0700 (PDT) Received: from [192.168.1.13] ([91.204.56.74]) by smtp.gmail.com with ESMTPSA id y65sm1279637lje.84.2019.04.24.09.19.07 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 24 Apr 2019 09:19:08 -0700 (PDT) From: Igor Kozhukhov Message-Id: <4C9A42C6-5024-4A87-A45C-A43A4562C3BA@dilos.org> Mime-Version: 1.0 (Mac OS X Mail 12.4 \(3445.104.8\)) Subject: Re: [developer] Re: [zfs] Re: April OpenZFS Leadership Meeting Date: Wed, 24 Apr 2019 19:19:07 +0300 In-Reply-To: Cc: illumos-zfs , Matthew Ahrens , developer , zfs-devel@list.zfsonlinux.org, zfs-devel@freebsd.org, freebsd-fs , zfs-discuss To: openzfs-developer References: X-Mailer: Apple Mail (2.3445.104.8) X-Rspamd-Queue-Id: 3C1FC6D050 X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-6.98 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; REPLY(-4.00)[]; NEURAL_HAM_SHORT(-0.98)[-0.983,0] Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Apr 2019 16:19:11 -0000 >=20 > For anecdotal experience regarding root pool compression, the issue = was there with older GRUB loader of OpenSolaris and contemporary Solaris = 10; I believe it was fixed in the illumos branch after the fork (can't = find and point to a commit now, possibly soon after lz4 introduction; no = idea about state of later Solaris releases). >=20 > The problem manifested by GRUB refusing to read datasets (rpool, = rpool/ROOT, rpool/ROOT/$BENAME) that had the compression enabled (even = if no blocks were in fact compressed) or disabled at the moment but = compressed blocks encountered during a read. >=20 > This was one of main reasons early on for my split-root support = scripts, so /usr could be separated into a gzip-9'ed dataset and shrink = from ~3gb to ~1gb on a default install. >=20 > On my current laptop I see rpool has compression=3Doff, rpool/ROOT has = lz4 and rootfs(es) inherit that. Works well WRT stability and = performance. it is known issue with compression=3Don on rpool, but it is know working = scenario for compression=3Don rpool/ROOT as dataset with root fs - it is = working fine on Intel and SPARC it is why i tried to explain to: do not use compression=3Don on pool = itself , but use it on dataset where root file system available =3D = rpool/ROOT and inherit to BE in rpool/ROOT/*. but my English is not good in this context. -Igor >=20 > Hope this helps, > Jim >=20 > -- > Typos courtesy of K-9 Mail on my Android >=20 > ------------------------------------------ > openzfs: openzfs-developer > Permalink: = https://openzfs.topicbox.com/groups/developer/Td7b3a91ecbeec509-M4044865ed= d7f8b2e0eac8460 = > Delivery options: = https://openzfs.topicbox.com/groups/developer/subscription = From owner-zfs-devel@freebsd.org Tue May 28 03:51:37 2019 Return-Path: Delivered-To: zfs-devel@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4D03715B4A9C for ; Tue, 28 May 2019 03:51:37 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: from mail-vs1-xe35.google.com (mail-vs1-xe35.google.com [IPv6:2607:f8b0:4864:20::e35]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 86AD094276 for ; Tue, 28 May 2019 03:51:35 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: by mail-vs1-xe35.google.com with SMTP id o5so8414197vsq.4 for ; Mon, 27 May 2019 20:51:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=delphix.com; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to; bh=j6dLzbzSmv1xswgetI4RuFTSP4AJ8FDPDu9uxn1MShQ=; b=akbh1Efi5PLmn3p/NU1K1+PcUtQ45uRB0Lu5R8xoAeSQnGrEdshDSa3qqvzJ0Y5vQC 6ckEIVeOb/snSV9e6jd5ykCd8LP6jDUAOT7zjOp0/7JNsidH9b+MyGgt9+eGUFa/YVZ6 JwLkDeruTGKBWbQZrorgQuYxcpjA9awM7rBGo= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=j6dLzbzSmv1xswgetI4RuFTSP4AJ8FDPDu9uxn1MShQ=; b=Ep4vLm6rX7fMfyR2O96W6sKg3pB3zeCrcrQ6S9ycNJeH07lGrmU3ySfAGv08jaR/GP MR0CPrtBNfsggGLWIVihm7IlbnNmKK7fC6A/hbgDLBUKCVvhvlZROoFeV5+qtZzBw8gU W0z08Fx23X01vdrMaN7S61o/uq9h0j3J3qxXFHQ5WX6a+V1OHrOWhqAH2DxupzCCf28X 8R0uOcE/e8U8mRGY6W+O9+Ee3CbxPp7xYkLLhnpZQ4bkjRcbUOTjdr/CluqKxhs2efwV xzNNeTwlAwWPoL3HJ1M6RqkWzKJqJjFQLR0YtmviUzS6/FwJwig6AtzXUP3jAQGMnsKN AX8Q== X-Gm-Message-State: APjAAAWjN638+Dwof4YyZbkmkl8gmmz42c9fF4/Zd+Nk5t/9rPIYByXC 41QvXPPe1rUCpCTr7Un4BjyARkaoOx/MIf+18N3Y+A== X-Google-Smtp-Source: APXvYqybK2e+M00Eycqd5e0y7+ajyb9ODM0ICC+WECVaKrVmtvFvd1nrU+BE+YRKPIZpq3KeeaeUK6u8lFjHaczLvY8= X-Received: by 2002:a05:6102:101a:: with SMTP id q26mr34443992vsp.56.1559015494248; Mon, 27 May 2019 20:51:34 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Matthew Ahrens Date: Mon, 27 May 2019 20:51:23 -0700 Message-ID: Subject: May OpenZFS Leadership Meeting To: developer , illumos-zfs , zfs-devel@list.zfsonlinux.org, zfs-devel@freebsd.org, freebsd-fs , zfs-discuss X-Rspamd-Queue-Id: 86AD094276 X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org; dkim=pass header.d=delphix.com header.s=google header.b=akbh1Efi; dmarc=pass (policy=none) header.from=delphix.com; spf=pass (mx1.freebsd.org: domain of matthew.ahrens@delphix.com designates 2607:f8b0:4864:20::e35 as permitted sender) smtp.mailfrom=matthew.ahrens@delphix.com X-Spamd-Result: default: False [-6.72 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[delphix.com:s=google]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[zfs-devel@freebsd.org]; RCPT_COUNT_FIVE(0.00)[6]; NEURAL_HAM_SHORT(-0.99)[-0.991,0]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[delphix.com:+]; DMARC_POLICY_ALLOW(-0.50)[delphix.com,none]; RCVD_IN_DNSWL_NONE(0.00)[5.3.e.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; MX_GOOD(-0.01)[alt1.aspmx.l.google.com,aspmx.l.google.com,aspmx2.googlemail.com,alt2.aspmx.l.google.com,aspmx3.googlemail.com]; IP_SCORE(-3.02)[ip: (-9.45), ipnet: 2607:f8b0::/32(-3.29), asn: 15169(-2.28), country: US(-0.06)]; FORGED_SENDER(0.30)[mahrens@delphix.com,matthew.ahrens@delphix.com]; MIME_TRACE(0.00)[0:+,1:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[mahrens@delphix.com,matthew.ahrens@delphix.com]; RCVD_COUNT_TWO(0.00)[2] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 28 May 2019 03:51:37 -0000 The next OpenZFS Leadership meeting will be held tomorrow, May 28, 1pm-2pm Pacific time. We don't have much on the agenda for this month, so if there are things you'd like to discuss, this is a good opportunity - send me an email or edit the doc :) Everyone is welcome to attend and participate, and we will try to keep the meeting on agenda and on time. The meetings will be held online via Zoom, and recorded and posted to the website and YouTube after the meeting. The agenda for the meeting will be a discussion of the projects listed in the agenda doc. For more information and details on how to attend, as well as notes and video from the previous meeting, please see the agenda document: https://docs.google.com/document/d/1w2jv2XVYFmBVvG1EGf-9A5HBVsjAYoLIFZAnWHhV-BM/edit --matt From owner-zfs-devel@freebsd.org Wed May 29 16:15:18 2019 Return-Path: Delivered-To: zfs-devel@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3C4C315A69D5 for ; Wed, 29 May 2019 16:15:18 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: from mail-ua1-x944.google.com (mail-ua1-x944.google.com [IPv6:2607:f8b0:4864:20::944]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id BAD7A73AD9 for ; Wed, 29 May 2019 16:15:15 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: by mail-ua1-x944.google.com with SMTP id a95so1180568uaa.13 for ; Wed, 29 May 2019 09:15:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=delphix.com; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to; bh=+Sli7tmUKZg4GrJjMgfcjwvKUsrVpU8AZtqnqo7EN4U=; b=JLahOFFm6ampEp4XQGLrXQ4hwImoZiBMBU8z08W4tehGNR4owkKv180sPoaJmIuuYY zszQynQ2xLSgSHzoVlbjYjnDndb8RgcufoTR/P4IkkoC9GpLZsTHH+y77q+vEUdVulg9 cgvSn6orsENCKoLcjFEI/npMzv+2iO6ZPwW2U= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=+Sli7tmUKZg4GrJjMgfcjwvKUsrVpU8AZtqnqo7EN4U=; b=NkcfD7969PsTs1j183jCv/rjnGUbC3pqvik+xsN2Lhu3O2DNHwoWyBA7pMZM6XRK6N wZLIW3La7MZprdaI7C8D8T6S+Oaw+k7YNRrVu9YQEdBbB5QqJxfvaP1zyX8zLxxO/dWY oQq7bEzfmB7m/v1SUwwnlKco4gWmivpwEnKDn4PLl4ia7PIY1c9St0U2VDZmn2CdQcER THaFw28Q/Pnzo0hE99IyR0OS/uFSYfxfWOW0vT/lU6F1KPV5nPNDI5u2bH/s9mItBUsw sndoCk1LVngslm+TSE2pVt1Cy4zw0AWXP8SzT4AW7vU9sU3jFRGUgIkDk2SZFyBrsBoi u0qA== X-Gm-Message-State: APjAAAVcbfgZnEcDFl+wl+rFteU1NIGP6ouSt//PvL363/TKYMKWsYgX iaHWG6mBguGx8WAOs+YogJ9hwJ3YbJqadmV7GDVJgGlelUI= X-Google-Smtp-Source: APXvYqzI2Tg9Vc4kPvg0HuA+9cRIsVbdutM3bGOEMHEjvLrRZZFa4OjdfaHYuSrGE+UHZJBO74d8IixD+AP0AdMlzp8= X-Received: by 2002:ab0:7346:: with SMTP id k6mr44237053uap.89.1559146513867; Wed, 29 May 2019 09:15:13 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Matthew Ahrens Date: Wed, 29 May 2019 09:15:02 -0700 Message-ID: Subject: Re: May OpenZFS Leadership Meeting To: developer , illumos-zfs , zfs-devel@list.zfsonlinux.org, zfs-devel@freebsd.org, freebsd-fs , zfs-discuss X-Rspamd-Queue-Id: BAD7A73AD9 X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=delphix.com header.s=google header.b=JLahOFFm; dmarc=pass (policy=none) header.from=delphix.com; spf=pass (mx1.freebsd.org: domain of matthew.ahrens@delphix.com designates 2607:f8b0:4864:20::944 as permitted sender) smtp.mailfrom=matthew.ahrens@delphix.com X-Spamd-Result: default: False [-3.15 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.998,0]; R_DKIM_ALLOW(-0.20)[delphix.com:s=google]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[zfs-devel@freebsd.org]; URI_COUNT_ODD(1.00)[17]; RCPT_COUNT_FIVE(0.00)[6]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[delphix.com:+]; DMARC_POLICY_ALLOW(-0.50)[delphix.com,none]; RCVD_IN_DNSWL_NONE(0.00)[4.4.9.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; MX_GOOD(-0.01)[alt1.aspmx.l.google.com,aspmx.l.google.com,aspmx2.googlemail.com,alt2.aspmx.l.google.com,aspmx3.googlemail.com]; NEURAL_HAM_SHORT(-0.85)[-0.854,0]; IP_SCORE(-0.59)[ip: (2.69), ipnet: 2607:f8b0::/32(-3.30), asn: 15169(-2.29), country: US(-0.06)]; FORGED_SENDER(0.30)[mahrens@delphix.com,matthew.ahrens@delphix.com]; MIME_TRACE(0.00)[0:+,1:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[mahrens@delphix.com,matthew.ahrens@delphix.com]; RCVD_COUNT_TWO(0.00)[2] Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 29 May 2019 16:15:18 -0000 Thanks everyone who participated in yesterday's meeting. The video recording is now up: https://youtu.be/EzyPr3BuePo The next meeting will be June 25, at 1pm pacific time. The meeting notes follow: - Happy ZoL 0.8! - Lots of downloads and excited people. Successful so far! - Illumos encryption status (Jerry/Jason/J=C3=B6rgen) - Jerry has all of J=C3=B6rgen=E2=80=99s work pulled over and building = on illumos. Resolving some test failures and then will put it out for an illumos = code review. - The code review will go out to the illumos developer mailing list and ask that they identify issues with the platform-dependent changes in = the port. People will have ample time to comment and raise issues before = the RTI is submitted. - RAIDZ Expansion status (Matt) - Will have an Alpha out this week or next. Pools will need to be destroyed if you use it. - The change lets you add a device to an existing pool and reflows the data between disks. - All of the caveats and details of what does and doesn=E2=80=99t work = will be included in the PR. - Call for testing! Once the code is up, any manual testing under different workloads and setups will be greatly appreciated. It would = also be great if people could help write systemic tests. - BSDCAN update - The future of ZFS on FreeBSD (Allan) - One of the goals of the talk was to motivate the ZFS repo of FreeBSD rebasing over ZoL. The advantages of having the consumers of ZFS closer together was highlighted. - There was some initial pushback people thought that the rebase will make ZFS use the Linux Compatibility layer of FreeBSD, which is not the case. Once this was clear, there were no further pushbacks - Matt brought up that once the rebase is done and stable, discussions may be initiated for potentially renaming the ZoL repo= . - snapshot scalability (Matt) - Slides are posted and video will be up eventually. Please check them out if you=E2=80=99re interested in learning more about snaps= hots. - Matt will post a link to #openzfs once he has one. - DRAID meeting recap (Richard Elling) - There were about 11 participants - There is a design doc (linked above) with all the history of its edits based on the discussions that took place. Please add comments to the doc to further the discussion. - There is still work to be done but since the code changes are sizeable, we=E2=80=99ll try to split them into individual commits whe= n it makes sense. - Sef says he is available to do reviews at any time, just reach out= . - CoC update (Matt/Karyn) - Code of Conduct was put into place since the last meeting. We made several changes based on community feedback. Thanks for participating= ! - We will have a call for nominees for the Working Group in the fall, and then will review the CoC at that point and have another community comment period. The goal is to complete this before the Dev Summit. - The Dev Summit is Nov 4-5 in SF. See open-zfs.org for upcoming dates (e.g., call for presentations). - Registration opens July 8th - Send presentations to Matt by August 19th - Thanks to Datto, Delphix, and Intel for already sponsoring! Reach out to @kritter if you want to sponsor and she can give you the details. - Request from sef! The On-Disk Format document is out of date, and doesn=E2=80=99t reflect recent changes (e.g., encryption). It would = be really helpful to update this and keep it up-to-date going forward (Perhaps in = a semi-automated way?) - Tom: The info about how things look today exists in various places, so it is just a question of pulling it together into a single place. - Matt: We only have a pdf for that, and we probably can=E2=80=99t just copy-paste it somewhere else. It would be good to have a document with this info. Let=E2=80=99s put out a call for a volunteer to do this. - Paul pointed out that it is licensed under a Berkeley License - Compressed arc performance issue - single threaded sequential read - Sequential reads are decompressed by the (one) user=E2=80=99s thread. - With non-compressed ARC, they are decompressed by the (several) zio threads, leading to more parallelism and improved performance. - This is a good reason to disable compressed ARC (for workloads with single threaded sequential read). - We could make the prefetches decompress into the dbuf cache, to get this performance back. - If we do that, could we then make compressed ARC mandatory? - Maybe=E2=80=A6 but Allan may have found another way to address the= problem that motivated him to make compressed ARC mandatory - Please take a look at the redacted send-receive requests as soon as you get a chance so we can push this now that 0.8 is out. It=E2=80=99s in sy= nc with 0.8. - Next meeting will be in 4 weeks: June 25 On Mon, May 27, 2019 at 8:51 PM Matthew Ahrens wrote: > The next OpenZFS Leadership meeting will be held tomorrow, May 28, > 1pm-2pm Pacific time. We don't have much on the agenda for this month, s= o > if there are things you'd like to discuss, this is a good opportunity - > send me an email or edit the doc :) > > Everyone is welcome to attend and participate, and we will try to keep th= e > meeting on agenda and on time. The meetings will be held online via Zoom= , > and recorded and posted to the website and YouTube after the meeting. > > The agenda for the meeting will be a discussion of the projects listed in > the agenda doc. > > For more information and details on how to attend, as well as notes and > video from the previous meeting, please see the agenda document: > > > https://docs.google.com/document/d/1w2jv2XVYFmBVvG1EGf-9A5HBVsjAYoLIFZAnW= HhV-BM/edit > > --matt > From owner-zfs-devel@freebsd.org Tue Jun 25 16:56:55 2019 Return-Path: Delivered-To: zfs-devel@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D03BE15D1505 for ; Tue, 25 Jun 2019 16:56:54 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: from mail-vs1-xe2f.google.com (mail-vs1-xe2f.google.com [IPv6:2607:f8b0:4864:20::e2f]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id AA4406C081 for ; Tue, 25 Jun 2019 16:56:53 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: by mail-vs1-xe2f.google.com with SMTP id k9so11377562vso.5 for ; Tue, 25 Jun 2019 09:56:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=delphix.com; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to; bh=bys3TCvTD+6qPsYPe4r2BIrF+9JFY4PsQKGaEMtDNWQ=; b=Vwk+tTknWe60Ik7Pa6s1yK/WVDYFiHXnTzvqimsyOe42rY6c9/SCAziIkB1ROjm8nZ NdABqLNjD5Ob7nSA1zChJOZ6IuCE7p2a3LngWtPdZJthHo/hUO7PKP9c7+FBvLMq+rTT M6zbp90+afyltBpX99APtfJmff8leaK8oEhGY= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=bys3TCvTD+6qPsYPe4r2BIrF+9JFY4PsQKGaEMtDNWQ=; b=nYyBLzCVuthXdHFF6XX88u5avmon6gSWydwlMFvvW3sjr5t3Lnsqvetc5OwqMaNy3O Ki//Tg1wWTMVZ+kerqt5QcuBFdLsKuxd4t+g59H3FtkDZJWlS6A7dQq+uvLPSFTmbYpm HrA+vm3S2M0rAeq3A4ruIslQKbbFIBeosHsOvz3ByjuaLzqlOfb1N+bEGfnJ1HxMA5GV g5qVBS/pl5Vc8ifhk2dUHZ1uitxRtVjFNA6SZjozewIqPgpbwn+MI/gvPEYEpPT5CHUT KFiHO8qtxVaBkuMorTRBM956Yy0Kmr5SeL9VezfzncA0+221RcwNnrLuur/MVb0aNrXL y6qw== X-Gm-Message-State: APjAAAXdz+fVmCx9n85VxwjhJ0ma7DQMm8/HFWAQgcaKJNNA96xloDN/ H6/EE8e6alVmddJ7AfUfXadbEE8Ze3qRdp8oFU+RUQ== X-Google-Smtp-Source: APXvYqzx8MQo6r+4GQmy0wHL+p+OtswX8j0ZJKBIqjI0qO4eMTocDzuHIaSXbmkV44nH84AIJ98dJ2f9yru0gvQ1Rew= X-Received: by 2002:a67:7cd0:: with SMTP id x199mr79469366vsc.233.1561481812783; Tue, 25 Jun 2019 09:56:52 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Matthew Ahrens Date: Tue, 25 Jun 2019 09:56:42 -0700 Message-ID: Subject: June OpenZFS Leadership Meeting To: developer , illumos-zfs , zfs-devel@list.zfsonlinux.org, zfs-devel@freebsd.org, freebsd-fs , zfs-discuss X-Rspamd-Queue-Id: AA4406C081 X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org; dkim=pass header.d=delphix.com header.s=google header.b=Vwk+tTkn; dmarc=pass (policy=none) header.from=delphix.com; spf=pass (mx1.freebsd.org: domain of matthew.ahrens@delphix.com designates 2607:f8b0:4864:20::e2f as permitted sender) smtp.mailfrom=matthew.ahrens@delphix.com X-Spamd-Result: default: False [-6.75 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[delphix.com:s=google]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[zfs-devel@freebsd.org]; RCPT_COUNT_FIVE(0.00)[6]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[delphix.com:+]; MX_GOOD(-0.01)[alt1.aspmx.l.google.com,aspmx.l.google.com,aspmx2.googlemail.com,alt2.aspmx.l.google.com,aspmx3.googlemail.com]; RCVD_IN_DNSWL_NONE(0.00)[f.2.e.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; NEURAL_HAM_SHORT(-0.99)[-0.994,0]; DMARC_POLICY_ALLOW(-0.50)[delphix.com,none]; IP_SCORE(-3.05)[ip: (-9.72), ipnet: 2607:f8b0::/32(-3.14), asn: 15169(-2.33), country: US(-0.06)]; FORGED_SENDER(0.30)[mahrens@delphix.com,matthew.ahrens@delphix.com]; MIME_TRACE(0.00)[0:+,1:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[mahrens@delphix.com,matthew.ahrens@delphix.com]; RCVD_COUNT_TWO(0.00)[2] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Jun 2019 16:56:55 -0000 The next OpenZFS Leadership meeting will be held today, June 25, 1pm-2pm Pacific time. We don't have much on the agenda for this month, so if there are things you'd like to discuss, this is a good opportunity - send me an email or edit the doc :) Everyone is welcome to attend and participate, and we will try to keep the meeting on agenda and on time. The meetings will be held online via Zoom, and recorded and posted to the website and YouTube after the meeting. The agenda for the meeting will be a discussion of the projects listed in the agenda doc. For more information and details on how to attend, as well as notes and video from the previous meeting, please see the agenda document: https://docs.google.com/document/d/1w2jv2XVYFmBVvG1EGf-9A5HBVsjAYoLIFZAnWHhV-BM/edit --matt From owner-zfs-devel@freebsd.org Thu Jun 27 21:21:57 2019 Return-Path: Delivered-To: zfs-devel@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B0B6515CEA6B for ; Thu, 27 Jun 2019 21:21:56 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: from mail-vk1-xa44.google.com (mail-vk1-xa44.google.com [IPv6:2607:f8b0:4864:20::a44]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id DCA5A71F24 for ; Thu, 27 Jun 2019 21:21:53 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: by mail-vk1-xa44.google.com with SMTP id w186so786970vkd.11 for ; Thu, 27 Jun 2019 14:21:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=delphix.com; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to; bh=KW1lbs4zNcrSNYrRG8DiYjM2apt/4GbfijLQQaHa+YU=; b=YOn93TjZOaO+lbcSI9iAQQ+cOFHJxcYfjpva9ji/tA29aTMl22I5aqbw3CJd9r21LY sax8tz9MgO0U7+tE2+eEQbUvnjyCcR1oQ4M/Gv1OJsvRxb5K09qHcnDPMzVjJzZTuokw KKceoeidq+nVLaB1WzN2beF8h37zYQhm8x3S0= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=KW1lbs4zNcrSNYrRG8DiYjM2apt/4GbfijLQQaHa+YU=; b=hgV5FeTnNz68+CkYRGsyWGGlzPNPp9hkiVkuuRUabsegc5pSjTFEU2T+UBNxZk48tM VFMO8vqjHJv6+Z+UcBlQFZrNuk2ufm/J90e8niSMI9fC/vUunSuV/vO/80sLnIr4Ki6r +mzehUds2l0xx3rQcAPmcDm4Vqc2vdDVqyxrAM4RzrS6Zu5Coj9lI9WWR6bXAqMeNB4d isUfHHT4rG/BGD3/Kb3TsIDe8xon9LDZ1M3PXtZhydDOCl2H3t4lpRx8FQQMGjd3J2Vg awAYxb193zCyeGiWvGBrH1XKK5GkvkR+KcYkDxzmof9dgTakezUIA3isZ7VB+IHV4VV3 E8Pw== X-Gm-Message-State: APjAAAVfJSKIul3n7XPqWnKDDHELdW0wgfM2Lzf/Zb9bgXyxMihf5gT9 WKoTznRCfX80F4h8Xvy05Ysql3qm2fpi/ektvELL1ZIZ7C6vAw== X-Google-Smtp-Source: APXvYqzBHU4zu7LtUD+y+4FcY2swOoFHcmQeCUyiXV60uobjGYxgxXN6M2eAZYOr/aA/LcpToyYpLyayrwIhsTNs3tg= X-Received: by 2002:a1f:be51:: with SMTP id o78mr2445158vkf.66.1561670513025; Thu, 27 Jun 2019 14:21:53 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Matthew Ahrens Date: Thu, 27 Jun 2019 14:21:41 -0700 Message-ID: Subject: Re: June OpenZFS Leadership Meeting To: developer , illumos-zfs , zfs-devel@list.zfsonlinux.org, zfs-devel@freebsd.org, freebsd-fs , zfs-discuss X-Rspamd-Queue-Id: DCA5A71F24 X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=delphix.com header.s=google header.b=YOn93TjZ; dmarc=pass (policy=none) header.from=delphix.com; spf=pass (mx1.freebsd.org: domain of matthew.ahrens@delphix.com designates 2607:f8b0:4864:20::a44 as permitted sender) smtp.mailfrom=matthew.ahrens@delphix.com X-Spamd-Result: default: False [-3.88 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[delphix.com:s=google]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[zfs-devel@freebsd.org]; RCPT_COUNT_FIVE(0.00)[6]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[delphix.com:+]; MX_GOOD(-0.01)[alt1.aspmx.l.google.com,aspmx.l.google.com,aspmx2.googlemail.com,alt2.aspmx.l.google.com,aspmx3.googlemail.com]; RCVD_IN_DNSWL_NONE(0.00)[4.4.a.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; NEURAL_HAM_SHORT(-0.58)[-0.578,0]; DMARC_POLICY_ALLOW(-0.50)[delphix.com,none]; IP_SCORE(-0.59)[ip: (2.57), ipnet: 2607:f8b0::/32(-3.13), asn: 15169(-2.33), country: US(-0.06)]; FORGED_SENDER(0.30)[mahrens@delphix.com,matthew.ahrens@delphix.com]; MIME_TRACE(0.00)[0:+,1:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[mahrens@delphix.com,matthew.ahrens@delphix.com]; RCVD_COUNT_TWO(0.00)[2] Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Jun 2019 21:21:57 -0000 At this week's meeting we discussed FreeBSD/ZoL integration; error reporting infrastructure; and using pyzfs to get zpool config information. The video recording is now posted: https://www.youtube.com/watch?v=3DTJwykiJmH0M The next meeting will be at 11am US/Pacific on July 23rd. Full notes below: - FreeBSD / ZoL integration update - Goal: Single repo that builds for both platforms - https://github.com/zfsonfreebsd/ZoF/tree/projects/zfsbsd/module - Goal is for FreeBSD 13 to use ZoF by default (but may slip to FreeBSD 14) - Goal to get it into current repo around end of September? - Jorgen: Where should I direct Windows/OSX questions? (#openzfs or ZoL Developer list) - Do a PR for additional platforms/tree for additional platforms - Change zfsonlinux/zfs =3D> openzfs/zfs. Brian & Matt will come up wit= h a proposal for naming and structure that reflects what developers do/= use - Announcements of big, long-standing projects being available: - Linux - Redacted send/recv merged! - Illumos - ZFS crypto landed! - Discuss error reporting (Tom) - Do we want to move to something more general (e.g., have the kernel return strings)? - Continue to extend zfs_errno_t, though this doesn=E2=80=99t help prov= ide additional context/information, and then add new strings for user - pyzfs PR (Richard E) - Specifically how to get the pool config info programmatically - Programmatic =E2=80=9Czpool status=E2=80=9D that is retrievable, and = any time you add something to zpool you need to make sure status continues to be retrievable. (some mistake that Joshua pointed out...) - Translation of the config to a stable nvlist? Not sure if this should be in libzfs_core or the kernel. Kernel is better but might ignore backward compatibility. - Newer version of userland continue to work without rebooting the kernel. - Could a spanning mode option be added for multi-vdev zpools instead of the default striping data across all vdevs? The intent is to prioritize data integrity over access speed for large archival / backup pools, and = a workaround with multiple pools joined with mergerfs seems suboptimal if = the LOE of adding support for this is low. (Guirara DaiKaiju) - Matt: I think you=E2=80=99re suggesting that `zpool create disk1 disk= 2 =E2=80=A6` create the pool as a mirror, as though you typed `zpool create mirror disk1 disk 2 =E2=80=A6`. And perhaps that `zpool add=E2=80=A6` should inst= ead do `zpool attach =E2=80=A6`. On Tue, Jun 25, 2019 at 9:56 AM Matthew Ahrens wrote: > The next OpenZFS Leadership meeting will be held today, June 25, 1pm-2pm > Pacific time. We don't have much on the agenda for this month, so if the= re > are things you'd like to discuss, this is a good opportunity - send me an > email or edit the doc :) > > Everyone is welcome to attend and participate, and we will try to keep th= e > meeting on agenda and on time. The meetings will be held online via Zoom= , > and recorded and posted to the website and YouTube after the meeting. > > The agenda for the meeting will be a discussion of the projects listed in > the agenda doc. > > For more information and details on how to attend, as well as notes and > video from the previous meeting, please see the agenda document: > > > https://docs.google.com/document/d/1w2jv2XVYFmBVvG1EGf-9A5HBVsjAYoLIFZAnW= HhV-BM/edit > > > > --matt > From owner-zfs-devel@freebsd.org Tue Jul 16 21:51:45 2019 Return-Path: Delivered-To: zfs-devel@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id A8E53BB96D for ; Tue, 16 Jul 2019 21:51:45 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: from mail-vs1-xe36.google.com (mail-vs1-xe36.google.com [IPv6:2607:f8b0:4864:20::e36]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4916077093 for ; Tue, 16 Jul 2019 21:51:44 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: by mail-vs1-xe36.google.com with SMTP id h28so14986926vsl.12 for ; Tue, 16 Jul 2019 14:51:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=delphix.com; s=google; h=mime-version:from:date:message-id:subject:to; bh=NPfFSyjsEDHMgSnxcUA2wIA2eWITzQhxzd997Wrd6sI=; b=SjeH6seH2tkoIglCPJm1fMGbDCl7+1Eomt55c7n3biFYERxpHu1YS7ZyUQTVTW30M1 liL/44siESc8x2GgQR1RaI7HbRywEFElExA13nos/lPt0xN4wp228sK6bvp9Oi7B2fh6 zwx2WI9Jr/y2UKjWujeG6Xp+ADPfJELcQB4VA= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=NPfFSyjsEDHMgSnxcUA2wIA2eWITzQhxzd997Wrd6sI=; b=DDxou2CdOCLQ2RfD98jKqaChBVfoPNZ2DHZGJELzvjhMdw6GCzHXEgOVfKl7Kj6xzq IREhfEn4UPmOdfBYK312ojbxrsqg03KuM9Dwibt+UvexJvnk0ajNolf9GexKhceP/4x1 +/Bhlb9KCAeNVE2kxaOCtdjjArXk3I/APZ6NRVBu2WHJH9Bd4+Uw4aSIUz187VUxTzLb IR+FbqfKMMrgVPBdROEW90foEAcPTpYP/m/+E8JJaoyXtSbLNAAiGqhGRXnJAa60ppaJ G26cdNT0moZnzRFAJlrwe3vVTVAdkA8QbZ/zs7ZpvkxzFZiL9Qvoxk89LL9QPb21jSn0 XSRA== X-Gm-Message-State: APjAAAViHoInrvCNa3v9YZ57HcFByPN9iKBGvkkTZNB+7KNhLd757yVe r5Th+veKWc5CtTxzqRYdrbrEa93z0EuKYpUAXAO9qw== X-Google-Smtp-Source: APXvYqyYtzkuBv9ZvyjfwLS3rup6tgdUc+srxsFChWtT5kdGoIDvSuAiWgCFr/DJTBNHtEivLg1/QEXzvP9z9VhfI0Q= X-Received: by 2002:a67:eb94:: with SMTP id e20mr22904707vso.68.1563313903456; Tue, 16 Jul 2019 14:51:43 -0700 (PDT) MIME-Version: 1.0 From: Matthew Ahrens Date: Tue, 16 Jul 2019 14:51:31 -0700 Message-ID: Subject: OpenZFS Developer Summit 2019 - announcement & CFP To: developer , illumos-zfs , zfs-devel@list.zfsonlinux.org, zfs-devel@freebsd.org, freebsd-fs , zfs-discuss X-Rspamd-Queue-Id: 4916077093 X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org; dkim=pass header.d=delphix.com header.s=google header.b=SjeH6seH; dmarc=pass (policy=none) header.from=delphix.com; spf=pass (mx1.freebsd.org: domain of matthew.ahrens@delphix.com designates 2607:f8b0:4864:20::e36 as permitted sender) smtp.mailfrom=matthew.ahrens@delphix.com X-Spamd-Result: default: False [-6.60 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.99)[-0.993,0]; R_DKIM_ALLOW(-0.20)[delphix.com:s=google]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[zfs-devel@freebsd.org]; RCPT_COUNT_FIVE(0.00)[6]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[delphix.com:+]; MX_GOOD(-0.01)[alt1.aspmx.l.google.com,aspmx.l.google.com,aspmx2.googlemail.com,alt2.aspmx.l.google.com,aspmx3.googlemail.com]; RCVD_IN_DNSWL_NONE(0.00)[6.3.e.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; NEURAL_HAM_SHORT(-0.82)[-0.821,0]; DMARC_POLICY_ALLOW(-0.50)[delphix.com,none]; IP_SCORE(-3.07)[ip: (-9.70), ipnet: 2607:f8b0::/32(-3.17), asn: 15169(-2.45), country: US(-0.06)]; FORGED_SENDER(0.30)[mahrens@delphix.com,matthew.ahrens@delphix.com]; MIME_TRACE(0.00)[0:+,1:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[mahrens@delphix.com,matthew.ahrens@delphix.com]; RCVD_COUNT_TWO(0.00)[2] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Jul 2019 21:51:45 -0000 The sixth annual OpenZFS Developer Summit will be held in San Francisco, *November 4-5, 2019*. All OpenZFS developers are invited to participate, and registration is now open. See http://www.open-zfs.org/wiki/OpenZFS_Developer_Summit for all of the details, including slides and videos from previous years' conferences. The goal of the event is to foster cross-community discussions of OpenZFS work and to make progress on some of the projects we have proposed. The first day of the conference will be set aside for presentations. *If you would like to give a talk, submit a proposal via email to matt@mahrens.org , including a 1-2 paragraph abstract.* We welcome proposals for any kinds of talks related to ZFS, and we are particularly interested in fostering a diverse group of speakers. In the past, the most popular talks have featured internal details of new or upcoming features in OpenZFS, and explanations of how existing subsystems work. If you're new to the OpenZFS community or new to giving talks at conferences, mentorship is available to help make your presentation the best it can be. The registration fee will be waived for speakers. The deadlines are as follows: *August 19, 2019 All abstracts/proposals submitted* to matt@mahrens.org September 6, 2019 Proposal submitters notified November 4-5, 2019 OpenZFS Developer Summit The second day of the event will be a hackathon, where you will have the opportunity to work with OpenZFS developers that you might not normally sit next to, with the goal of having something, no matter how insignificant, to demo at the end of the day. This event is only possible because of the efforts and contributions of the community of developers and companies that sponsor the event. Special thanks to our early Diamond sponsors: Delphix , Datto , and Intel . Additional sponsorship opportunities are available. Please see the website for details and send email to Karyn.Ritter@delphix.com if you have any questions. --matt From owner-zfs-devel@freebsd.org Mon Jul 22 20:03:14 2019 Return-Path: Delivered-To: zfs-devel@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 7B8F2BAA75 for ; Mon, 22 Jul 2019 20:03:14 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: from mail-vs1-xe30.google.com (mail-vs1-xe30.google.com [IPv6:2607:f8b0:4864:20::e30]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 2946688A66 for ; Mon, 22 Jul 2019 20:03:13 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: by mail-vs1-xe30.google.com with SMTP id r3so27150231vsr.13 for ; Mon, 22 Jul 2019 13:03:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=delphix.com; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to; bh=RuLFhJ4HB9BYC4XWCeMABloAwTL0AnF1dHUV9l/oDd0=; b=Ew2nWKhbxcgXockNLWnTQPeqH/ZaMUrFPZsCDxdc0IC9ECBbfTtFUAQkjC4CRlfqCx nTVcIxJx+nLo++Ut7ZtlgOB+cneahrA0qndRww4WUb/HQxnqzPK0QDErmIk1NuLPlXVe as+boKu17eMJ1H/Q5hB/AnQy3SxRIbDLUrXS4= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=RuLFhJ4HB9BYC4XWCeMABloAwTL0AnF1dHUV9l/oDd0=; b=iCuqx6iLsM/fUSk+qp/k2Vc+bYrP4BCkcYK0FSN4vnBttLPXds6q0A87LzbyxvHNkJ gkwW8hoERtxWQtt+PswfyqqgVCyYepHyejQlVIDzS1F+9YY0Lm3zj1m5PDIE1Ltq0HX+ iDjzprk6bMyToKrUc5SdtizTLrkdgYEKXREMziVJ5FdAMiLkhHXY6JXLSVxFpo2/CyWO gewUSw4hHrp8ncUG3effNvIs76xakl+49zap8YlOceApvupQXrxyYb34dsDO0bSiEHsg T87NIrjfr540YzJqQADRqSbVI72b5D7czQWC8Bap2F54/7dykucE7LFmWF0gl6TwXHD5 00eQ== X-Gm-Message-State: APjAAAWA9snOG+LqvQuL0zuTrEygwcKYvTieUkvNeRI6/ZJUgZsRpTA6 lj0avTjmFFXFZrfJk3FBWzLI3ZvCyUetKe9XiPpH7MAN X-Google-Smtp-Source: APXvYqwZmbrxMGS9Jr/t0GfAQdMQsL5g5BWMnp/+mC7Ndb97EbzcmAUTnmUyjqgUtJdNuX7skaBSjde1LVZZuuMn3Vk= X-Received: by 2002:a67:e8c3:: with SMTP id y3mr43337208vsn.94.1563825792304; Mon, 22 Jul 2019 13:03:12 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Matthew Ahrens Date: Mon, 22 Jul 2019 13:03:01 -0700 Message-ID: Subject: July OpenZFS Leadership Meeting To: developer , illumos-zfs , zfs-devel@list.zfsonlinux.org, zfs-devel@freebsd.org, freebsd-fs , zfs-discuss X-Rspamd-Queue-Id: 2946688A66 X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org; dkim=pass header.d=delphix.com header.s=google header.b=Ew2nWKhb; dmarc=pass (policy=none) header.from=delphix.com; spf=pass (mx1.freebsd.org: domain of matthew.ahrens@delphix.com designates 2607:f8b0:4864:20::e30 as permitted sender) smtp.mailfrom=matthew.ahrens@delphix.com X-Spamd-Result: default: False [-6.59 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[delphix.com:s=google]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[zfs-devel@freebsd.org]; RCPT_COUNT_FIVE(0.00)[6]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[delphix.com:+]; MX_GOOD(-0.01)[alt1.aspmx.l.google.com,aspmx.l.google.com,aspmx2.googlemail.com,alt2.aspmx.l.google.com,aspmx3.googlemail.com]; RCVD_IN_DNSWL_NONE(0.00)[0.3.e.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; NEURAL_HAM_SHORT(-0.82)[-0.825,0]; DMARC_POLICY_ALLOW(-0.50)[delphix.com,none]; IP_SCORE(-3.06)[ip: (-9.70), ipnet: 2607:f8b0::/32(-3.11), asn: 15169(-2.43), country: US(-0.05)]; FORGED_SENDER(0.30)[mahrens@delphix.com,matthew.ahrens@delphix.com]; MIME_TRACE(0.00)[0:+,1:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[mahrens@delphix.com,matthew.ahrens@delphix.com]; RCVD_COUNT_TWO(0.00)[2] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Jul 2019 20:03:14 -0000 The next OpenZFS Leadership meeting will be held tomorrow, July 23, 11am-noon Pacific time. Everyone is welcome to attend and participate, and we will try to keep the meeting on agenda and on time. The meetings will be held online via Zoom, and recorded and posted to the website and YouTube after the meeting. The agenda for the meeting will be a discussion of the projects listed in the agenda doc. For more information and details on how to attend, as well as notes and video from the previous meeting, please see the agenda document: https://docs.google.com/document/d/1w2jv2XVYFmBVvG1EGf-9A5HBVsjAYoLIFZAnWHhV-BM/edit --matt From owner-zfs-devel@freebsd.org Wed Jul 24 19:29:16 2019 Return-Path: Delivered-To: zfs-devel@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 56FFEB6123 for ; Wed, 24 Jul 2019 19:29:16 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: from mail-vs1-xe2e.google.com (mail-vs1-xe2e.google.com [IPv6:2607:f8b0:4864:20::e2e]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C9ACF6F722 for ; Wed, 24 Jul 2019 19:29:14 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: by mail-vs1-xe2e.google.com with SMTP id 2so32096277vso.8 for ; Wed, 24 Jul 2019 12:29:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=delphix.com; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to; bh=HQOhM3ZhO5C03Lzmk4gcwuPi6AhyzQACwz5PLc9zG9U=; b=XEyS3lk1XBgoVjwjWRGXADrHtv24YKl31jQtALlDdxxF+On92jvxutez8dexm6HZjJ Po0oq6zC1NDWm/n+HzInlRBZ/4nBoqIgJF29BJjNhxTNaMDkFQa+qXYYynwpuweG2ECk np5ZGKoRrH+nVlWU/RONxGtqj6eW6oIGI3PEM= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=HQOhM3ZhO5C03Lzmk4gcwuPi6AhyzQACwz5PLc9zG9U=; b=lfWB9XHtFwSwHHRuP7PVT9a1ltDdAKfGEUuJ1i6xkxygdlQRimwaDRR32CE91+zABv SKb1Rrw3HOY5jwMOmyTbm//adGmIzpVVk1+Tp+Sg6OdEFaZ07zir+48XzkdtenQrn3KS IsdnukfUrLidG9dwJkPs1R5AJZYujv9OgRFuCLKAI+XRCra6pdDlK1nxqPdieR2tWWzr 49zGXeE22rrLJyp73gAxHEGLr1UPkaUr8yCGtL3J2nIdYLEAhsgWiHK3+9GTpTO/oEkH zePOmAL7n557yyGRIX/OwWhkyS9Ht+gVWEL/O4mLILLyrASKgwnaDLZ8+kE1e8ngLVwY eLxw== X-Gm-Message-State: APjAAAXhuMQxf9tqT8TP9pS435heirZZ6OPSo+ZP4xlw2EdeeM2Pwl/b SrU2Tmbr6Ig9xnIcYwXGQlAltRWqvMBqpuZilyS1lw== X-Google-Smtp-Source: APXvYqwuR4OU/Ffh/C3QhlQlOQgWHDWvj2aQFi1YKmMLheirreOwSrkC+QE19/io/6tibGmWpnDc+iZi0Bnla24vEIQ= X-Received: by 2002:a05:6102:3c5:: with SMTP id n5mr14383131vsq.56.1563996553997; Wed, 24 Jul 2019 12:29:13 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Matthew Ahrens Date: Wed, 24 Jul 2019 12:29:02 -0700 Message-ID: Subject: Re: July OpenZFS Leadership Meeting To: developer , illumos-zfs , zfs-devel@list.zfsonlinux.org, zfs-devel@freebsd.org, freebsd-fs , zfs-discuss X-Rspamd-Queue-Id: C9ACF6F722 X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org; dkim=pass header.d=delphix.com header.s=google header.b=XEyS3lk1; dmarc=pass (policy=none) header.from=delphix.com; spf=pass (mx1.freebsd.org: domain of matthew.ahrens@delphix.com designates 2607:f8b0:4864:20::e2e as permitted sender) smtp.mailfrom=matthew.ahrens@delphix.com X-Spamd-Result: default: False [-6.71 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[delphix.com:s=google]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[zfs-devel@freebsd.org]; RCPT_COUNT_FIVE(0.00)[6]; NEURAL_HAM_SHORT(-0.94)[-0.941,0]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[delphix.com:+]; DMARC_POLICY_ALLOW(-0.50)[delphix.com,none]; RCVD_IN_DNSWL_NONE(0.00)[e.2.e.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; MX_GOOD(-0.01)[alt1.aspmx.l.google.com,aspmx.l.google.com,aspmx2.googlemail.com,alt2.aspmx.l.google.com,aspmx3.googlemail.com]; IP_SCORE(-3.06)[ip: (-9.71), ipnet: 2607:f8b0::/32(-3.09), asn: 15169(-2.43), country: US(-0.05)]; FORGED_SENDER(0.30)[mahrens@delphix.com,matthew.ahrens@delphix.com]; MIME_TRACE(0.00)[0:+,1:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[mahrens@delphix.com,matthew.ahrens@delphix.com]; RCVD_COUNT_TWO(0.00)[2] Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Jul 2019 19:29:16 -0000 The video and meeting notes are now available: https://www.youtube.com/watch?v=3DltIIDDq9qag&feature=3Dyoutu.be - ZoL + SIMD + Linux 5.0 (Matt) - Brian integrated some support for this already but he is currently on vacation - 5.0+ made some changes to the floating point registers that were used by ZoL for SIMD instructions - Brian reimplemented some of those in ZoL and now these instructions work again and will be backported in 0.8 - Needed for performant RAIDZ, Encryption, and some compression algorithms - Used only if the underlying CPU architecture actually supported (checked during runtime) - OpenZFS Developer Summit (Matt) - Announced: Nov 4-5. Same locations as last year in SF. - Registration and call for papers is open. Please submit a talk and tell us about the cool stuff you=E2=80=99re doing with ZFS - We appreciate people volunteering to help out with things for the conference. - Karyn will reach out to Rainbow about help with AV. - Ticket prices went up to $100. There are some discount tickets available, and let us know if the pricing is a hardship and we can tr= y to figure something out. - FreeBSD Foundation may have travel grants available if want to apply - Get all the details (including a link to the registration site) on the OpenZFS wiki page - Property to track amount of queued TRIM/UNMAP (like =E2=80=9Cfreeing=E2= =80=9D for async destroy) (Allan) - Allan Jude: users may want to know how many trim/unmapped is left or in-progress before performing other actions instead of performing the= pool - Matt: Maybe a flag to zpool sync =E2=80=94trim that could wait for al= l trim activity to be done. Probably better than the existing proposal where= we have a property like freeing for it, which would jump up and down - `zpool wait` that is about to be upstreamed from Delphix and should be a good candidate to add this functionality. You can look at the Delphix repo if you want to see it before the PR. - RAIDZ removal issue with design ideas (Igor) - RAIDZ expansion progress (Matt) - No progress to report since preview posted - The hope is to have takers to test it out - The functionality is there, there are on-disk format changes (so don't use it in production), a few race conditions being fixed and so= me more error handling for device failures. - Important to note that even though we can expand the RAIDZ we still can't remove devices (e.g. zpool remove) - ZoL PR exists with design considerations for the feature - OpenZFS feature matrix (everyone) - FreeBSD is hoping to catch up with ZoL through the ZoF work - NFSv4 ACL PR waiting for code review - should we consider tracking this in the feature Matrix? - Update: Added - Note: already on illumos On Mon, Jul 22, 2019 at 1:03 PM Matthew Ahrens wrote: > The next OpenZFS Leadership meeting will be held tomorrow, July 23, > 11am-noon Pacific time. > > Everyone is welcome to attend and participate, and we will try to keep th= e > meeting on agenda and on time. The meetings will be held online via Zoom= , > and recorded and posted to the website and YouTube after the meeting. > > The agenda for the meeting will be a discussion of the projects listed in > the agenda doc. > > For more information and details on how to attend, as well as notes and > video from the previous meeting, please see the agenda document: > > > https://docs.google.com/document/d/1w2jv2XVYFmBVvG1EGf-9A5HBVsjAYoLIFZAnW= HhV-BM/edit > > --matt > From owner-zfs-devel@freebsd.org Mon Aug 19 19:56:25 2019 Return-Path: Delivered-To: zfs-devel@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 8199AD01EB for ; Mon, 19 Aug 2019 19:56:25 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: from mail-vs1-xe34.google.com (mail-vs1-xe34.google.com [IPv6:2607:f8b0:4864:20::e34]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46C4S84Jj3z4G41 for ; Mon, 19 Aug 2019 19:56:24 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: by mail-vs1-xe34.google.com with SMTP id 66so1989420vsz.8 for ; Mon, 19 Aug 2019 12:56:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=delphix.com; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to; bh=85rcawT9k+6wJWDVsw4ilrIJL5plRM0xX5oDXDxfZIc=; b=Qr5KaeN6V+RvD4uRSHd0wii6cF/ESg3llpCNtAoe+jth/WINA0TU1qRxNcRzVSxaAD lut7bZP3O7AKPth8d4egMn0ZtS2qNfC8yxH+in6XJqledM2FOdyNAMfGnVX2KePYWzML rMKlPiZMpIVFjuyDq1TFyR5zrz9UtCBobaWQw= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=85rcawT9k+6wJWDVsw4ilrIJL5plRM0xX5oDXDxfZIc=; b=AQu92Y6f4JcVLRS56S436lYCguiGbpk1xlBMsRLLxG8vRc1UBOjlDoEsdDox8xOnnk 38nTVNiKjlYZAcGM75ghhy273qOX0l8KtLYOied1RESN5EBmmzChHdhISexBQLT9dhlK 6KlydzeNXagsw9f27D7Sh3fCeXGSstFaj7b0IWquRl3jR63J8gtgFA7ujqo0QrSJKeH+ 3AvGfTVvL5zJgMQcJU01wMFenrnb3XtjlQTtuoUm+h+uR4weu9ydB8AJtMLPoJz7sIxm ReZF2UyQXMKexqa+I8yGKdrnlFB+9N0ZCfA9KyCBVn44JUZSUV9xlI2dAnuVL2CeyySn 430w== X-Gm-Message-State: APjAAAXA9iBbF3IpAE7r1GjMWWXkHQeE1sDnaIj4cqwHdUtZIL18fUks 7+ZheIqSDaFxJl4qJ/8iyj6VnoW8I/w6qsOEwyLWsA== X-Google-Smtp-Source: APXvYqwkx+MkyfL/3DiiQzASrbY9QhoPLbW0vF2bnoZbvU8r6KErTs+J2DwIa7PsonSzOVRcnacazu9vA+VW2m8Lj0U= X-Received: by 2002:a67:ba12:: with SMTP id l18mr2460025vsn.56.1566244583179; Mon, 19 Aug 2019 12:56:23 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Matthew Ahrens Date: Mon, 19 Aug 2019 12:56:12 -0700 Message-ID: Subject: August OpenZFS Leadership Meeting To: developer , illumos-zfs , zfs-devel@list.zfsonlinux.org, zfs-devel@freebsd.org, freebsd-fs , zfs-discuss X-Rspamd-Queue-Id: 46C4S84Jj3z4G41 X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=delphix.com header.s=google header.b=Qr5KaeN6; dmarc=pass (policy=none) header.from=delphix.com; spf=pass (mx1.freebsd.org: domain of matthew.ahrens@delphix.com designates 2607:f8b0:4864:20::e34 as permitted sender) smtp.mailfrom=matthew.ahrens@delphix.com X-Spamd-Result: default: False [-5.71 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[delphix.com:s=google]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[zfs-devel@freebsd.org]; URI_COUNT_ODD(1.00)[3]; RCPT_COUNT_FIVE(0.00)[6]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[delphix.com:+]; DMARC_POLICY_ALLOW(-0.50)[delphix.com,none]; RCVD_IN_DNSWL_NONE(0.00)[4.3.e.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; NEURAL_HAM_SHORT(-0.99)[-0.994,0]; FORGED_SENDER(0.30)[mahrens@delphix.com,matthew.ahrens@delphix.com]; MIME_TRACE(0.00)[0:+,1:+,2:~]; IP_SCORE(-3.02)[ip: (-9.74), ipnet: 2607:f8b0::/32(-2.93), asn: 15169(-2.37), country: US(-0.05)]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[mahrens@delphix.com,matthew.ahrens@delphix.com]; RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Aug 2019 19:56:25 -0000 The next OpenZFS Leadership meeting will be held tomorrow, August 20, 1pm-2pm Pacific time. Everyone is welcome to attend and participate, and we will try to keep the meeting on agenda and on time. The meetings will be held online via Zoom, and recorded and posted to the website and YouTube after the meeting. The agenda for the meeting will be a discussion of the projects listed in the agenda doc. For more information and details on how to attend, as well as notes and video from the previous meeting, please see the agenda document: https://docs.google.com/document/d/1w2jv2XVYFmBVvG1EGf-9A5HBVsjAYoLIFZAnWHhV-BM/edit --matt From owner-zfs-devel@freebsd.org Mon Aug 19 19:59:09 2019 Return-Path: Delivered-To: zfs-devel@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 2D5F8D029E for ; Mon, 19 Aug 2019 19:59:09 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: from mail-vs1-xe33.google.com (mail-vs1-xe33.google.com [IPv6:2607:f8b0:4864:20::e33]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46C4WJ0mm1z4G8h for ; Mon, 19 Aug 2019 19:59:07 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: by mail-vs1-xe33.google.com with SMTP id j25so1991210vsq.12 for ; Mon, 19 Aug 2019 12:59:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=delphix.com; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to; bh=KqlB+G/xkp3u7Q5y+exajZ+vjV4ZgcUFQuKxi+p56Ik=; b=PYjVFBorAwMXioLktYTspB7sNfSat1uwfSak/Ua7sZqALNKgxYPn5G5gwH1hcq3af6 JaJz5JWXAmMi4B3tJm5XLzu8sZT+9c7H6Wl8ARGIwqRIaJPAASOCIoMsiHitVEmGdGI4 dQ2t3cRkiaQv7ZlhKIxKEk2eQpGC4cAYIWInY= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=KqlB+G/xkp3u7Q5y+exajZ+vjV4ZgcUFQuKxi+p56Ik=; b=IHCngO+nlck4AXtweelNko2WgxnwHfe/XQi3jne+HemzGc0lCX82Y9lGr0ltrgh9Dh Ef9AvINXlann6AOoEZy628kYmk1amFruG/3QsshIX6iOHCNRygS2/gxrykfj+C8Lp1It a6uKAcILAtWDWaDY7VmPF60/jw2oh8RouhFI2DpYaRsylSOGq5g/A/XI7Eezfv6OPXWn kzbAbL5d3E6F2JBMVCK1xACzLdwa+SO24zr82X9TtvKU6HiFeVITzM/2LCHLVapgdve5 mbWsbyWrzNqJtBSrB9rRDS0PsziMPGcFJo1dJZD/dSS+WT/xTn9270OthhhAv0cE2jPe stgQ== X-Gm-Message-State: APjAAAXNeIzXCxGlo0h0ApOJHl1y4bnfamcyS2VkiFRZQqXWPhaO9Sy1 rzXBlqHUSBDUz01q8bwBR+jK1DVGhUrEZTrj9frAZA== X-Google-Smtp-Source: APXvYqxq93L4hRNZiQd7At80C9fM8bE7Gqq21/EImf/KBvuzcOtuN9D71i0dESVrglPLtkiA8P1U5qTCz/qD2IdnA0A= X-Received: by 2002:a67:e45a:: with SMTP id n26mr2288740vsm.94.1566244744705; Mon, 19 Aug 2019 12:59:04 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Matthew Ahrens Date: Mon, 19 Aug 2019 12:58:53 -0700 Message-ID: Subject: Last day to submit OpenZFS Developer Summit talks! To: developer , illumos-zfs , zfs-devel@list.zfsonlinux.org, zfs-devel@freebsd.org, freebsd-fs , zfs-discuss X-Rspamd-Queue-Id: 46C4WJ0mm1z4G8h X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=delphix.com header.s=google header.b=PYjVFBor; dmarc=pass (policy=none) header.from=delphix.com; spf=pass (mx1.freebsd.org: domain of matthew.ahrens@delphix.com designates 2607:f8b0:4864:20::e33 as permitted sender) smtp.mailfrom=matthew.ahrens@delphix.com X-Spamd-Result: default: False [-5.68 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[delphix.com:s=google]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36:c]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[zfs-devel@freebsd.org]; URI_COUNT_ODD(1.00)[3]; RCPT_COUNT_FIVE(0.00)[6]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[delphix.com:+]; DMARC_POLICY_ALLOW(-0.50)[delphix.com,none]; RCVD_IN_DNSWL_NONE(0.00)[3.3.e.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; NEURAL_HAM_SHORT(-0.99)[-0.995,0]; SUBJECT_ENDS_EXCLAIM(0.00)[]; FORGED_SENDER(0.30)[mahrens@delphix.com,matthew.ahrens@delphix.com]; MIME_TRACE(0.00)[0:+,1:+,2:~]; IP_SCORE(-2.99)[ip: (-9.59), ipnet: 2607:f8b0::/32(-2.93), asn: 15169(-2.37), country: US(-0.05)]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[mahrens@delphix.com,matthew.ahrens@delphix.com]; RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Aug 2019 19:59:09 -0000 Today is the last day to submit a talk for this year's OpenZFS Developer Summit! The first day of the conference will be set aside for presentations. *If you would like to give a talk, submit a proposal via email to matt@mahrens.org , including a 1-2 paragraph abstract.* We welcome proposals for any kinds of talks related to ZFS, and we are particularly interested in fostering a diverse group of speakers. In the past, the most popular talks have featured internal details of new or upcoming features in OpenZFS, and explanations of how existing subsystems work. If you're new to the OpenZFScommunity or new to giving talks at conferences, mentorship is available to help make your presentation the best it can be. The registration fee will be waived for speakers. The sixth annual OpenZFS Developer Summit will be held in San Francisco, *November 4-5, 2019*. All OpenZFS developers are invited to participate, and registration is now open. See http://www.open-zfs.org/wiki/OpenZFS_Developer_Summit for all of the details, including slides and videos from previous years' conferences. --matt From owner-zfs-devel@freebsd.org Fri Aug 23 18:26:43 2019 Return-Path: Delivered-To: zfs-devel@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 164FACB561 for ; Fri, 23 Aug 2019 18:26:43 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: from mail-vs1-xe43.google.com (mail-vs1-xe43.google.com [IPv6:2607:f8b0:4864:20::e43]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46FVGp3255z43qk for ; Fri, 23 Aug 2019 18:26:42 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: by mail-vs1-xe43.google.com with SMTP id q16so6864054vsm.2 for ; Fri, 23 Aug 2019 11:26:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=delphix.com; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to; bh=p61fMSEdFBokvj/3uq6VpZ3q8e7gQs6isff/2r/ti1w=; b=aXcw8eEJ+YcycnUXUnBElWEWt6DvMgHQW1GNUibjyBr+a1Mhlsl5LLByZ65Uxtg4hP fWaUvhkUE9l3RWtkgQb2v2A90YAt5BT64nGDfn9sow9oCsIqc2+45ZMUe9vgBCYZBmBO V+v7G3BnqLKoKpr1lySVg9P9aJlPbxsr3NQ2g= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=p61fMSEdFBokvj/3uq6VpZ3q8e7gQs6isff/2r/ti1w=; b=seIQ7uHQ/uTlB+l8Z8HEQuQ3pnGq/7JyKhzpL5j9vawSq8mCUEi9JUGykqZIXsUu7T Ywu4QdYQnEta3NL7E8Gzx5yooZIpiM8vKKk7Ic/NN1QqrzrSJ+i0DsEjP1NY04XPWCcQ zfogqjCdV5EmuWZlymTLh2byDAsaDkXtQ4dilIeptHdrM0/SHhk2O6cbSqQ/qj6Q6NDM dMWy3UeBtwt46xu4Rs9ybv0xMFxuVBisI6EyvdI7s85rh1bhVJYT/Baz56iZ29emI4fG j805Y06s/ZBL2YXKfU21hmvjOGBhAVyiq8AQmQXrIuWZdLhp8KgdshVcdfckdXbgzm0l QMKQ== X-Gm-Message-State: APjAAAUg62e3fekzEawuaGi2ZKhGRBbxtfDZUrJBBOu2JDu60RbJc1ib Qznf7pN+uYeC3Tg5ETro9Tt6c8yMEHenWLwly6A3HA== X-Google-Smtp-Source: APXvYqxYJxniMzLbSx4gKcGlrch+7wgZY7ypODFYuN+kXVz6wygD+ucfSg6V7VzQizRcQ/yqw2ZR6yU98bDGRRIRX8k= X-Received: by 2002:a67:e45a:: with SMTP id n26mr3604523vsm.94.1566584800662; Fri, 23 Aug 2019 11:26:40 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Matthew Ahrens Date: Fri, 23 Aug 2019 11:26:28 -0700 Message-ID: Subject: Re: August OpenZFS Leadership Meeting To: developer , illumos-zfs , zfs-devel@list.zfsonlinux.org, zfs-devel@freebsd.org, freebsd-fs , zfs-discuss X-Rspamd-Queue-Id: 46FVGp3255z43qk X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=delphix.com header.s=google header.b=aXcw8eEJ; dmarc=pass (policy=none) header.from=delphix.com; spf=pass (mx1.freebsd.org: domain of matthew.ahrens@delphix.com designates 2607:f8b0:4864:20::e43 as permitted sender) smtp.mailfrom=matthew.ahrens@delphix.com X-Spamd-Result: default: False [-3.28 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[delphix.com:s=google]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[zfs-devel@freebsd.org]; URI_COUNT_ODD(1.00)[9]; RCPT_COUNT_FIVE(0.00)[6]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[delphix.com:+]; DMARC_POLICY_ALLOW(-0.50)[delphix.com,none]; RCVD_IN_DNSWL_NONE(0.00)[3.4.e.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; NEURAL_HAM_SHORT(-0.98)[-0.984,0]; FORGED_SENDER(0.30)[mahrens@delphix.com,matthew.ahrens@delphix.com]; MIME_TRACE(0.00)[0:+,1:+,2:~]; IP_SCORE(-0.60)[ip: (2.28), ipnet: 2607:f8b0::/32(-2.88), asn: 15169(-2.34), country: US(-0.05)]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[mahrens@delphix.com,matthew.ahrens@delphix.com]; RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2] Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 Aug 2019 18:26:43 -0000 Thanks to everyone who participated at this month's meeting. We discussed OpenZFS DevSummit talks; the Linux Plumbers Conference; 32-bit kernel atomic reads; zfs share -a on Linux; and dedup improvements. The video is available here: https://www.youtube.com/watch?v=3Dzl4CUMOIi18 Notes (thanks Karyn): - OpenZFS DevSummit talks (matt) - Lots of great talks so far, but there is still room. - If you forgot to submit a talk before yesterday=E2=80=99s deadline, p= lease reach out right away. Matt will announce selected speakers in 2 weeks= , so there is a small grace period. - Registration is open now, and Matt will publicize this more once we announce the speakers. - Linux Plumbers conf (Serapheim) - George WIlson and Serapheim will be at the Conference if you want to talk ZFS or Debugging the Linux Kernel - If so, please reach out so the ZFS folks can meet up and talk about all things ZFS. - OpenZFS on 32-bit architectures (relevant for FreeBSD, Linux, =E2=80=A6)= : plain reads of 64-bit =E2=80=9Catomics=E2=80=9D (Andriy) - Issue discovered: We use atomic operations to modify 64-bit values that we treat as atomic, but to read them we use plain C assignment operations, which works ok for 64-bit platforms, but not for 32-bit. Those plain reads are implemented as 2 instructions there (32-bit), which i= s incorrect. - At this point it is mostly theoretical (its is rare to hit a write while crossing the 32-bit boundary) but it is still a valid concern. - The code paths are not very critical - mostly stats that we send to user space. - Proposal for discussion: Should we introduce some kind of format wrappers for those operations? - Paul Dagnelie: Has done some similar work for the SPL back in the day and this should be straightforward. - Question: How many 32-bit platforms do we actually care for at this point? - Andriy: Have heard of a few 32-bit Intel users out there - Tom Caputi: Has some experience for compile-time checks for structures of these nature. - It seems like people are for it. Andriy will first talk with FreeBSD developers, figure out how they want to proceed, and try to propose designs common for all platforms. - zfs share -a (George) - Issue: Stale file handles using zfs share -a and the sharenfs property, by restarting or rebooting the nfs server. - The reason is that ZFS plugs into SystemD in a cumbersome/non-natural way. - NFS shares actually start before the ZFS service. Mountd responds to the client before the ZFS share is actually completed leading to the stale handles. - Hacky Approach: - Introduce a new component to zfs share. Instead of zfs share -a , zfs share -g (short for generate) - The idea is to add ZFS logic that ties it closer to NFS - Change of dependency ordering, zfs share -g generates the handles for NFS, before NFS, and NFS consumes those generated handles. - Cons: A bit hacky for now - Pros: Simple and fast - Plan: To do more redesign on top of this for a cleaner solution - Others that have similar issues with George/Delphix feel free to reach out. - Tom Caputi: Datto had issues with NFS in the past, and don't use sharenfs, but they are interested on having control over th= is functionality - Allan Jude: Seems like FreeBSD already does something similar, so there may be some convergence of goals. - Submitted this as lightning talk at the Dev Summit - Anyone interested in working on implementing Matt=E2=80=99s ideas for de= dup ceiling and on disk format log ( http://open-zfs.org/w/images/8/8d/ZFS_dedup.pdf) please contact allan@klarasystems.com to discuss. (Allan) - Problems using the feature for Terabytes of data due to known memory issues - There is budget for a contract to implement the limit of the size of the DDT proposed during one of the past Dev Summits - Feel free to reach out if you are interested in undertaking that work On Mon, Aug 19, 2019 at 12:56 PM Matthew Ahrens wrote= : > The next OpenZFS Leadership meeting will be held tomorrow, August 20, > 1pm-2pm Pacific time. > > Everyone is welcome to attend and participate, and we will try to keep th= e > meeting on agenda and on time. The meetings will be held online via Zoom= , > and recorded and posted to the website and YouTube after the meeting. > > The agenda for the meeting will be a discussion of the projects listed in > the agenda doc. > > For more information and details on how to attend, as well as notes and > video from the previous meeting, please see the agenda document: > > > https://docs.google.com/document/d/1w2jv2XVYFmBVvG1EGf-9A5HBVsjAYoLIFZAnW= HhV-BM/edit > > --matt > From owner-zfs-devel@freebsd.org Mon Sep 9 20:35:38 2019 Return-Path: Delivered-To: zfs-devel@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 1DCBDE0685 for ; Mon, 9 Sep 2019 20:35:38 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: from mail-ua1-x941.google.com (mail-ua1-x941.google.com [IPv6:2607:f8b0:4864:20::941]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46S0Kj0HlLz4cBm for ; Mon, 9 Sep 2019 20:35:36 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: by mail-ua1-x941.google.com with SMTP id 107so4798753uau.5 for ; Mon, 09 Sep 2019 13:35:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=delphix.com; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to; bh=x8TFBmNd96XnUodIAqkQh9BfvobSDA7czqkuGfoHj5Q=; b=bFo7sttXv62Sy4gJ5kspiW7GXrGnI9ovu6/PQFkDfazTXcglyuTdBZODNcGuhM3sdg LA8anGp0ZAZ1Jh1bxwVSduEjDqzLuJg7NnQGgPpMJPun138+PQo9C3DkeL+HbBfQFW1U HUBEgH111GjaczOCNo9KfNqhqOFTJiUTsBlHI= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=x8TFBmNd96XnUodIAqkQh9BfvobSDA7czqkuGfoHj5Q=; b=INM4qc3dt8tekbAQ+UIRJ32kb4v3tpqlOcV9f4K1hFIcxtF7PAHHZjv0Bv0e+dG99w iAVLvW0x7v2jkUUnNupK1ySUHRVJGzOa0DuBLKTKPcGBFPGhjKJ3BV0ywDus5IffGCLp HCxyibjRYu+lehawDDhgyHiQa+SFfTX8jTpTuuZjp+FTvdGWsoAj8AB8zHeWxJxl5d5O G8XFqf2jAWGvC0waRxARfkXLdvGP1ze0FPa6xCiq988iHe16OVUFnZi30OMC+UpFMh62 aMmvc0r3C+68TFTj0KS1kSmn5JtMqR+wEQokci/bx3893BkWOtL6g+3SKxZgNCDNuxJy oTUA== X-Gm-Message-State: APjAAAVPx1a7UN5Fik52TsWf6Nk6F1n8zLGS/S8B/FRcRkILpJ0qojqi 3mx+Nf1ju48tPKfemQpG2O1/653EOTXXFCJClcxExTNcWK4= X-Google-Smtp-Source: APXvYqxw8XzD8Y3o6ILRSlpnIl0wCce9tTo4TzNT4UAuIjRrVSDVwJgrYLnoVDuANG8acmjErKEtZ9TdfLBWmjjYGHQ= X-Received: by 2002:ab0:7382:: with SMTP id l2mr12066941uap.64.1568061335532; Mon, 09 Sep 2019 13:35:35 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Matthew Ahrens Date: Mon, 9 Sep 2019 13:35:24 -0700 Message-ID: Subject: OpenZFS Developer Summit 2019 - Presentations announced To: developer , illumos-zfs , zfs-devel@freebsd.org, freebsd-fs X-Rspamd-Queue-Id: 46S0Kj0HlLz4cBm X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=delphix.com header.s=google header.b=bFo7sttX; dmarc=pass (policy=none) header.from=delphix.com; spf=pass (mx1.freebsd.org: domain of matthew.ahrens@delphix.com designates 2607:f8b0:4864:20::941 as permitted sender) smtp.mailfrom=matthew.ahrens@delphix.com X-Spamd-Result: default: False [-3.14 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[delphix.com:s=google]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[zfs-devel@freebsd.org]; TO_DN_SOME(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[delphix.com:+]; DMARC_POLICY_ALLOW(-0.50)[delphix.com,none]; RCVD_IN_DNSWL_NONE(0.00)[1.4.9.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; FORGED_SENDER(0.30)[mahrens@delphix.com,matthew.ahrens@delphix.com]; MIME_TRACE(0.00)[0:+,1:+,2:~]; IP_SCORE(-0.44)[ip: (2.84), ipnet: 2607:f8b0::/32(-2.74), asn: 15169(-2.26), country: US(-0.05)]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[mahrens@delphix.com,matthew.ahrens@delphix.com]; RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 09 Sep 2019 20:35:38 -0000 The sixth annual OpenZFS Developer Summit will be held in San Francisco, *November 4-5, 2019*. All OpenZFS developers are invited to participate, and registration is now open. We have a great line-up of talks to announce: TitleSpeakerCompany State of OpenZFS Matt Ahrens Delphix Metaslab Allocation Performance Paul Dagnelie Delphix Storage Configurator Steven Umbehocker OSNexus Capacity Usage Calculator Kody Kantor Joyent OpenZFS Everywhere Michael Dexter & Jorgen Lundman Debugging ZFS: State of the Art on Linux Tom Caputi Datto Debugging ZFS: From Illumos to Linux Serapheim Dimitropoulos Delphix ZFS TRIM Explained Brian Behlendorf LLNL Healing With ZFS Receive Alek Pinchuk Datto Securing the Cloud with ZFS Encryption Jason King Joyent libshare on Linux is Broken George Wilson & Don Brady Delphix VDEV Properties Allan Jude Klara Systems A Device by Any Other Name: Common Pitfalls in Device Naming for ZFS on Linux Sara Hartse & Don Brady Delphix Illumos Brings the SAS Kody Kantor Joyent Title TBD, re: Dual-Actuator Drives Muhammad Ahmad & Tim Walker Seagate See http://www.open-zfs.org/wiki/OpenZFS_Developer_Summit for all of the details. This event is only possible because of the efforts and contributions of the community of developers and companies that sponsor the event. Special thanks to our Diamond sponsors: Datto , Delphix , Intel , and OSNexus ; and our Gold sponsors: Argo Tech, Nexenta, OpenDrives, and RackTop. Additional sponsorship opportunities are available. Please see the website for details and send email to Karyn.Ritter@delphix.com if you have any questions. --matt > > The sixth annual OpenZFS Developer Summit will be held in San Francisco, *November > 4-5, 2019*. > > See http://www.open-zfs.org/wiki/OpenZFS_Developer_Summit for all of the > details, including slides and videos from previous years' conferences. > > The goal of the event is to foster cross-community discussions of OpenZFS work > and to make progress on some of the projects we have proposed. > > The first day of the conference will be set aside for presentations. *If > you would like to give a talk, submit a proposal via email > to matt@mahrens.org , including a 1-2 paragraph abstract.* We > welcome proposals for any kinds of talks related to ZFS, and we are > particularly interested in fostering a diverse group of speakers. In the > past, the most popular talks have featured internal details of new or > upcoming features in OpenZFS, and explanations of how existing subsystems > work. If you're new to the OpenZFS community or new to giving talks at > conferences, mentorship is available to help make your presentation the > best it can be. The registration fee will be waived for speakers. > > The deadlines are as follows: > > *August 19, 2019 All abstracts/proposals submitted* to matt@mahrens.org > September 6, 2019 Proposal submitters notified > November 4-5, 2019 OpenZFS Developer Summit > > The second day of the event will be a hackathon, where you will have the > opportunity to work with OpenZFS developers that you might not normally > sit next to, with the goal of having something, no matter how > insignificant, to demo at the end of the day. > > This event is only possible because of the efforts and contributions of > the community of developers and companies that sponsor the event. > Special thanks to our early Diamond sponsors: Delphix > , Datto , and Intel > . > > Additional sponsorship opportunities are available. Please see the website > for details and send email to Karyn.Ritter@delphix.com if you have any > questions. > > --matt > > From owner-zfs-devel@freebsd.org Tue Sep 17 15:47:52 2019 Return-Path: Delivered-To: zfs-devel@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 7E50F1279BD for ; Tue, 17 Sep 2019 15:47:52 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: from mail-vk1-xa31.google.com (mail-vk1-xa31.google.com [IPv6:2607:f8b0:4864:20::a31]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46XnYz2CYZz49k5 for ; Tue, 17 Sep 2019 15:47:50 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: by mail-vk1-xa31.google.com with SMTP id d66so838154vka.2 for ; Tue, 17 Sep 2019 08:47:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=delphix.com; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to; bh=sKuuOR9bV1ki/RC5fklVYl99NKMxNYIPQXDNGYdiP9I=; b=BUnk9hZYgWcohh8w5eHoUkg6eJ2eBc2Ok7rDtU3lZUIc7DrWjtCpdJFw0BFZ0rieiO DQAbZkUcrj2n1RoQqJ//NEzkTZdWN/LyQddLHXflPf6X7q9kwdfUm0kW/WJlKiqHuaX5 RRX7Lb3vCGWAox7e1qRZtelpPnNB/ny01OmnI= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=sKuuOR9bV1ki/RC5fklVYl99NKMxNYIPQXDNGYdiP9I=; b=nbpsHOqdM0J+KJ60Bjh6oMZrKQ6TdJ4TBehg8FrX8FYKRwxxU9JDtT0DjNjuJoOrWp g0W0Sx09IPEw8G5OMrdtswWm9fQXVQMzQVDTuiG5T6vQM/Fc5aNZOZi7WERBukJZsd2c y3GGBTAXj8uOEDA7fRtAx/HDb/+il1PV9Vn/M7Xd11KzWEQczRl8TWj1nmr58ch1+al2 w2DgJ2PnttpCpn1mAQu/irNAZW84gGwhIJzYBGbwrTJQck3uRsA5lCV55bSHTMyPxaBS ATyGyIEaczD5xUDB9fPMch1xpX0r1A3vd5GNlNFNZEUunYrlzisD8Pgy54F8l6T/kT8K w5aw== X-Gm-Message-State: APjAAAVGvvDtm1j6umKomA9GoD1+cI250FHujAoinOMj0Rg+tMUGDrl+ b4p/bzpfMZi0FY6pGhU89V2ATKzj0JLIbonfYX6nvQ== X-Google-Smtp-Source: APXvYqzD1+dp8m1Ov2PiC8QwrSHT2+AI7yF/mg91K9NVytvao0RqROiNlcO1GzChENDnlCX1+itWBJs2viJGgyo04p4= X-Received: by 2002:a1f:2192:: with SMTP id h140mr1786632vkh.92.1568735269773; Tue, 17 Sep 2019 08:47:49 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Matthew Ahrens Date: Tue, 17 Sep 2019 08:47:39 -0700 Message-ID: Subject: September OpenZFS Leadership Meeting To: developer , illumos-zfs , zfs-devel@freebsd.org, freebsd-fs , zfs-discuss X-Rspamd-Queue-Id: 46XnYz2CYZz49k5 X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=delphix.com header.s=google header.b=BUnk9hZY; dmarc=pass (policy=none) header.from=delphix.com; spf=pass (mx1.freebsd.org: domain of matthew.ahrens@delphix.com designates 2607:f8b0:4864:20::a31 as permitted sender) smtp.mailfrom=matthew.ahrens@delphix.com X-Spamd-Result: default: False [-4.47 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[delphix.com:s=google]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[zfs-devel@freebsd.org]; URI_COUNT_ODD(1.00)[3]; RCPT_COUNT_FIVE(0.00)[5]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[delphix.com:+]; DMARC_POLICY_ALLOW(-0.50)[delphix.com,none]; RCVD_IN_DNSWL_NONE(0.00)[1.3.a.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; FORGED_SENDER(0.30)[mahrens@delphix.com,matthew.ahrens@delphix.com]; MIME_TRACE(0.00)[0:+,1:+,2:~]; IP_SCORE(-2.77)[ip: (-8.86), ipnet: 2607:f8b0::/32(-2.69), asn: 15169(-2.23), country: US(-0.05)]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[mahrens@delphix.com,matthew.ahrens@delphix.com]; RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 17 Sep 2019 15:47:52 -0000 The next OpenZFS Leadership meeting will be held today, September 17, 1pm-2pm Pacific time. Everyone is welcome to attend and participate, and we will try to keep the meeting on agenda and on time. The meetings will be held online via Zoom, and recorded and posted to the website and YouTube after the meeting. The agenda for the meeting will be a discussion of the projects listed in the agenda doc. For more information and details on how to attend, as well as notes and video from the previous meeting, please see the agenda document: https://docs.google.com/document/d/1w2jv2XVYFmBVvG1EGf-9A5HBVsjAYoLIFZAnWHhV-BM/edit --matt From owner-zfs-devel@freebsd.org Mon Sep 23 16:22:25 2019 Return-Path: Delivered-To: zfs-devel@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 6C3B6F9504 for ; Mon, 23 Sep 2019 16:22:25 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: from mail-vs1-xe31.google.com (mail-vs1-xe31.google.com [IPv6:2607:f8b0:4864:20::e31]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46cV343pnNz4Cxt for ; Mon, 23 Sep 2019 16:22:24 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: by mail-vs1-xe31.google.com with SMTP id s7so9866861vsl.2 for ; Mon, 23 Sep 2019 09:22:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=delphix.com; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to; bh=m65j7pLCAtI71CtXbN5ZDPfSloKggZfwS4l/niGcA5E=; b=MUKftgt4asyOfAYPx0C4C+NkwzhnuVi2w/FjYIKvh8Jktgz2rH21QFJBJ/1plg+BYU 2vE0MmU6ZRQEl1NQ6HwAkfMvmJJwSuZJl3bpXeh6Iqy68rMb0dy4LR3a18F+76ItXad5 wSEAlxXQUR4sCsY372Fh2PE5Oc1RfdEMJ3MYc= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=m65j7pLCAtI71CtXbN5ZDPfSloKggZfwS4l/niGcA5E=; b=N6gilpTxC1R/C1B1eH6DPW/FLmHzE81/AdNZMOwN29mlTLUJGGAznFoLygyprkCdYW o2d3IEiKWd364t6pMXlWdIzjoVP9RLYCxrwOShftyPAy1E6RbdKMhsq0UIZCf8HFhmUV PiCSFFx+cg1GHqHszC94ZZUxBcoD3grpN5GZk/D4GqXkpqivQkDH2aCEjbFy+fCxjFqE gT58Sp1ykygBZ1N1hgPkYWb1byWprYPWvsEr0qPUgIABKXoSD+HyJ3Pi5LIuHzTTJdlZ ymZp7ZYb0PggZdNTwKnIepBUTFembKGodbGsuauEFjZgx73Mngvx/4YjNERi4nxD8JBx Xlag== X-Gm-Message-State: APjAAAVCLbnQ+krI5uco5a7mxo0MU1zYdt7EnBPm6l6gmGoneXRBQJQ8 T7MqKBar62+V9i4EEnUjY7VtrEUN6fkAE5F13MaX8jwAIJZfhA== X-Google-Smtp-Source: APXvYqyxL9osq8FMBBvHZ+ZfnmCKVchlhsKIgD5I1e8cYacNLpS0u+RqT1FSFsScym/0+QVKVV5Ib1QLfwbHtYjoUhI= X-Received: by 2002:a67:fe0b:: with SMTP id l11mr33560vsr.68.1569255742775; Mon, 23 Sep 2019 09:22:22 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Matthew Ahrens Date: Mon, 23 Sep 2019 09:22:11 -0700 Message-ID: Subject: Re: September OpenZFS Leadership Meeting To: developer , illumos-zfs , zfs-devel@freebsd.org, freebsd-fs , zfs-discuss X-Rspamd-Queue-Id: 46cV343pnNz4Cxt X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=delphix.com header.s=google header.b=MUKftgt4; dmarc=pass (policy=none) header.from=delphix.com; spf=pass (mx1.freebsd.org: domain of matthew.ahrens@delphix.com designates 2607:f8b0:4864:20::e31 as permitted sender) smtp.mailfrom=matthew.ahrens@delphix.com X-Spamd-Result: default: False [-5.63 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[delphix.com:s=google]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[zfs-devel@freebsd.org]; RCPT_COUNT_FIVE(0.00)[5]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[delphix.com:+]; DMARC_POLICY_ALLOW(-0.50)[delphix.com,none]; RCVD_IN_DNSWL_NONE(0.00)[1.3.e.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; FORGED_SENDER(0.30)[mahrens@delphix.com,matthew.ahrens@delphix.com]; MIME_TRACE(0.00)[0:+,1:+,2:~]; IP_SCORE(-2.93)[ip: (-9.75), ipnet: 2607:f8b0::/32(-2.64), asn: 15169(-2.20), country: US(-0.05)]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[mahrens@delphix.com,matthew.ahrens@delphix.com]; RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2] Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Sep 2019 16:22:25 -0000 At this month's meeting we discussed: - ZoL EOL of RHEL 6 - Xattr cross-platform compatibility - Relaxed quota semantics for improved performance - zpool replace of log vdev - temporal dedup Video is now up on youtube: https://youtu.be/kjBWhEE8tZ8 Full notes below (thanks Serapheim): - EOL ZoL on RHEL 6 (Brian Behlendorf) - RHEL 6 could be old enough that we could drop support for it on master (still supported for 0.8) - Technically will be EOL'd by Red Hat in November 2020. - Feedback from the community: Given enough Notifications beforehand, people should be fine - Actual change needed in ZoL: - Go through build system code and remove any references of v3.10 kernel and older (new oldest supported kernel would be 3.11). The process should be similar on what ZoL did for deprecating RHEL5. - Action Items: - Brian/Matt will give a heads up in the mailing list, in the release notes of each versions until then, open PR for this - We need volunteers for the build system changes - Xattr cross-platform compatibility (Andrew Walker) - Problem: - ixSystems works with services that receive alternate data streams written as xattrs in FreeBSD in the user namespace, which is implemented slightly different in Linux (there is "user." prefix - FreeBSD use= s "freebsd." prefix(?) - Solaris uses "smb." prefix). Their applicat= ion (Samba) is doing the same thing in Linux and FreeBSD, but ZFS represents them different on-disk between each platform. As a result, xattrs that are written in FreeBSD are visible in other OSes except from ZoL where= the metadata disappears. - Potential Solutions: - Brian: ZoL has around 4 prefixes, so one solution would be to have user as a fallback choice (e.g. if it is not part of any namespace, it is part of the user namespace). - Andrew Walker: Have a zfs dataset property to be able to tell which format is used - Andriy Gapon: Add some OS info on the actual attribute and have ZFS interpret them differently - Sef: Some form a feature flag that would fix the prefixes. - Matt Ahrens: First make it possible to read xattrs from all platforms, even if the names show up differently. A potential long-term solution: New stuff is written in some new format that is portable across platforms (e.g. in the zfs.* namespace) and each platform translates the ZFS prefixes to the local platform=E2=80=99s prefixes. - Question: Is it an incompatibility between different OSes? or an incompatibility between different implementations of ZFS? Shall we ha= ve a translation layer outside of ZFS? - A bit of both but mostly VFS layer (outside of ZFS code). Assuming it is only on the VFS layer, it would be reasonable to still have some way of accessing these attributes. A point for this, is that in ZoL there is little flexibility in changing the VFS code. - Action Items: - Proposal & Next steps - Andrew can start a writeup and coordinate with Alexander from iXSystems - Relax quota semantics for improved performance (Allan Jude) - Problem: As you approach quotas, ZFS performance degrades. - Proposal: Can we have a property like quota-policy=3Dstrict or loose, where we can optionally allow ZFS to run over the quota as long as performance is not decreased. - People's Feedback/Questions: - Richard Elling - Isn't it the same problem when the pool is almost full (SLOP space)? Answer: This is slightly different, but the mechanism is the same, and we don't want to break that (e.g. run beyond SLOP space just like that). - Tangent: Should we scale the SLOP space appropriately? The SLOP space can bite a big chunk of space in big pools. - Feedback: That seems reasonable, though the use cases may not be that many (fragmentation issues in such big pools will probably ar= ise before encountering the SLOP space issue). See discussion on previous PR. - zpool replace of a log (and maybe a cache) vdev =E2=80=93 does this work= well? Can it be improved? (Andriy Gapon) - Problem: a user had to replace a log device using the replace command and it took a long time (dozens of gigabytes were scanned). Can we do better? It seems like there is not special logic for devices like that, do we want to do something different for log vdevs? Even maybe prohibit using replace for these devices and advice the remove & add workflow. - Feedback: the above sound reasonable except for one thing. Log devices can have actual data on them. If you crash and you have block= s in the log device and you've removed the device, and you don't mount the specific filesystems, these blocks will stay there. Encryption should also make this more common. We need to retain the ability for the scrub-ba= sed replace/attach. We could improve the performance by looking at all th= e blocks of all the logs instead of looking at all the blocks in the po= ol. - Action Item: Andriy will look into this and create a doc - Renaming bookmarks =E2=80=93 are there any pitfalls? Seems like a useful= feature that=E2=80=99s not been implemented in a long time (Andriy Gapon) - Feedback: It should just work - one more thing to plumb through the CLI, libzfs, etc=E2=80=A6 internally, removing the ZAP entry and re-adding it with the new name should do the trick - Panzura to open source their temporal dedup implementation (Josh P) - Panzura will be open-sourcing some parts of their self-contained ZFS implementation of temporal dedup on Github. There is hope from Panzura that this will be integrated within OpenZFS but at least for now there are= no concrete plans of getting this code upstreamed without volunteers. - Question: What is temporal dedup? - A dedup scheme that groups blocks by the time that they are created/modified etc... Grouping blocks in such way should allow for faster access to the data due to caching based on temporal locality On Tue, Sep 17, 2019 at 8:47 AM Matthew Ahrens wrote: > The next OpenZFS Leadership meeting will be held today, September 17, > 1pm-2pm Pacific time. > > Everyone is welcome to attend and participate, and we will try to keep th= e > meeting on agenda and on time. The meetings will be held online via Zoom= , > and recorded and posted to the website and YouTube after the meeting. > > The agenda for the meeting will be a discussion of the projects listed in > the agenda doc. > > For more information and details on how to attend, as well as notes and > video from the previous meeting, please see the agenda document: > > > https://docs.google.com/document/d/1w2jv2XVYFmBVvG1EGf-9A5HBVsjAYoLIFZAnW= HhV-BM/edit > > --matt > From owner-zfs-devel@freebsd.org Mon Oct 14 17:40:13 2019 Return-Path: Delivered-To: zfs-devel@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 94A7413168D for ; Mon, 14 Oct 2019 17:40:13 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: from mail-vs1-xe36.google.com (mail-vs1-xe36.google.com [IPv6:2607:f8b0:4864:20::e36]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46sQn848Rlz4cv1 for ; Mon, 14 Oct 2019 17:40:12 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: by mail-vs1-xe36.google.com with SMTP id y129so11336667vsc.6 for ; Mon, 14 Oct 2019 10:40:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=delphix.com; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to; bh=4jRz60jvjylTbwVadW7p+Wu8lze8N/5gFlSbwPtCw98=; b=KbZY1CCv38KJyrFwF35bYXSkU+Pkckg1xjwuzeqtWC05cfTq4oD2eG0i92XfLE3n73 IXZo33cUs25ho6DpjZTKODY4mQPnB0kPd1/rtJHAaC8mhhNzvoeo2dPAOeZIYmeCMPbx orgOyJOn7oQVTYJ0Xj+m8/8AIbDrqtQuB86T0= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=4jRz60jvjylTbwVadW7p+Wu8lze8N/5gFlSbwPtCw98=; b=lJB5UxotgZVfnPdsx76twvuQxEPP5KFnN31X7YOyI+TxqRdtAqmwUUcQB/Pih/Ex+C cXOLlgVe6FE7Ii25XIHtjKX8vJM8denG5cJI+nAhvbhEv/au6uAnRtFmjPcE+6RTWnPi ySVBwx7kMwUBmL3X1qtGJZBAt3VrdDu6rCiIqZ+l8E4Jj52SsMFYMGD1B6MXO6E2KXhQ IZ9sLDDj3wpyZcwUHeee/SGG0s6CxOQSjebRdEw4wkGqoAE5H6JEceftw6QL7Lhg1JpF W5n5uJ0fc68dq0OqobzQM4Rsue3iOGg0RbPvvQIhHrWZC3xgsfpScG/AU2TetEDxuwyT 7UZA== X-Gm-Message-State: APjAAAWeBLwZY+utKaj3Dsle+kWG0L9GclmsWWSLGNRQs5vpB9bjDcLG xdruEEAZALnNtFdcoBlUEgW8TTuBUnJvRUgmU5fCQ+YzcNcEhA== X-Google-Smtp-Source: APXvYqwjMbZvReXiLwaaBsu6wZ/2nCeMxUIM8rK301AnfJp7NiVyVCtwVn+p7CpZPXDBVidvBvYGcZMUUU1Seq/sFiI= X-Received: by 2002:a05:6102:3010:: with SMTP id s16mr16518926vsa.237.1571074810889; Mon, 14 Oct 2019 10:40:10 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Matthew Ahrens Date: Mon, 14 Oct 2019 10:39:59 -0700 Message-ID: Subject: October OpenZFS Leadership Meeting To: developer , illumos-zfs , zfs-devel@freebsd.org, freebsd-fs X-Rspamd-Queue-Id: 46sQn848Rlz4cv1 X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=delphix.com header.s=google header.b=KbZY1CCv; dmarc=pass (policy=none) header.from=delphix.com; spf=pass (mx1.freebsd.org: domain of matthew.ahrens@delphix.com designates 2607:f8b0:4864:20::e36 as permitted sender) smtp.mailfrom=matthew.ahrens@delphix.com X-Spamd-Result: default: False [-5.57 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[delphix.com:s=google]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[zfs-devel@freebsd.org]; TO_DN_SOME(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[delphix.com:+]; DMARC_POLICY_ALLOW(-0.50)[delphix.com,none]; RCVD_IN_DNSWL_NONE(0.00)[6.3.e.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; FORGED_SENDER(0.30)[mahrens@delphix.com,matthew.ahrens@delphix.com]; MIME_TRACE(0.00)[0:+,1:+,2:~]; IP_SCORE(-2.87)[ip: (-9.69), ipnet: 2607:f8b0::/32(-2.50), asn: 15169(-2.11), country: US(-0.05)]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[mahrens@delphix.com,matthew.ahrens@delphix.com]; RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 14 Oct 2019 17:40:13 -0000 It's only 3 weeks until the OpenZFS Developer Summit. Time to register if you have not already! http://open-zfs.org/wiki/OpenZFS_Developer_Summit_2019 The next OpenZFS Leadership meeting will be held tomorrow, October 15th, 11am-noon Pacific time. Everyone is welcome to attend and participate, and we will try to keep the meeting on agenda and on time. The meetings will be held online via Zoom, and recorded and posted to the website and YouTube after the meeting. The agenda for the meeting will be a discussion of the projects listed in the agenda doc. For more information and details on how to attend, as well as notes and video from the previous meeting, please see the agenda document: https://docs.google.com/document/d/1w2jv2XVYFmBVvG1EGf-9A5HBVsjAYoLIFZAnWHhV-BM/edit --matt From owner-zfs-devel@freebsd.org Wed Oct 16 19:51:12 2019 Return-Path: Delivered-To: zfs-devel@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id DD08714E76E for ; Wed, 16 Oct 2019 19:51:12 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: from mail-vs1-xe2f.google.com (mail-vs1-xe2f.google.com [IPv6:2607:f8b0:4864:20::e2f]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46tjbN0HzNz4C3M for ; Wed, 16 Oct 2019 19:51:11 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: by mail-vs1-xe2f.google.com with SMTP id d3so16461593vsr.1 for ; Wed, 16 Oct 2019 12:51:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=delphix.com; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to; bh=3uNhVQ3diHFPgCIQmYuph3LQ383e6BatWJLkuEXkpRA=; b=K1GY1N8aXwhoO5jgOzQ7V1EBagR70DmJT+YGntBjyvJdKwM8EpUpN5pYxrdJm+d+ck HcHYWbq0ppkwGHVwId3lAX8gE310epfa4A6muNkv8nBW8lES/AuerXffUQwsJzHvn7Cj d2bXTS8JaofzREr+Iy+p+dyXaz8UJryZmYuXc= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=3uNhVQ3diHFPgCIQmYuph3LQ383e6BatWJLkuEXkpRA=; b=hN2YHuUU9LyVqIhGAdxmWWM3BoUwFg8eHiM7WAWOr58aOOuNYVm7IbF/MspdmK8PFW XDiJq6hVWfbeyECLFQ84waqbDedun3bTSz0rjKkwaexBsZfDJcsLMc/Ovg+mW4VJ1hvs sLNUCHH2Me1gbml91e2bqhdo3dqgExfGvlrukkI9WfYlU68OieQEvx2/saXLAKS3kj3g 8aEpm98EoI/GtGruWubgerE0qRom2Be+9KBH5+GXVVCciC/JXaqX8ltIvm8php6JGmcN HWlE4QQRiwEKxMYICXi1cd5ZoL7EQfG5zSE3fsn4vXCl70dR3Xo+HhxDh9Sf4Z8app0k 3n9g== X-Gm-Message-State: APjAAAV+k8yhhOalS0/8kpCzXXs6rJoOSy/RlWDwmkVquCmMiItr2rCg 0pgark46XDDGDfqIDaIbjtcrEt4MyKk/L2HLV0UCSw== X-Google-Smtp-Source: APXvYqwl1swh4b2FKVfg1U74vgonMhpE+o9pB8joOB0SObzEA4M/PlrqlI1VXhPwA86sFvgtNp5SOiQuF+ScopExh40= X-Received: by 2002:a67:6242:: with SMTP id w63mr25288571vsb.233.1571255470286; Wed, 16 Oct 2019 12:51:10 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Matthew Ahrens Date: Wed, 16 Oct 2019 12:50:59 -0700 Message-ID: Subject: Re: October OpenZFS Leadership Meeting To: developer , illumos-zfs , zfs-devel@freebsd.org, freebsd-fs X-Rspamd-Queue-Id: 46tjbN0HzNz4C3M X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=delphix.com header.s=google header.b=K1GY1N8a; dmarc=pass (policy=none) header.from=delphix.com; spf=pass (mx1.freebsd.org: domain of matthew.ahrens@delphix.com designates 2607:f8b0:4864:20::e2f as permitted sender) smtp.mailfrom=matthew.ahrens@delphix.com X-Spamd-Result: default: False [-5.56 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[delphix.com:s=google]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[zfs-devel@freebsd.org]; TO_DN_SOME(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[delphix.com:+]; DMARC_POLICY_ALLOW(-0.50)[delphix.com,none]; RCVD_IN_DNSWL_NONE(0.00)[f.2.e.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; FORGED_SENDER(0.30)[mahrens@delphix.com,matthew.ahrens@delphix.com]; MIME_TRACE(0.00)[0:+,1:+,2:~]; IP_SCORE(-2.86)[ip: (-9.66), ipnet: 2607:f8b0::/32(-2.47), asn: 15169(-2.10), country: US(-0.05)]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[mahrens@delphix.com,matthew.ahrens@delphix.com]; RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2] Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 Oct 2019 19:51:12 -0000 At this month's meeting we discussed: zpool ddtload; xattrs status; OpenZFS repo; ZoF status; staging branch; release notes preview; early meeting time= . Video is available here: https://youtu.be/hL31yMN6Eq0 Detailed notes (thanks Serapheim): - Dev Summit - Please register if you haven't already - almost at capacity - Recorded and live-stream (same service as last year) - zpool ddtload - A new subcommand proposed by Will Andrews - Use-case: Large DDT table - after reboot write-performance is bad because we have to read from the DDT - Proposed Solution: The new command will preload the DDT in the ARC on-demand - Current Status: Looking into plugging it in zpool wait so we know when it is finished and looking for reviewers. PR Link: https://github.com/zfsonlinux/zfs/pull/9464 - xattrs update: - Update from Gordon Ross: - Still gathering information. Currently looking for contacts and survey of how things happen today, especially for Windows and OSX (Jorgen Lundman is probably a good point of contact for that). - ACLs and DOS-style attribute bits are also an issue, but less critical. They are stored in a platform-agnostic way, but some platforms don=E2=80=99t support them so this state is not visible on all platforms. Just a heads up, we are still trying to document those. - Question: What is the status of OpenZFS repo - no sync with illumos a long time? - Currently planning to retire that repo - an email will be sent to let everyone know. The plan is to reinstate a repo with the same name (OpenZFS) but based on ZoL (and ZoF soon) - There is no real-deadline or schedule for this. Probably going to have it finalized once ZoF is part of ZoL and we have CI setup - More info on this and the status of ZoF at the Dev Summit - Status of ZoF (ZFS on FreeBSD using ZoL codebase) (Post-meeting update): - Code is mostly written. - Working on upstreaming to ZoL. Probably at least a few more months to finish that. - Testing - ZTS runs on ZoF prototype - Still need to hook up buildbot to FreeBSD - Would like to upgrade buildbot - Igork - Integration process idea: - Problem: The master branch has shown to have test regressions (or even panics) in the past. This is especially bad when you want to introduce a change and your PR fails for unrelated issues. - Idea: Have a separate branch (like a staging branch) and collect weekly/bi-weekly changes on it and test it thoroughly before merging it to 'master' (or potentially use different tag). Ideally we'd want a bran= ch with no sporadic failures. - People agree that this definitely has been an issue in the past, but the proposed solution may increase the maintenance/coordination/relea= se burden of our release/branching process. Additionally, testing may be incomplete as people test against a clean build but not the final pro= duct where everything will be merged together. This could surface new bugs= and merging issues. - Another way would be to look into ways that we can tackle the same problem by having different kinds of testing, reverting commits faste= r, etc.. - Igork: It would be nice to see the drafts of the release notes of future versions. Helps people plan their upgrades accordingly for they things t= hat matter to them. - We'd have to talk to the folks at Livermore who are currently organizing all the release logistics. Use of Milestones by Brian in Github achieves part of that but it is not a full solution to the problem. I= n general it would be nice to communicate more about branching and our release cycle. - Matt will work with Brian and others on this starting from the next release (transparent and communicate better, so we don't have to ping Brian personally) - Possibility to move the "early" meeting even earlier (request from Ubuntu ZFS people in the UK who have family commitments at our usual tim= es) - Current pattern: 2 at a later time for folks in Asia and 2 at an earlier time for folks in Europe. Proposal: 9am (2 hrs earlier) - Most folks seem to agree that this is acceptable On Mon, Oct 14, 2019 at 10:39 AM Matthew Ahrens wrote= : > It's only 3 weeks until the OpenZFS Developer Summit. Time to register i= f > you have not already! > http://open-zfs.org/wiki/OpenZFS_Developer_Summit_2019 > > The next OpenZFS Leadership meeting will be held tomorrow, October 15th, > 11am-noon Pacific time. > > Everyone is welcome to attend and participate, and we will try to keep th= e > meeting on agenda and on time. The meetings will be held online via Zoom= , > and recorded and posted to the website and YouTube after the meeting. > > The agenda for the meeting will be a discussion of the projects listed in > the agenda doc. > > For more information and details on how to attend, as well as notes and > video from the previous meeting, please see the agenda document: > > > https://docs.google.com/document/d/1w2jv2XVYFmBVvG1EGf-9A5HBVsjAYoLIFZAnW= HhV-BM/edit > > --matt > From owner-zfs-devel@freebsd.org Tue Nov 12 20:44:27 2019 Return-Path: Delivered-To: zfs-devel@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 162091BA92E for ; Tue, 12 Nov 2019 20:44:27 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: from mail-vk1-xa2a.google.com (mail-vk1-xa2a.google.com [IPv6:2607:f8b0:4864:20::a2a]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 47CKVL20SPz4544 for ; Tue, 12 Nov 2019 20:44:26 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: by mail-vk1-xa2a.google.com with SMTP id 70so12034vkz.8 for ; Tue, 12 Nov 2019 12:44:25 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=delphix.com; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to; bh=s2xe5H7P5Hqn7N+1t8bpD1JixojzCrBiNgilRTZscUU=; b=WgEuEGfUe5L6YqeVjO9H1k8fxgWAJHXze/lgpyHYkqsnjJmKcqr+afTJl1I3tK17Q7 ISiINozcMtNSvvQneJSRmX3a049TJjsDRUX/6ApwO9fykHx1sQ3pWDTaCaGS1ctL7K/8 koLbSfjRBBAv8umq4bQ2gARuYc7jOdmTi2/Ao= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=s2xe5H7P5Hqn7N+1t8bpD1JixojzCrBiNgilRTZscUU=; b=FZvY1UEUf3rAbsp8U6IXeRc+ry+QBuRmV/uuxM6A+SGeweiRtzsiqz4kowIlhdmYvU Ja05lajIdSf6PAiUjN4tvIo8mZ5/i89aTwyfU7sNIDD21LXihOv+g2p3cOEIETuW9wSW fvverD6RpyURWcjI/iPRA5Eur1oQxuqEks7EaI8z6AqMhM6pZZhwirEG2w3hnJSptPD7 fEImySLHn9uZwTOyC4GSMFmBHNrHLdnqfyrH9TSUsdPWYveeWaca3U7/sLHFg9aahrPq sIBH6V/fLTyeS2LJp/y7mkKUb/8ZaC52AIH9JyokEfePvtHEImU8snSfzkGwtSNTXkei Zykw== X-Gm-Message-State: APjAAAU1nB+IpWDNpePv5a6xuLo+LRYPQ/z7we7mqxY00y0aKOwCVPgV rT05hV23iPP1XDJqckPLm/nTQDAh1wp5DIPuOcxV+A== X-Google-Smtp-Source: APXvYqxC6AYGWZlkQM4V7/AmFBUvqWeGHAg9FOPfYbQce+rIn/Q2G4Qgp2h+ZgtpY1ESa55pewaCshhFsvey2o2LJ1o= X-Received: by 2002:a1f:18ca:: with SMTP id 193mr22569828vky.66.1573591464585; Tue, 12 Nov 2019 12:44:24 -0800 (PST) MIME-Version: 1.0 References: In-Reply-To: From: Matthew Ahrens Date: Tue, 12 Nov 2019 12:44:13 -0800 Message-ID: Subject: November OpenZFS Leadership Meeting To: developer , illumos-zfs , zfs-devel@freebsd.org, freebsd-fs X-Rspamd-Queue-Id: 47CKVL20SPz4544 X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=delphix.com header.s=google header.b=WgEuEGfU; dmarc=pass (policy=none) header.from=delphix.com; spf=pass (mx1.freebsd.org: domain of matthew.ahrens@delphix.com designates 2607:f8b0:4864:20::a2a as permitted sender) smtp.mailfrom=matthew.ahrens@delphix.com X-Spamd-Result: default: False [-4.49 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[delphix.com:s=google]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[zfs-devel@freebsd.org]; TO_DN_SOME(0.00)[]; URI_COUNT_ODD(1.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[delphix.com:+]; DMARC_POLICY_ALLOW(-0.50)[delphix.com,none]; RCVD_IN_DNSWL_NONE(0.00)[a.2.a.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; FORGED_SENDER(0.30)[mahrens@delphix.com,matthew.ahrens@delphix.com]; MIME_TRACE(0.00)[0:+,1:+,2:~]; IP_SCORE(-2.79)[ip: (-9.58), ipnet: 2607:f8b0::/32(-2.33), asn: 15169(-2.00), country: US(-0.05)]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[mahrens@delphix.com,matthew.ahrens@delphix.com]; RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Nov 2019 20:44:27 -0000 The next OpenZFS Leadership meeting will be held today, November 12th, 1pm-2pm Pacific time. Today Jason King will give his postponed talk from the OpenZFS Developer Summit, titled "Securing the Cloud with ZFS Encryption" Everyone is welcome to attend and participate, and we will try to keep the meeting on agenda and on time. The meetings will be held online via Zoom, and recorded and posted to the website and YouTube after the meeting. The agenda for the meeting will be a discussion of the projects listed in the agenda doc. For more information and details on how to attend, as well as notes and video from the previous meeting, please see the agenda document: https://docs.google.com/document/d/1w2jv2XVYFmBVvG1EGf-9A5HBVsjAYoLIFZAnWHhV-BM/edit --matt From owner-zfs-devel@freebsd.org Mon Nov 25 17:10:41 2019 Return-Path: Delivered-To: zfs-devel@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id B81A01B2CD8 for ; Mon, 25 Nov 2019 17:10:41 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: from mail-vs1-xe32.google.com (mail-vs1-xe32.google.com [IPv6:2607:f8b0:4864:20::e32]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 47MD7h3sNjz4f0P for ; Mon, 25 Nov 2019 17:10:40 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: by mail-vs1-xe32.google.com with SMTP id a143so10584721vsd.9 for ; Mon, 25 Nov 2019 09:10:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=delphix.com; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to; bh=063Uc6waFVxIi0BGKixcULHx9XKjOdCNmGuZq02UZ+s=; b=F6IR2XFTk6+ImPcgKrz49VlM6AgheZWhefNg+2Y53tzc7ycG3aJx4XTWzchI8PqUmO d+6OOhaKWgc2pnAg5FIM/IyA7OCvJXInfiTbE3Rs8kEvaP69kp7UrlRBl7By+VUFXvk2 KDS+tMBCu2MVIMq0/Ban1imp6MXho7zfqtC00= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=063Uc6waFVxIi0BGKixcULHx9XKjOdCNmGuZq02UZ+s=; b=bX531DtWGXonIVwG9dz/AYGaOD/zuGYPQ9jIoYNRpAy04W/Y8aRfAD25sUXgPVtmil OL+fGk/sl9yHMEXCl/mpt/9O4XBkGH/CsAs4GRCNeSKqgXQTV6qVo9ltAYEVKCZRLS1D 3ftI2H+4A7tLTQ1CoCblpMDQYgVOMfAeVW/cvKg4eYyrcqm/Ue2XlQcBIiDBXGIeqlLU cIS6XibGw+3KutY67G16OWxOYMVKLUANvnN5FUJ8pUsT2FmgnBKYF0Uw9bCMP/SRra5L ua62Glkk7V4ILiXum3SK0xlXMAEQaumxH78ISeZDhD05H60KZ5KGCOKmITQPth7lee8b zjMQ== X-Gm-Message-State: APjAAAXKjxDZ5gX5CI/nxMdsv5Lu56qUjOZCHTR2Vf0sg4BkZktkIU6p 4WSepD9JwquHPX1Agx+ApemLRgIoq0nQe7yZO2S4jW6W X-Google-Smtp-Source: APXvYqy1mNYkHxAbviJZqYwwVrKqqs7ZDE9W+EVeqFVnZHeoka1msIATtvBYFmI/QcRna6UwRzYkrn2ZAFRTjzfLIns= X-Received: by 2002:a67:b302:: with SMTP id a2mr7455665vsm.237.1574701838184; Mon, 25 Nov 2019 09:10:38 -0800 (PST) MIME-Version: 1.0 References: In-Reply-To: From: Matthew Ahrens Date: Mon, 25 Nov 2019 09:10:26 -0800 Message-ID: Subject: Re: November OpenZFS Leadership Meeting To: developer , illumos-zfs , zfs-devel@freebsd.org, freebsd-fs X-Rspamd-Queue-Id: 47MD7h3sNjz4f0P X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=delphix.com header.s=google header.b=F6IR2XFT; dmarc=pass (policy=none) header.from=delphix.com; spf=pass (mx1.freebsd.org: domain of matthew.ahrens@delphix.com designates 2607:f8b0:4864:20::e32 as permitted sender) smtp.mailfrom=matthew.ahrens@delphix.com X-Spamd-Result: default: False [-5.48 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[delphix.com:s=google]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[zfs-devel@freebsd.org]; TO_DN_SOME(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[delphix.com:+]; DMARC_POLICY_ALLOW(-0.50)[delphix.com,none]; RCVD_IN_DNSWL_NONE(0.00)[2.3.e.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; FORGED_SENDER(0.30)[mahrens@delphix.com,matthew.ahrens@delphix.com]; MIME_TRACE(0.00)[0:+,1:+,2:~]; IP_SCORE(-2.78)[ip: (-9.59), ipnet: 2607:f8b0::/32(-2.27), asn: 15169(-1.96), country: US(-0.05)]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[mahrens@delphix.com,matthew.ahrens@delphix.com]; RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2] Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 25 Nov 2019 17:10:41 -0000 Meeting recording available here: https://youtu.be/WKfS3lAmLGc At this month's meeting we discussed: new ZoL minimum kernel version; ZoL repo move/rename; issue tracker curatioon Meeting notes (thanks Serapheim): - Rescheduled OpenZFS DevSummit talk: Securing the Cloud with ZFS Encryption (Jason King) - New ZoL minimum kernel version (Brian) - Change: Increase the minimum version to 3.10 kernel (from 2.6) - Practical implication: 1. Dropping some support for very old enterprise releases of Linux 2. Net removal of around 2,000 lines of code - Timeline: will not roll out until OpenZFS 2.0 version - ZoL repo move/rename (Matt) - From ZFSonLinux to OpenZFS (mentioned at the conference during Matt= =E2=80=99s Talk) - Transfer ownership feature on Github is to be used (all PRs, issues, even URLs should be moved/redirected to the new home) 1. If anyone has had a bad experience with this feature, please let us now - Should be a non-event for people doing development - Currently targeting to do this by the end of the year - Notes: We will keep all the existing ZoL resources like email list and IRC channel under the same name (ZFSonLinux, not OpenZFS) - OpenZ= FS mailing list will remain the canonical place for developer discussions for new features. Leave user-centered questions or platform-specific issues to per-platform mailing lists - Issue tracker curation (Matt) - Request for input from folks: Several issues that have been filed against ZoL end up being open for a long time and contain a lot of off-topic discussion (some getting close to an actual flamewar). What should we do about this? - Consequences of this issue: 1. Discourages new members from getting involved with the community 2. It is counter-productive for existing members- especially the ones who can fix the actual issue - Potential ways forward: 1. Ignore these messages (status quo) 2. Try to respond to every message (re-iterate the current status, explain the situation or dependencies on other issues, etc..) - showing that we understand the situation - diffusing the situation a bit 3. Hide the off-topic comments - this could backfire a bit and cause more impolite messages. 4. A combination of 2 & 3 - where they do get a reply and then we hide the messages. - Idea for dealing with incoming bugs: 1. After initial triaging or skimming through the issue, assign a specific tag or component and have a certain group of folks responsible for each component. 2. Have more folks help with labeling incoming issues - Unfortunately (and incredibly) github permissions only allow people with write access to the repo to do this. - Discussion 1. The most popular =E2=80=9Cpotential ways forward=E2=80=9D were i (= status quo) and ii (try to respond to every comment) - Action Item: Brian and Matt will draft a proposal for this and present it in the next meeting and the mailing list. On Tue, Nov 12, 2019 at 12:44 PM Matthew Ahrens wrote= : > The next OpenZFS Leadership meeting will be held today, November 12th, > 1pm-2pm Pacific time. > > Today Jason King will give his postponed talk from the OpenZFS Developer > Summit, titled "Securing the Cloud with ZFS Encryption" > > Everyone is welcome to attend and participate, and we will try to keep th= e > meeting on agenda and on time. The meetings will be held online via Zoom= , > and recorded and posted to the website and YouTube after the meeting. > > The agenda for the meeting will be a discussion of the projects listed in > the agenda doc. > > For more information and details on how to attend, as well as notes and > video from the previous meeting, please see the agenda document: > > > https://docs.google.com/document/d/1w2jv2XVYFmBVvG1EGf-9A5HBVsjAYoLIFZAnW= HhV-BM/edit > > --matt > From owner-zfs-devel@freebsd.org Mon Dec 9 21:23:56 2019 Return-Path: Delivered-To: zfs-devel@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id C00DB1DF253 for ; Mon, 9 Dec 2019 21:23:56 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: from mail-vs1-xe29.google.com (mail-vs1-xe29.google.com [IPv6:2607:f8b0:4864:20::e29]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 47Wx5R5nY7z44mN for ; Mon, 9 Dec 2019 21:23:55 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: by mail-vs1-xe29.google.com with SMTP id f8so11468544vsq.8 for ; Mon, 09 Dec 2019 13:23:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=delphix.com; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to; bh=ctnfTghfm6FLbVNsl3FcTHRtUsjctAsHJuXAiMPXLvE=; b=DYaA0f3gbkpEYxI9Kpv+bTdGpZ9MyUrq2oFlIgOl+qezmSvN1ujr9uTVsF5eg7Ebow /uAk15zsKCFHUdFTApimXlmkHIMpoWJ4a01EOj4toVizyv0Rz2sutjjlW1UlCW3s9Z7w Dmy95ViMyx6zjX+NGhoZtiW7SetoPW/3ZNxyc= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=ctnfTghfm6FLbVNsl3FcTHRtUsjctAsHJuXAiMPXLvE=; b=F5SectvgqsoNBVhUjLyTa06L32AzchY7cGeI+ZB8XYkD1iT8SgFy0ziP9Xm+fUoYlt QcZajYebYt8BFR/po00hsywyWYdTHISPMNnpwNUrmg5Rqury+P4nHVq3Qnm6tzFkphsi HjLaG41YB0oN3b7y7r7D3vB36eSN+WtV61qweOmknUdGVd/En/kCP20OdlhSLEpRF4Ic Ylh7Lewh3e25RIE3AJq4fV1cA58qdULML3qvsqAusilljdW4+UyaWL6y6LwGXg/uRaI/ p92bNc81g7FC4d2k8x/kyHqTalJt6zG3/ZWhq2qIg7hwHslMqYtFpNdE4MlI8QL8KCNF yXpw== X-Gm-Message-State: APjAAAU4NkvCiabDFdW7iW09x+ZG4r8w7+MBmF9gzbkjJYQIzn8vDDry 6po6udvUWHLgQejdqDjXGxGvkiavi9sBb14bPtbtH4WFMxQ= X-Google-Smtp-Source: APXvYqwvMPY1X/DURlES5j26nf9uQh8WXFALG+qwS/PB4E+/FndBzg0bqEoK+apuT4uYb41xpFsD5n6OuIiduwpwCwU= X-Received: by 2002:a05:6102:405:: with SMTP id d5mr22274425vsq.94.1575926634351; Mon, 09 Dec 2019 13:23:54 -0800 (PST) MIME-Version: 1.0 References: In-Reply-To: From: Matthew Ahrens Date: Mon, 9 Dec 2019 13:23:43 -0800 Message-ID: Subject: December OpenZFS Leadership Meeting To: developer , illumos-zfs , zfs-devel@freebsd.org, freebsd-fs X-Rspamd-Queue-Id: 47Wx5R5nY7z44mN X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=delphix.com header.s=google header.b=DYaA0f3g; dmarc=pass (policy=none) header.from=delphix.com; spf=pass (mx1.freebsd.org: domain of matthew.ahrens@delphix.com designates 2607:f8b0:4864:20::e29 as permitted sender) smtp.mailfrom=matthew.ahrens@delphix.com X-Spamd-Result: default: False [-4.46 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[delphix.com:s=google]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[zfs-devel@freebsd.org]; TO_DN_SOME(0.00)[]; URI_COUNT_ODD(1.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[delphix.com:+]; DMARC_POLICY_ALLOW(-0.50)[delphix.com,none]; RCVD_IN_DNSWL_NONE(0.00)[9.2.e.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; FORGED_SENDER(0.30)[mahrens@delphix.com,matthew.ahrens@delphix.com]; MIME_TRACE(0.00)[0:+,1:+,2:~]; IP_SCORE(-2.76)[ip: (-9.61), ipnet: 2607:f8b0::/32(-2.22), asn: 15169(-1.92), country: US(-0.05)]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[mahrens@delphix.com,matthew.ahrens@delphix.com]; RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 09 Dec 2019 21:23:56 -0000 The next OpenZFS Leadership meeting will be held tomorrow, December 10, 1pm-2pm Pacific time. The agenda is pretty light this month, so if you have any topics you'd like to discuss with the group, tomorrow will be a good opportunity :-) Everyone is welcome to attend and participate, and we will try to keep the meeting on agenda and on time. The meetings will be held online via Zoom, and recorded and posted to the website and YouTube after the meeting. The agenda for the meeting will be a discussion of the projects listed in the agenda doc. For more information and details on how to attend, as well as notes and video from the previous meeting, please see the agenda document: https://docs.google.com/document/d/1w2jv2XVYFmBVvG1EGf-9A5HBVsjAYoLIFZAnWHhV-BM/edit --matt From owner-zfs-devel@freebsd.org Sat Dec 14 05:36:14 2019 Return-Path: Delivered-To: zfs-devel@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id DE10A1E2596 for ; Sat, 14 Dec 2019 05:36:14 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: from mail-vs1-xe33.google.com (mail-vs1-xe33.google.com [IPv6:2607:f8b0:4864:20::e33]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 47Zbqf02whz40Nd for ; Sat, 14 Dec 2019 05:36:13 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: by mail-vs1-xe33.google.com with SMTP id x4so938144vsx.10 for ; Fri, 13 Dec 2019 21:36:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=delphix.com; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to; bh=H6GJ+ON70GcZG4318HucCqpmLnZ9XRHn2WWi2vPHLOw=; b=bTPluC8QVjybtJdJ20LMOyBq7o/hSyJvEJl6cFnmjOrl0k10vdjEE7jAeQaoneh9Wm 8FR95UgTi8V0da6rmzvLii+VDamJ4iYw2QheYcD59i4PZnMkwG4yaONuGd7XaZ7waUi0 SX/tjvZsRv8eXiuFj69Mexiu9pEO69VtkRUJo= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=H6GJ+ON70GcZG4318HucCqpmLnZ9XRHn2WWi2vPHLOw=; b=Q0EW2+bg3yKg/smQLt/kcf91vETn45jjKYwkBsFVD8g05ZsEvTwVCu2WQ0+krNX1/w nxbNBHf6BpnTcQlUYE3nohzX6KDs9wQBD625Ok1/5D2TlFp8W11T+wuk2tJFbqx++/KI TU6dkdpbbzMAgE8Uc9zg7ojCtKEZdVROGRvsT2FISeg8r4qKVpAv53BK2HeZv6F2h4Cb ubbXtgNogQS2VrH1heC3vegy64K96xGV+I7H+BaQuisiqxRS4HfxZnP4pF0D9FkOPu6E 3TWuLgcE4RMxf0ZGxgxXw6Qx1DvT8vv+vo3o5fLV2d0jPuD7d83bxCUkR+OPB9S0YfbB yINw== X-Gm-Message-State: APjAAAXZWwHYiXzuz6tuMnmyLzOmapow5c1NOisFEmE3uyU8lsRXaE+j U+vIB63W5ppAyugg1KahuDigJd98NtgKMwXyTR6RFEJcple7EQ== X-Google-Smtp-Source: APXvYqwOjhLYk4G3ESS3583FbyXdOp05PSGIzBeFc/GLWW6+IbyhaVWOwyVVYwsWLXmxXTEYhwewzLMMi/I8r0CSgTk= X-Received: by 2002:a05:6102:405:: with SMTP id d5mr13358648vsq.94.1576301772495; Fri, 13 Dec 2019 21:36:12 -0800 (PST) MIME-Version: 1.0 References: In-Reply-To: From: Matthew Ahrens Date: Fri, 13 Dec 2019 21:36:01 -0800 Message-ID: Subject: Re: December OpenZFS Leadership Meeting To: developer , illumos-zfs , zfs-devel@freebsd.org, freebsd-fs X-Rspamd-Queue-Id: 47Zbqf02whz40Nd X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=delphix.com header.s=google header.b=bTPluC8Q; dmarc=pass (policy=none) header.from=delphix.com; spf=pass (mx1.freebsd.org: domain of matthew.ahrens@delphix.com designates 2607:f8b0:4864:20::e33 as permitted sender) smtp.mailfrom=matthew.ahrens@delphix.com X-Spamd-Result: default: False [-4.42 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[delphix.com:s=google]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[zfs-devel@freebsd.org]; TO_DN_SOME(0.00)[]; URI_COUNT_ODD(1.00)[11]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[delphix.com:+]; DMARC_POLICY_ALLOW(-0.50)[delphix.com,none]; RCVD_IN_DNSWL_NONE(0.00)[3.3.e.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; FORGED_SENDER(0.30)[mahrens@delphix.com,matthew.ahrens@delphix.com]; MIME_TRACE(0.00)[0:+,1:+,2:~]; IP_SCORE(-2.72)[ip: (-9.46), ipnet: 2607:f8b0::/32(-2.20), asn: 15169(-1.91), country: US(-0.05)]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[mahrens@delphix.com,matthew.ahrens@delphix.com]; RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2] Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 14 Dec 2019 05:36:14 -0000 Thanks to everyone who participated at this month's meeting. We discussed: - saved send feature - ZSTD - Encrypted dedup The recording is now available: https://www.youtube.com/watch?v=3DWq64VoZhD= MY Thanks Serapheim for taking notes: - Saved send feature (Tom) - Be able to get a partially received dataset from one machine to another - do a ZFS =E2=80=9Csend of a partially received dataset=E2= =80=9D - Add-on functionality: ability to resume a saved receive from a bookmark - Status: Testing is almost done and Datto will soon start using it in production - Looking for extra reviewers for the PR - https://github.com/zfsonlinux/zfs/pull/9007 - Paul Dagnelie and Matt Ahrens will take a look - Action Item: Send a heads up in the mailing list about the feature for other platforms - ZSTD rework (WIP PR ) (Kjeld) - The goal of this new PR is to get the best of all the existing forks of this feature - Status: Sebastien Gottschall (Brainslayer) have completed the majority of the design and thorough testing has been done. Code is be= ing cleaned up and restructured. - Action Item: Sync with Allan who has also done some work with this - Action item: Get reviewers and feedback for the change - Feature Request: Encryption to work with dedup across multiple datasets - Tom: - Today different =E2=80=9Cclone families=E2=80=9D have different maste= r keys (the key actually used to encrypt the blocks), so blocks with the same plainte= xt will have different cyphertext if they are in the same clone family - even if they have the same wrapping (user) key (i.e. same/inherited keysou= rce property) - Want to add a mechanism to have the same master key for different clone families - Need to design user interface to make it clear what=E2=80=99s going o= n. - Suggestion: use a property to indicate that all children have the same master key - Need to work out the details of how this would interact with things like rename (into / out of the =E2=80=9Csame master key hie= rarchy=E2=80=9D) - Pull Request Open for Persistent L2ARC in ZoL - Brian Behlendorf: - This is a port of Sasso=E2=80=99s/Nexenta=E2=80=99s work (PR 9582) - Reviews requested - ZoF update: - Getting closer! - Bulk of refactoring is done ~ approximately 4 files left to go On Mon, Dec 9, 2019 at 1:23 PM Matthew Ahrens wrote: > The next OpenZFS Leadership meeting will be held tomorrow, December 10, > 1pm-2pm Pacific time. The agenda is pretty light this month, so if you > have any topics you'd like to discuss with the group, tomorrow will be a > good opportunity :-) > > Everyone is welcome to attend and participate, and we will try to keep th= e > meeting on agenda and on time. The meetings will be held online via Zoom= , > and recorded and posted to the website and YouTube after the meeting. > > The agenda for the meeting will be a discussion of the projects listed in > the agenda doc. > > For more information and details on how to attend, as well as notes and > video from the previous meeting, please see the agenda document: > > > https://docs.google.com/document/d/1w2jv2XVYFmBVvG1EGf-9A5HBVsjAYoLIFZAnW= HhV-BM/edit > > --matt > From owner-zfs-devel@freebsd.org Mon Jan 6 17:56:46 2020 Return-Path: Delivered-To: zfs-devel@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 74BBD1D9C1C for ; Mon, 6 Jan 2020 17:56:46 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: from mail-ua1-x931.google.com (mail-ua1-x931.google.com [IPv6:2607:f8b0:4864:20::931]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 47s39T2YQcz41Ky for ; Mon, 6 Jan 2020 17:56:45 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: by mail-ua1-x931.google.com with SMTP id c7so14464088uaf.5 for ; Mon, 06 Jan 2020 09:56:45 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=delphix.com; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to; bh=XBc0m+1x/su3e8BNgOuPfqgt2usqQmZRfvcGuYMx1ws=; b=ZRRary1qRSobyk3pDxlD/zlZYRNG4kDMCec++r+3PBwbHp7VYGOjIdlDNZxfoEWjVD ZgrhyRPbklXXIbA1hJ58xzBQ1WBl8Mql6Kqr7cNkezz48krxy2YKvJl3Te9IlIY+c95U 2RLWtTQiD1l0xMerui/cWlHjezNkjs70rzCbg= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=XBc0m+1x/su3e8BNgOuPfqgt2usqQmZRfvcGuYMx1ws=; b=VqDXbkYQPmtdiU+8TTJs7NeDwUkmBUhdETO2awvwI9dHJoZpIdNb3tALm+e8cb3MUL h5TJFrub1Hrg7fmFJ/hifj+W6LzN4cIfVTp35K8kn7+pkd62lUf+sog3rfvIg+MAbYMS YQ9w5ubwrBqlySRhp8ajsX0VaPvZ0Vg93rP+RDNvpUwITOmrsZ3QqTzJaUvfuYAhFn7J SPizh3SczRPWCltQ3DvdGVm7oVcSbho4/TJUoBBGcmTNCECdZnyIqghsJBj+NlP21Qlm KOk62JkgFd9ogCO7h6EQOR2Kxi7taaG7gRcqcV3bb+pI1D9f/Pjf6gb+tN0xbNgy8P2s VHxA== X-Gm-Message-State: APjAAAW7ELlKARP0utROpajAC7jI/t+sSXVOUpGkAS8nFNVkfjt4+W/l OtpDfq4Ls6j7EMQylD8/DtzsC2tvuetI28CiMa4mjw== X-Google-Smtp-Source: APXvYqyZLiuLwYqgBfQTjxc6JofKfCwYNYX40u5wt4pqpWkAV8l1E/8BDxK+m23vr9LYYLXTHu+aaVTmPQ8r9dFrD6E= X-Received: by 2002:ab0:4aca:: with SMTP id t10mr57667613uae.89.1578333403587; Mon, 06 Jan 2020 09:56:43 -0800 (PST) MIME-Version: 1.0 References: In-Reply-To: From: Matthew Ahrens Date: Mon, 6 Jan 2020 09:56:32 -0800 Message-ID: Subject: January OpenZFS Leadership Meeting To: developer , illumos-zfs , zfs-devel@freebsd.org, freebsd-fs X-Rspamd-Queue-Id: 47s39T2YQcz41Ky X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=delphix.com header.s=google header.b=ZRRary1q; dmarc=pass (policy=none) header.from=delphix.com; spf=pass (mx1.freebsd.org: domain of matthew.ahrens@delphix.com designates 2607:f8b0:4864:20::931 as permitted sender) smtp.mailfrom=matthew.ahrens@delphix.com X-Spamd-Result: default: False [-4.33 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[delphix.com:s=google]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36:c]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[zfs-devel@freebsd.org]; TO_DN_SOME(0.00)[]; URI_COUNT_ODD(1.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[delphix.com:+]; DMARC_POLICY_ALLOW(-0.50)[delphix.com,none]; RCVD_IN_DNSWL_NONE(0.00)[1.3.9.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; FORGED_SENDER(0.30)[mahrens@delphix.com,matthew.ahrens@delphix.com]; MIME_TRACE(0.00)[0:+,1:+,2:~]; IP_SCORE(-2.63)[ip: (-9.14), ipnet: 2607:f8b0::/32(-2.13), asn: 15169(-1.85), country: US(-0.05)]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[mahrens@delphix.com,matthew.ahrens@delphix.com]; RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 06 Jan 2020 17:56:46 -0000 The next OpenZFS Leadership meeting will be held tomorrow, January 7, 9am-10am Pacific time. The agenda is pretty light this month, so if you have any topics you'd like to discuss with the group, tomorrow will be a good opportunity :-) Everyone is welcome to attend and participate, and we will try to keep the meeting on agenda and on time. The meetings will be held online via Zoom, and recorded and posted to the website and YouTube after the meeting. The agenda for the meeting will be a discussion of the projects listed in the agenda doc. For more information and details on how to attend, as well as notes and video from the previous meeting, please see the agenda document: https://docs.google.com/document/d/1w2jv2XVYFmBVvG1EGf-9A5HBVsjAYoLIFZAnWHhV-BM/edit --matt From owner-zfs-devel@freebsd.org Fri Jan 17 17:15:29 2020 Return-Path: Delivered-To: zfs-devel@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id E600F1F4A66 for ; Fri, 17 Jan 2020 17:15:29 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: from mail-vs1-xe31.google.com (mail-vs1-xe31.google.com [IPv6:2607:f8b0:4864:20::e31]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 47znkn1X5Sz3Jsl for ; Fri, 17 Jan 2020 17:15:28 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: by mail-vs1-xe31.google.com with SMTP id s16so15257172vsc.10 for ; Fri, 17 Jan 2020 09:15:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=delphix.com; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to; bh=N0t1km/d25Tf82lyO/sHI4Df0Y0qIplhYu3EcOgwDmM=; b=Gj1dISKV3duFlECC590qUcagmgY3bXncGImoLmcTsc4YhKQbDqOD9eoMwtJ2Vkz1DY CE4+atjEfqPyC7n69A93B2iQur1PmxH8Kt+ohPSc3sloXOc71ZmaSaI+7AIlKWGwCdNP oid2GZuwSrxnbmeSI9ghjZzq1qtDTdMTYlt6E= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=N0t1km/d25Tf82lyO/sHI4Df0Y0qIplhYu3EcOgwDmM=; b=FTB3L7kHYIXfLL1Ml2JEwzc2JQ1v8l3X8549fSo9Ly8kRkY1I7C9Z9+zJyYD7rXwY+ dpt/T6019FmvYJB10k3Rch6PmSJ7t3PfYyBo+MWS5m4hOwtRlESf9i24tlA1uCTmHHgs 01tDeBr5XyrnVn3j7kcAjGtwFEq++2qCTgUeyjwMWVVE4rIfb4787aVwpa5w5g8PtSdh zTxxv9kF9h5PTVVERlEvgPvQooD9fTpWA0QCTBubI61ro4riafCH3S+Hjo4WZcthYiqH KxPGAWscz8pXpBFFTgHk1jhWKw/8kSNaYkJkw/BkAHs5fm15H79wryNaJUl06JfK8NoG pxXQ== X-Gm-Message-State: APjAAAWY3MSdz+XViDnczG1svBHNLFrUy8nENCT7M9MJ6oYpCAugzevI m+mGpNOlkBONO4gUIHUHNAMzOF/liQeC68Rdid3qcw== X-Google-Smtp-Source: APXvYqxBLk9IGY98sUv2aXj6cmKyWYfs4QO0+Cr5IHL6vcuWBlX1qp8vLzguyWoUsSrefyeSzGiDoxD990XqzPrljg4= X-Received: by 2002:a67:cd16:: with SMTP id u22mr5549146vsl.56.1579281327145; Fri, 17 Jan 2020 09:15:27 -0800 (PST) MIME-Version: 1.0 References: In-Reply-To: From: Matthew Ahrens Date: Fri, 17 Jan 2020 09:15:14 -0800 Message-ID: Subject: Re: January OpenZFS Leadership Meeting To: developer , illumos-zfs , zfs-devel@freebsd.org, freebsd-fs X-Rspamd-Queue-Id: 47znkn1X5Sz3Jsl X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=delphix.com header.s=google header.b=Gj1dISKV; dmarc=pass (policy=none) header.from=delphix.com; spf=pass (mx1.freebsd.org: domain of matthew.ahrens@delphix.com designates 2607:f8b0:4864:20::e31 as permitted sender) smtp.mailfrom=matthew.ahrens@delphix.com X-Spamd-Result: default: False [-5.43 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[delphix.com:s=google]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[zfs-devel@freebsd.org]; TO_DN_SOME(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[delphix.com:+]; DMARC_POLICY_ALLOW(-0.50)[delphix.com,none]; RCVD_IN_DNSWL_NONE(0.00)[1.3.e.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; FORGED_SENDER(0.30)[mahrens@delphix.com,matthew.ahrens@delphix.com]; MIME_TRACE(0.00)[0:+,1:+,2:~]; IP_SCORE(-2.73)[ip: (-9.67), ipnet: 2607:f8b0::/32(-2.09), asn: 15169(-1.83), country: US(-0.05)]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[mahrens@delphix.com,matthew.ahrens@delphix.com]; RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2] Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 17 Jan 2020 17:15:30 -0000 Thanks to everyone who participated at this month's meeting. We discussed: - ZoL 0.8.3 contents and schedule - checksum algorithm feature flags refcount bug - zfs change-key security implications - changing default encryption algorithm to GCM The recording is now available: https://www.youtube.com/watch?v=3Dx9-wua_mz= t0 Thanks Serapheim for taking notes: - When will 0.8.3 ship and will it include the large-dnode =E2=80=9Czfs di= ff=E2=80=9D fix (FireSnake) - Currently there is an outstanding patch PR for 0.8.3 (#9776) - it should be landing when we wrap up our testing so very soon (probably within this month!) - The goal is to have it pulled in for Ubuntu and Debian LTS - Let us know if there is a patch that you really want to see there - Changing the checksum algorithm at an existing dataset - even if no new blocks have been written - exporting and then importing that pool from a different version of ZFS hits an assertion error. [Allan Jude] - That was an oversight on the design of feature flags. - We should be able to handle this more gracefully. - A few ideas were thrown including: - Bump feature refcount for every dataset that has the property set - Being able to open the pool but not that dataset (property would show as =E2=80=9Cunsupported=E2=80=9D) - Any further comments on this Ubuntu developer's proposal to enable encryption by default with a fixed passphrase: https://bugs.launchpad.net/bugs/1857398 (rlaager) - Is there any additional feedback that we haven=E2=80=99t already prov= ided them? Feel free to add in the thread. - Tangent on secure-erase and changing encryption keys: - Secure erase and not having dataset=E2=80=99s being partially secu= re were two of the main motivators on the current encryption design. - The latter is ensured by setting encryption on at creation but not guaranteed as the encryption key may later be changed and will affect only new blocks. - Even if encryption key (user/wrapping key) is changed, new blocks can be read/manipulated if the old (compromised/public) key is known, and the old-key-wrapped master key is found (e.g. by forensic analysis of the disk). - The trade-off being usability and practicality varies wildly between cases. - Given the above tangent, we should really understand what the Canonical folks want to do and try to come up with the best practises= and design. This includes communicating best practices, and potentially implementing something different than what we currently have. - We need to be more clear about the security implications of =E2=80=9C= zfs change-key=E2=80=9D - PR filed - Change encryption=3Don from aes-256-ccm to aes-256-gcm? See especially t= he comments starting here: https://github.com/zfsonlinux/zfs/pull/9749#issuecomment-568633557 (rlaager) - The two main motivators of this proposal are security and performance= . - From a security standpoint, Mozilla and TLS default to gcm. - According to Richard=E2=80=99s estimates, performance could get a = ~3x improvement with gcm. - There seems to be an overall consensus on this but we should really check with Tom Caputi. - encryption: from_ivset_guid check missing on resumed recv? (Christian Schwarz) - turned into issue post-meeting, do not include in upcoming agenda - bookmark cloning & zcp bookmark support PR (Christian Schwarz) - open design question: what about redaction bookmarks? - reviewers needed once above question is resolved - outcome: - redaction object might be too large to copy it in syncing context, would have to be done in the background - =3D> don=E2=80=99t implement redaction bookmark cloning at this po= int - =3D> create thin bookmark: zfs bookmark #redactionbook #newbook - will not contain the redaction object itself On Mon, Jan 6, 2020 at 9:56 AM Matthew Ahrens wrote: > The next OpenZFS Leadership meeting will be held tomorrow, January 7, > 9am-10am Pacific time. The agenda is pretty light this month, so if you > have any topics you'd like to discuss with the group, tomorrow will be a > good opportunity :-) > > Everyone is welcome to attend and participate, and we will try to keep th= e > meeting on agenda and on time. The meetings will be held online via Zoom= , > and recorded and posted to the website and YouTube after the meeting. > > The agenda for the meeting will be a discussion of the projects listed in > the agenda doc. > > For more information and details on how to attend, as well as notes and > video from the previous meeting, please see the agenda document: > > > https://docs.google.com/document/d/1w2jv2XVYFmBVvG1EGf-9A5HBVsjAYoLIFZAnW= HhV-BM/edit > > --matt > From owner-zfs-devel@freebsd.org Fri Jan 17 18:42:06 2020 Return-Path: Delivered-To: zfs-devel@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 092B61F73C1 for ; Fri, 17 Jan 2020 18:42:06 +0000 (UTC) (envelope-from rlaager@wiktel.com) Received: from smtp1.wiktel.com (smtp1.wiktel.com [IPv6:2600:2600::151]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 47zqfj00Zjz3QqM for ; Fri, 17 Jan 2020 18:42:04 +0000 (UTC) (envelope-from rlaager@wiktel.com) Received: from [172.16.0.114] (thief-pool2-122-35.mncable.net [24.225.122.35]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp1.wiktel.com (Postfix) with ESMTPSA id 47zqfX4jbYzxF4; Fri, 17 Jan 2020 12:41:56 -0600 (CST) Subject: Re: [developer] Re: January OpenZFS Leadership Meeting To: developer@lists.open-zfs.org, zfs@lists.illumos.org, zfs-devel@freebsd.org References: From: Richard Laager Autocrypt: addr=rlaager@wiktel.com; prefer-encrypt=mutual; keydata= xsFNBF2mXIMBEADD6qOWoPZwMsV9XijFhHh+IeeKS6FG0xTe+aNny8ZjNsiDPPlJVIZz+udg p0FwGnB/QjTl1k/BhfUuuFbO8ZQii9eHbFrpqApnKYmRXh64TNApdYAAim5gmwdw0IGEfsp2 gjJHi4ems+PPbikf9xtPjiz1EjVSaLsNF3vo9cADpgUa62Ba9kxmZ2PBw5rWttbwlXOhpHlh rv96pGRXvsP/h2ih+bOsASkx1sV/7gykNyJlDkKK6sFCSBG7hIClj5g8c+Myx+cxYC9qDUa1 mZtpjfU+fy4xQ0X6x0b22pOdKJGNfYBSslRiAa5WYSidU0MQ+s6YNftYbW8/0qSGu6OF4DzN 9lMMxIT3bzny+lb7YQPDl00XkuCZyCfk1k5VLi1VPAq4DJAybNgxBXx5yD7Sl916P1L9xy7Z yqYrTLC/Te3kOc3WuGDCyY3Qv7f4ZblIWNscG13fHKLD+UNJMJObAoqBs5Ovc+tFpqUuacsl OI+dKr6Sg+3izFIVIwaAOKyKHeynaEIviJe6r2cc4ZCSlff+jH9pA/jZ+bq+SrWMm4OwWR8m KNAA8tS9H3bXdVqrD0oy9c1I7tNa2btQxKz7TstTVAsjzuYFN2R4FLNDYjZt5RMSmItspW5d 7mEfDp1cQAEn1nNdmVsR3ETaV/78tpaFeaV9mJV7zPchkgM9NwARAQABzSJSaWNoYXJkIExh YWdlciA8cmxhYWdlckBwaWRnaW4uaW0+wsGrBBMBCgA+FiEE1Ot9lOeOTujs4H+U+HlhmcBF hs4FAl2mXN4CGwMFCRLMAwAFCwkIBwMFFQoJCAsFFgIDAQACHgECF4AAIQkQ+HlhmcBFhs4W IQTU632U545O6Ozgf5T4eWGZwEWGzsDgEAClg2zJLcEO4aNO77NSJmdQ+rNa/87+A4v3TXEx 1OqyJ90rZfk/qzvS67jCF9qXMR7ic54zQ5fHf3+NuL0ZOXzu9CA+7HYmnq6UHqWYl4YHARj6 MEpbhGWW5I7IuCxNWTxNndst2lxwhDDQEOg4SkWAf+WtcrewFeYn7A3xTrvZxyRLcoQRPWwI i6CKFgHM27GYIbvChDr7J9EKwuY3Rsqke6Xs4wwUE4XAHn88OtxPrU5Zz6UyI8soDLk8QrTb zB3s6rjcNsn2bWOWaOscHoEi0Nkm9VJqBlYdPOYar4zm1odc9cf75UF3Hq6dYhiQV7LGY8Qj zUpQANetcCDBaG+fOdAES/6l1ArewRgqFcIvEkoMudvJNwy1oCB7pmYmn3SaJsaN9rk30k8c lh2LmFudrBl+SFZPpLbSQCZvTDakJPdSqABb6o/H1vbIkqQCjilA1PGqj2JJNHdsihEwn+qx ZUMTScjXlxXw8JdhMWCR0C6QOyiWNxqMKyFCBbnF++/rdPRdYqNtW+yHwIt+NpjjuWtguyNC e1UXAjISBLKKcGL2MfG02nLhaOq7YxnjUMDHqDyJ+fXa8FOdttvspVB6TURFcGW2OHI0oO9L 5NINXqUAREu3zk4LCswI63bMFUYTi5DNeCbNGB3bmxUXoQZt9aDOlJGnJzzhsre+a+Mc6s7B TQRdplyDARAAuISKrKfmcRa1O0ChF18fUAAc9OEf26PjuHq76eWCGPn52FQiQFH5myMcV23e oHqfJvobDUQp+VMdiwd90FAYLTi5kiqdFHk29h3OEk2DQZzJulD5szRzvBdrbTZyxKUG4bIv VWQuhWxzJr758iVxNRFgJn7VJy+n5AXOvpVJ8MQY3sxuWT/GhbScm07UOJr3S6XRCezJ1/So YmCZvMh9VyKlTro/nSLIA08nzJJYfovD/iLCIOhwjW3gjoDQl8OZntHccu09g1+6QY6664Y5 2VY+yGjjkFtvrW6FwT3ZzFYGHWP4kXmXnyBtd9xKsrSyh2+4FUfsRYKu1piSR89AgyGkeNgv vSNM8mGJH3FEBLavb6/woIc5ZJk/+FM1RdOZ2cMRfu8AL1OPGXeWp5tChBYEiTAO4Qp61JH6 VM3ngFb2i7ho+O861QXUodII+N4jDVeHVqhwcNeRD2WNbfCpqC24YNGP4WexLPyGIW5BNr34 3hfg61Sc23NAAv9LGb1ux7+m1pEg+2BBFKPo0PBw8LtKsFHU93elWX78qD88uhhrsV+R17NZ LwJvehFMftOyY1FzLCFVOjxMlMT8Ki0kC8MxhaC4r60Y3049mFuCYyyQ/kmA4ytv5nbBNxSu 6ElbmAoYnS+MGqFJg6JYiFNg0uEQ2WWENSHZmG6Qq+ywhKsAEQEAAcLBkwQYAQoAJhYhBNTr fZTnjk7o7OB/lPh5YZnARYbOBQJdplyDAhsMBQkSzAMAACEJEPh5YZnARYbOFiEE1Ot9lOeO Tujs4H+U+HlhmcBFhs5JYg//ZXJ6ZUxGdIHF+OruI+ShN6vcncQv3a4LUqhoBo11vdbrI29k UXib6GU0IkhA/2YF2p9wIWCkN3d2BhqV45Gh13x7IlcLYRdFHuw8iJ0aGcpYM/viwOD0wd5b ezJ/fl7G7PdxOi3Yb/dTpICAdz9JxJuI6bHDob46ZVispS8hiHPnxxpTDk3mmF7mSSZesu6U KEw3UnA2nXa0ccvERjSsYm+/HVkwIGi6jT4IRZunsJpmicdK0MbR6dz0VuR79I2hb79XCqoM RxwaS4CIeAK11mE9cn9qhjx8XOh8XCq6xBQrxwb1fNJxKTQ2ToxLcdpy9wYSe20IKhkt2JkG RpYLRRDWZ4S2waBhmLNWTTa18AUMz0NxVxlu2RGG5fmGEZ5qwgLX2e4GzJWQFY23eCH8eg+w 7vz8DLNqgomm5x/CqyRENjf/Vqx0vaq4nthp9Y55uhzS49N9hTlmZ7Ob3OCzkIW8Q4q+lL/s 9b4PxIHXAvw9DZa9Jf3CDO9ODNnF2IMXw/bTvV4DD4DbQ270TE3jeaXL/Bp8DcLOgTltE8aK yiqOT4QmgbCjjarzVT0EZQQ8MHG7DyiRqWiVOfjOELfiRyF3nJH7ns0nY5uK4518aDZP7YR6 hC6135at3RVvx3XSdROy85FhP86Es+nJuIhzwQ7u1/X6gCNPfjNuZ55TYGE= Message-ID: <0b9d336f-c18c-e937-9ace-9575ada26223@wiktel.com> Date: Fri, 17 Jan 2020 12:41:56 -0600 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.4.1 MIME-Version: 1.0 In-Reply-To: Content-Language: en-US X-bounce-key: wiktel.com-1; rlaager@wiktel.com; 1579286516; oM2ILarpkvUwYkt9AWDMJY6osUU; DKIM-Signature: v=1; a=rsa-sha256; c=relaxed; d=wiktel.com; h=subject:to :references:from:message-id:date:mime-version:in-reply-to :content-type; s=wiktel1; bh=ZwTbybTjMFZX5lIzq/2Kjp4CwU5vLJBA7ki AVQYsXeE=; b=jh8oUVo/6vqQ3TfCAblQwJyP6KfkoExafjL3ExG6S8Ym9srfYK2 mqzy1VOoEKD5Vnlr8EIjXb8O2rh832jUMJ0LSy84hXR1X11zOPEwCMhzJuNkm5ZM 6VUl62/hjC9vM3XsYOpocZscel2JFRiAmoL4Zl9okfM4Lvs75nJc8w9HP2tsrkPX YCjrJmhSkTy+Oajz2CIMHln3VXIkJDXGtUGvsAFgwkZyEp8o73YpA0qukxOjd/ji wSyu51ZyeKHq4hYwcasRR5GAyryrQSSBxsnnnBkPMaJy3WMkXEt7MIWKfu4vmejY yOzlZ8/0bfylqQxUwTToZQpLDrhseudVioA== X-Spam-Level: X-Rspamd-Queue-Id: 47zqfj00Zjz3QqM X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; dkim=pass header.d=wiktel.com header.s=wiktel1 header.b=jh8oUVo/; dmarc=none; spf=pass (mx1.freebsd.org: domain of rlaager@wiktel.com designates 2600:2600::151 as permitted sender) smtp.mailfrom=rlaager@wiktel.com X-Spamd-Result: default: False [0.93 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_DKIM_ALLOW(-0.20)[wiktel.com:s=wiktel1]; NEURAL_HAM_MEDIUM(-0.60)[-0.603,0]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; R_SPF_ALLOW(-0.20)[+ip6:2600:2600::/64]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; TO_DN_NONE(0.00)[]; DMARC_NA(0.00)[wiktel.com]; URI_COUNT_ODD(1.00)[13]; DWL_DNSWL_NONE(0.00)[wiktel.com.dwl.dnswl.org : 127.0.5.0]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[wiktel.com:+]; MIME_BASE64_TEXT(0.10)[]; NEURAL_SPAM_LONG(0.94)[0.939,0]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; IP_SCORE(-0.01)[country: US(-0.05)]; ASN(0.00)[asn:33362, ipnet:2600:2600::/28, country:US]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2] Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 17 Jan 2020 18:42:06 -0000 On 1/17/20 11:15 AM, Matthew Ahrens wrote: > > Change encryption=on from aes-256-ccm to aes-256-gcm? See especially > the comments starting here: > https://github.com/zfsonlinux/zfs/pull/9749#issuecomment-568633557(rlaager) > > * > > The two main motivators of this proposal are security and performance. > > o > > From a security standpoint, Mozilla and TLS default to gcm. > > o > > According to Richard’s estimates, performance could get a ~3x > improvement with gcm. > One minor nit, it's really "Attila Fülöp's estimates", not mine. I don't want to be seen as stealing credit for someone else's (excellent!) work. :) I was repeating Attila Fülöp's comments from PR 9749: GCM is 1.15x the speed of CCM before PR 9749: "I did run the fio tests above on an aes-256-gcm and an aes-256-ccm dataset and the GCM run is approximately 1.15 times faster than the CCM run." -- https://github.com/zfsonlinux/zfs/pull/9749#issuecomment-569132997 The PR gives "up to approximately 12x throughput increase for large (128 KiB) blocks." See the Description section in the PR description: https://github.com/zfsonlinux/zfs/pull/9749 "If there's enough interest I could be beaten to port the openssl CCM assembler routines too, but the improvements won't be as big as in the GCM case. Here is the output of openssl speed indicating that GCM performs 3-4 times faster then CCM." https://github.com/zfsonlinux/zfs/pull/9749#issuecomment-570065780 I did test just now to confirm those results personally. With OpenSSL, I'm seeing GCM as 2.6x to 4.8x the speed of CCM, depending on block size. You can test on your system with:     openssl speed -evp aes-256-gcm     openssl speed -evp aes-256-ccm -- Richard From owner-zfs-devel@freebsd.org Mon Feb 3 21:26:52 2020 Return-Path: Delivered-To: zfs-devel@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id C97C022F17E for ; Mon, 3 Feb 2020 21:26:52 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: from mail-vs1-xe2b.google.com (mail-vs1-xe2b.google.com [IPv6:2607:f8b0:4864:20::e2b]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 48BLVz4QhYz48hq for ; Mon, 3 Feb 2020 21:26:51 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: by mail-vs1-xe2b.google.com with SMTP id x123so9979944vsc.2 for ; Mon, 03 Feb 2020 13:26:51 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=delphix.com; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to; bh=oGPI1G76MfucQt4zbJaLh17p8v4kplJrFFRnywj5GDQ=; b=GgtAqFSVoDFjUyNS0tc40D+fAdSJTrK+kMwIVvxD/bT5fPkbYoqmZYv/H/hTxXUndN Nt+6WM5kaKUFlxDG0WjOjHflxIuUb1m2Uu+chHyp30Iup9Ojdn0isxcxzoS7gH9R2wHc LwQVcWzzcUNLcyUlxRvc29RJ4n0aZZLKoAbxg= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=oGPI1G76MfucQt4zbJaLh17p8v4kplJrFFRnywj5GDQ=; b=jOgMZ3r3h/mr//29CGS7S0TxkMoygB9S335ezz5C5uCsNralxR+lKCmzPfOW3XfRFe B+WdjTNI/D8ryLHhAR2EOnpccZNRjjqfwDXMoBlWFDn+bGd5mISVTty6pwOrA0zJ6NBF sGgoBYUHXxl5FEiSRmx/pRSD9Np+7V5FhlWo80A835FsWcpbFzP2g5cHIcznvMitr88v bZcjrXrymjokKUjMUpY4Rsc86T+qzbFL86HwhbjJ9FLma77TP97JHcd880yR2+Ts4U2+ Txc840BnlBxIoQi18iuHfIdK5VW9XXeEbqZ9w8NPsXfafWfzK2jqPrM1MCPoRkIXNi18 noKQ== X-Gm-Message-State: APjAAAW8UpaQHaEflF5t0+xVbHCV3orZw3/tLR19RFdSUu/B0cqMVtUF wt8yL3cgK7eLw5XPA4w9qplz86ONMsptkd5j/WR0ksd8idA= X-Google-Smtp-Source: APXvYqzFAOQavSpsO7qV1+yFjj5lhPEanvkX0PG6JmUcQsA2MNixinM4qdWIkyOnQ4EgNv3KADETdDwP1Cdlsb1W6Zs= X-Received: by 2002:a05:6102:405:: with SMTP id d5mr15685500vsq.94.1580765210464; Mon, 03 Feb 2020 13:26:50 -0800 (PST) MIME-Version: 1.0 References: In-Reply-To: From: Matthew Ahrens Date: Mon, 3 Feb 2020 13:26:39 -0800 Message-ID: Subject: February OpenZFS Leadership Meeting To: developer , illumos-zfs , zfs-devel@freebsd.org, freebsd-fs X-Rspamd-Queue-Id: 48BLVz4QhYz48hq X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=delphix.com header.s=google header.b=GgtAqFSV; dmarc=pass (policy=none) header.from=delphix.com; spf=pass (mx1.freebsd.org: domain of matthew.ahrens@delphix.com designates 2607:f8b0:4864:20::e2b as permitted sender) smtp.mailfrom=matthew.ahrens@delphix.com X-Spamd-Result: default: False [-4.36 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[delphix.com:s=google]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[zfs-devel@freebsd.org]; TO_DN_SOME(0.00)[]; URI_COUNT_ODD(1.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[delphix.com:+]; DMARC_POLICY_ALLOW(-0.50)[delphix.com,none]; RCVD_IN_DNSWL_NONE(0.00)[b.2.e.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; FORGED_SENDER(0.30)[mahrens@delphix.com,matthew.ahrens@delphix.com]; MIME_TRACE(0.00)[0:+,1:+,2:~]; IP_SCORE(-2.66)[ip: (-9.51), ipnet: 2607:f8b0::/32(-1.99), asn: 15169(-1.75), country: US(-0.05)]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[mahrens@delphix.com,matthew.ahrens@delphix.com]; RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Feb 2020 21:26:52 -0000 The next OpenZFS Leadership meeting will be held tomorrow, February 4, 1pm-2pm Pacific time. The agenda is pretty light this month, so if you have any topics you'd like to discuss with the group, tomorrow will be a good opportunity :-) Everyone is welcome to attend and participate, and we will try to keep the meeting on agenda and on time. The meetings will be held online via Zoom, and recorded and posted to the website and YouTube after the meeting. The agenda for the meeting will be a discussion of the projects listed in the agenda doc. For more information and details on how to attend, as well as notes and video from the previous meeting, please see the agenda document: https://docs.google.com/document/d/1w2jv2XVYFmBVvG1EGf-9A5HBVsjAYoLIFZAnWHhV-BM/edit --matt From owner-zfs-devel@freebsd.org Wed Feb 5 16:56:50 2020 Return-Path: Delivered-To: zfs-devel@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 8CA4024EF25 for ; Wed, 5 Feb 2020 16:56:50 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: from mail-ua1-x931.google.com (mail-ua1-x931.google.com [IPv6:2607:f8b0:4864:20::931]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 48CSQT4kXhz3GqZ for ; Wed, 5 Feb 2020 16:56:49 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: by mail-ua1-x931.google.com with SMTP id 73so1107660uac.6 for ; Wed, 05 Feb 2020 08:56:49 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=delphix.com; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to; bh=0IHS+78JDbreLg2QZzOKIUcrAFfMl5153QK/EOkptWU=; b=e45y7dAeNtFKf7kjlrq6y70LYgF1ReO8gcPzjAkqiidytNjadppppsGSoO5dvWo0MV 3yh9FfFdClhc/mhF9kw2/Iu2R5fgdMgDHCEJAggBlJwkcYLpJ0h5nodAOKZmGgRmsjdo eylFZ83h6ovzMCS2kIt5ylBBpxGDfvjxszvd4= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=0IHS+78JDbreLg2QZzOKIUcrAFfMl5153QK/EOkptWU=; b=gzNjanDdWoVbdwofKUbBOz9RIM45/Y99MnYhkwllH6gPqDpQ1ZyAbb49uy2ImvW/wt YicdjmzJpGYm+WnpNzzTDhabtSdssEF4RyJ8vajCtEIC67n6aEtfqLKOaBfni07k89YM 3Ha/1km430kAqzQfQVSqfbxTnNQk3Z1BhmtU8wxTSrqj+tE2n9Q0YDrSEuO2Lnk+SU3Q lLVxfH3XvzkhQUGU50WEbJ2EXsbEewqSnpfInGvvk2UlHrXm+ed07zDRKOsoN03bYBc3 EUL1PJ3dfisco7pRMu3Vyf/b/R0DIVpQxdpn2sd2vKGd1JpeyujWa+xVwvr/qOEsbLoy sm0Q== X-Gm-Message-State: APjAAAV/90nBPE0IUUUnONanJhn3BoCWyGK2b5ufqq7vS++yh88JRKab hE2IKDTU+MFTTo9VoMH2TdnQsDzqkMoNrjajft4Y4A== X-Google-Smtp-Source: APXvYqyVtlaOnI2BSgmGFzcJ7aOsugp9o5ZWXuSWdHMAeOXoC1sSRlOn4OWdTtu7vqSP1XmcexcS25h6WQTEtw5hvTY= X-Received: by 2002:a9f:3e84:: with SMTP id x4mr20369004uai.83.1580921808157; Wed, 05 Feb 2020 08:56:48 -0800 (PST) MIME-Version: 1.0 References: In-Reply-To: From: Matthew Ahrens Date: Wed, 5 Feb 2020 08:56:37 -0800 Message-ID: Subject: Re: February OpenZFS Leadership Meeting To: developer , illumos-zfs , zfs-devel@freebsd.org, freebsd-fs X-Rspamd-Queue-Id: 48CSQT4kXhz3GqZ X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=delphix.com header.s=google header.b=e45y7dAe; dmarc=pass (policy=none) header.from=delphix.com; spf=pass (mx1.freebsd.org: domain of matthew.ahrens@delphix.com designates 2607:f8b0:4864:20::931 as permitted sender) smtp.mailfrom=matthew.ahrens@delphix.com X-Spamd-Result: default: False [-4.28 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[delphix.com:s=google]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[zfs-devel@freebsd.org]; TO_DN_SOME(0.00)[]; URI_COUNT_ODD(1.00)[9]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[delphix.com:+]; DMARC_POLICY_ALLOW(-0.50)[delphix.com,none]; RCVD_IN_DNSWL_NONE(0.00)[1.3.9.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; FORGED_SENDER(0.30)[mahrens@delphix.com,matthew.ahrens@delphix.com]; MIME_TRACE(0.00)[0:+,1:+,2:~]; IP_SCORE(-2.58)[ip: (-9.11), ipnet: 2607:f8b0::/32(-1.98), asn: 15169(-1.74), country: US(-0.05)]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[mahrens@delphix.com,matthew.ahrens@delphix.com]; RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2] Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Feb 2020 16:56:50 -0000 Thanks to everyone who participated at this month's meeting. We discussed: - Thread priorities on linux - Feature flag activation bug - Moving of the OpenZFS website to University of Washington - FOSDEM, Scale, and other conferences The recording is now available: https://www.youtube.com/watch?v=3DupUOXBhx-= us Thanks Serapheim for taking notes: - Thread priorities on linux (Paul D) - Context: Performance analysis of ZFS send on ZFSonLinux found discrepancies with illumos - Root cause: Linux the threads are minimum priority, lower than user threads (as opposed to Illumos) - Workaround: Increased the priority for now - Question: How do we want to deal with this among different OS=E2=80= =99s? - FreeBSD has the same priority scheme as illumos (Linux being the outlier of the 3) - In Linux there were performance issues related with thread priorities in the past but there hasn=E2=80=99t been any recent in= vestigations - There is no silver bullet for this - Consensus: - There are not many thread cases whose performance we don=E2=80=99t= care about but we should at least break down threads into different groups and decide on the priority of each group - This issue is to be added to the list of issues that we need an open PR on GIthub for discussion - Do we need to enhance the feature flag activation code. Setting checksum=3Dsha512 or compress=3Dzstd but not writing any blocks, can cau= se the pool to panic on older systems - Summary: Setting the compression to zstd but you don=E2=80=99t write = any blocks, and then re-import that pool in a ZFS version that doesn=E2= =80=99t understand zstd triggers an assertion failure - Consensus: - Bump the counter when the feature gets activated AND for each block created with it (refcount never goes to zero until the dataset is destroyed). - Although seemingly difficult to implement we should at least pay a close look at the code paths as it may not be as intractable as it seems. - Support for ignoring (not being able to mount) datasets with unsupported feature flags - Consensus: This is a good idea in general but it should not be considered a solution for the aforementioned zstd problem. - Moving of the OpenZFS website from Joyent to University of Washington is in-progress - No problems with Joyent, just planning for the long-term - No actual website changes, just the DNS endpoint will change - Still would be happy to update any contributions to update the website=E2=80=99s content :) - Changes that need reviewers: - Persistent L2ARC - https://github.com/zfsonlinux/zfs/pull/9582 - Performance Optimization for Encryption - Includes new algorithms and changing the default algorithm used - Action Item: Send an email to the mailing list as a heads up about changing the default before going ahead an applying that change - FOSDEM, Scale, and other conferences - There doesn=E2=80=99t seem to be any talks related to ZFS in the aforementioned conferences but people attending can submit BoF discus= sions - iXsystems will potentially be at Scale this year - Allan Jude has done a ZFS-related talk in the main track of FOSDEM in the past - There will be a ZFS BoF in this year=E2=80=99s BSDCan and potentially= one ZFS-related talk too. - The BSDNow podcast is open to interview people doing interesting work on ZFS - General Idea: It would be good to be active in conferences and groups outside of the OpenZFS summit and BSD - On this note, OpenZFS is considering spinning up funds for people that want to work on that (e.g. setting up a booth on those conferences and getting the word out). On Mon, Feb 3, 2020 at 1:26 PM Matthew Ahrens wrote: > The next OpenZFS Leadership meeting will be held tomorrow, February 4, > 1pm-2pm Pacific time. The agenda is pretty light this month, so if you > have any topics you'd like to discuss with the group, tomorrow will be a > good opportunity :-) > > Everyone is welcome to attend and participate, and we will try to keep th= e > meeting on agenda and on time. The meetings will be held online via Zoom= , > and recorded and posted to the website and YouTube after the meeting. > > The agenda for the meeting will be a discussion of the projects listed in > the agenda doc. > > For more information and details on how to attend, as well as notes and > video from the previous meeting, please see the agenda document: > > > https://docs.google.com/document/d/1w2jv2XVYFmBVvG1EGf-9A5HBVsjAYoLIFZAnW= HhV-BM/edit > > --matt > From owner-zfs-devel@freebsd.org Sat Feb 8 06:19:21 2020 Return-Path: Delivered-To: zfs-devel@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id B21002398F2 for ; Sat, 8 Feb 2020 06:19:21 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 48F27Y4Jwsz4SYp for ; Sat, 8 Feb 2020 06:19:21 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 8B282191F1 for ; Sat, 8 Feb 2020 06:19:21 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 0186JLIW058669 for ; Sat, 8 Feb 2020 06:19:21 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 0186JL0q058659 for zfs-devel@FreeBSD.org; Sat, 8 Feb 2020 06:19:21 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: zfs-devel@FreeBSD.org Subject: [Bug 243973] [zfs] rollback segmentation fault Date: Sat, 08 Feb 2020 06:19:21 +0000 X-Bugzilla-Reason: AssignedTo CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: bin X-Bugzilla-Version: 12.1-RELEASE X-Bugzilla-Keywords: crash, needs-qa X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: koobs@FreeBSD.org X-Bugzilla-Status: Open X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: zfs-devel@FreeBSD.org X-Bugzilla-Flags: mfc-stable12? mfc-stable11? X-Bugzilla-Changed-Fields: flagtypes.name short_desc keywords bug_status cc assigned_to Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 08 Feb 2020 06:19:21 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D243973 Kubilay Kocak changed: What |Removed |Added ---------------------------------------------------------------------------- Flags| |mfc-stable12?, | |mfc-stable11? Summary|[zfs] zfs rollback |[zfs] rollback segmentation |segmentation fault |fault Keywords| |crash, needs-qa Status|New |Open CC| |zfs-devel@FreeBSD.org Assignee|bugs@FreeBSD.org |zfs-devel@FreeBSD.org --=20 You are receiving this mail because: You are the assignee for the bug. You are on the CC list for the bug.= From owner-zfs-devel@freebsd.org Sat Feb 8 07:23:33 2020 Return-Path: Delivered-To: zfs-devel@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 66E4423ACDB for ; Sat, 8 Feb 2020 07:23:33 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 48F3Yd0Kzkz4Vts for ; Sat, 8 Feb 2020 07:23:33 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 06CCB19E8A for ; Sat, 8 Feb 2020 07:23:33 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 0187NWUq088711 for ; Sat, 8 Feb 2020 07:23:32 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 0187NWSR088704 for zfs-devel@FreeBSD.org; Sat, 8 Feb 2020 07:23:32 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: zfs-devel@FreeBSD.org Subject: [Bug 243973] [zfs] rollback segmentation fault Date: Sat, 08 Feb 2020 07:23:33 +0000 X-Bugzilla-Reason: CC AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: bin X-Bugzilla-Version: 12.1-RELEASE X-Bugzilla-Keywords: crash, needs-qa X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: avg@FreeBSD.org X-Bugzilla-Status: Open X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: zfs-devel@FreeBSD.org X-Bugzilla-Flags: mfc-stable12? mfc-stable11? X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 08 Feb 2020 07:23:33 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D243973 --- Comment #2 from Andriy Gapon --- Kubilay, I think that zfs-devel@ list is effectively dead. --=20 You are receiving this mail because: You are on the CC list for the bug. You are the assignee for the bug.= From owner-zfs-devel@freebsd.org Sun Feb 9 08:38:15 2020 Return-Path: Delivered-To: zfs-devel@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 6941F231527 for ; Sun, 9 Feb 2020 08:38:15 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 48Fj9M2CdPz3Lvp for ; Sun, 9 Feb 2020 08:38:15 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 476473760 for ; Sun, 9 Feb 2020 08:38:15 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 0198cFdH098673 for ; Sun, 9 Feb 2020 08:38:15 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 0198cFM9098672 for zfs-devel@FreeBSD.org; Sun, 9 Feb 2020 08:38:15 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: zfs-devel@FreeBSD.org Subject: [Bug 243973] [zfs] rollback segmentation fault Date: Sun, 09 Feb 2020 08:38:15 +0000 X-Bugzilla-Reason: AssignedTo CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: bin X-Bugzilla-Version: 12.1-RELEASE X-Bugzilla-Keywords: crash, needs-qa X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: koobs@FreeBSD.org X-Bugzilla-Status: Open X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: fs@FreeBSD.org X-Bugzilla-Flags: mfc-stable12? mfc-stable11? X-Bugzilla-Changed-Fields: assigned_to cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 09 Feb 2020 08:38:15 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D243973 Kubilay Kocak changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|zfs-devel@FreeBSD.org |fs@FreeBSD.org CC|zfs-devel@FreeBSD.org | --- Comment #3 from Kubilay Kocak --- Apologies, mistake in drop-down selection, thanks for the heads-up, and everyone (@freebsd.org) can triage :) --=20 You are receiving this mail because: You are the assignee for the bug. You are on the CC list for the bug.= From owner-zfs-devel@freebsd.org Tue Mar 3 04:48:09 2020 Return-Path: Delivered-To: zfs-devel@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 605DB265107 for ; Tue, 3 Mar 2020 04:48:09 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: from mail-vs1-xe34.google.com (mail-vs1-xe34.google.com [IPv6:2607:f8b0:4864:20::e34]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 48WkzB3XtTz4TtB for ; Tue, 3 Mar 2020 04:48:06 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: by mail-vs1-xe34.google.com with SMTP id r18so1546861vso.5 for ; Mon, 02 Mar 2020 20:48:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=delphix.com; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to; bh=HB6ch5cjivpO3Gu9ya0yjrn/NEM+2EGFS4dflwqjHjQ=; b=PAW04eWYWe/+6E2QR7LLj1EW1sr0GY4dnyzEPNDsRPTB4gRcJPteP0wRTeh7L7mRbt jkRE2757uks7+aChY156qVC7uNmGqUsk7AnAdCF3IKCNmESQCB6hI0WE8XigroUVR/it KipHQikjHdWCyHc2G6HnNSYMVENGgV+zE5s18= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=HB6ch5cjivpO3Gu9ya0yjrn/NEM+2EGFS4dflwqjHjQ=; b=d0mv/9DXAJyBVpn2wEAg8Q+vw3bTkyrv4h3erNn78np05mXnt3hD7Qs3smiZnXr4TI h5Id7YpIaD0U+hgIicIvSghgYc7Est4MBA2nAxreqxZ4q8Jd1C2+YfaeqTHmpJQajodz XMlAQOw0fr7Ztdqxk9103r9VdZ5EDykQvR7cp/lBGja/q5y1sDMvZaBjZ9THZ1dfdj4P bJon24P1LxZajvrHkb5srmfLh3Rwn6ta8NGm8KYA1ZDA8q7wu0wQoXMuB0AkIN6skxaf Zu2EhhuU+TKu4V3JJXhgh/cQF3doGOHDPFBUDskEnxbdV3TfMBhD6aB23gqi3+yWk4xc Qp7A== X-Gm-Message-State: ANhLgQ0mwDbO1PD2dsh2zFSYH/lt0SbgIujA/WtPmSM+ko78L9m1ZtDy UhpYg5N/AI+4VGrQ9h7qafD+iaufxEsUB2sX9Ki+Ig== X-Google-Smtp-Source: ADFU+vvZUkqkVhZ9i6V23/Sca/ff0OfuPP1MOT2ktFieNOSKLLtAHwQmWEhQx3xqY8YtDcvTuW7MTA/TTl1jjzYaIwI= X-Received: by 2002:a05:6102:405:: with SMTP id d5mr1415139vsq.94.1583210884048; Mon, 02 Mar 2020 20:48:04 -0800 (PST) MIME-Version: 1.0 References: In-Reply-To: From: Matthew Ahrens Date: Mon, 2 Mar 2020 20:47:53 -0800 Message-ID: Subject: March OpenZFS Leadership Meeting To: developer , illumos-zfs , zfs-devel@freebsd.org, freebsd-fs X-Rspamd-Queue-Id: 48WkzB3XtTz4TtB X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=delphix.com header.s=google header.b=PAW04eWY; dmarc=pass (policy=none) header.from=delphix.com; spf=pass (mx1.freebsd.org: domain of matthew.ahrens@delphix.com designates 2607:f8b0:4864:20::e34 as permitted sender) smtp.mailfrom=matthew.ahrens@delphix.com X-Spamd-Result: default: False [-4.34 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.996,0]; R_DKIM_ALLOW(-0.20)[delphix.com:s=google]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[zfs-devel@freebsd.org]; TO_DN_SOME(0.00)[]; URI_COUNT_ODD(1.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[delphix.com:+]; DMARC_POLICY_ALLOW(-0.50)[delphix.com,none]; RCVD_IN_DNSWL_NONE(0.00)[4.3.e.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; FORGED_SENDER(0.30)[mahrens@delphix.com,matthew.ahrens@delphix.com]; MIME_TRACE(0.00)[0:+,1:+,2:~]; IP_SCORE(-2.64)[ip: (-9.63), ipnet: 2607:f8b0::/32(-1.86), asn: 15169(-1.66), country: US(-0.05)]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[mahrens@delphix.com,matthew.ahrens@delphix.com]; RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 03 Mar 2020 04:48:09 -0000 The next OpenZFS Leadership meeting will be held tomorrow, March 3, 1pm-2pm Pacific time. The agenda is pretty light this month, so if you have any topics you'd like to discuss with the group, tomorrow will be a good opportunity :-) Everyone is welcome to attend and participate, and we will try to keep the meeting on agenda and on time. The meetings will be held online via Zoom, and recorded and posted to the website and YouTube after the meeting. The agenda for the meeting will be a discussion of the projects listed in the agenda doc. For more information and details on how to attend, as well as notes and video from the previous meeting, please see the agenda document: https://docs.google.com/document/d/1w2jv2XVYFmBVvG1EGf-9A5HBVsjAYoLIFZAnWHhV-BM/edit --matt From owner-zfs-devel@freebsd.org Wed Mar 4 20:58:51 2020 Return-Path: Delivered-To: zfs-devel@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 2DE36251063 for ; Wed, 4 Mar 2020 20:58:51 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: from mail-ua1-x930.google.com (mail-ua1-x930.google.com [IPv6:2607:f8b0:4864:20::930]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 48XmSn5zkgz3LXX for ; Wed, 4 Mar 2020 20:58:49 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: by mail-ua1-x930.google.com with SMTP id a18so307364uao.1 for ; Wed, 04 Mar 2020 12:58:49 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=delphix.com; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to; bh=F6hjn7z5B0wjNMlT+I7iPUZt8doDaisd/4MZfHFFG10=; b=a9W1R1qNaB6JQJeG2dQd1d0nboyE6oXYV87H4Ek1+cd5DjQVVs4skRdMhYS+B5PXKb Aj5mtbaKMsxabY9SfzFwpTH7+s6bY/ANp5F15SizHOBqOrNmkCMcNsaJlC8BEiZsuPgq Gden68YVkYOewNA7F7H0pTcs+sQ8aag5JRWmA= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=F6hjn7z5B0wjNMlT+I7iPUZt8doDaisd/4MZfHFFG10=; b=kxt+AXjmd9HUzP0P5YUOZXO1Y+TV8r814rlTcxSTtWEY2QMxVOlKumXvtbFJROx9CM Ld/KGBdF347y7goapGBZk5drsZNY+2d6VamAJfXomlEWaRzHLNOjW18hjLpacicjWR5H TMcDTBTfW3N0D11lRW+Mcbap6eU5RpZeykEZe160BzcNHOa7PA58Zk/vKnGCAuyoy2hg m3WTA0QErw3QYDt2sg+nRTE+05lGA9JB1Vka2tjaDUHEhUeZybLeBkBWHsfpntzPR+IB 5yQtSmJYnXlUxCym3xnU7/IRTLvD/9iBktHQyDZ4uWG7HQpvTa7OPhrPdTUlDcbg+U7b DxFQ== X-Gm-Message-State: ANhLgQ1LSmEpjStauVtkuKVZPUoWAxQKvT9eGsx6TCpyEotgww+ib87y B1iiY8KbiPmzdECtzxym1uBRqAtBYzsAlXVw23sz4Ief1K0= X-Google-Smtp-Source: ADFU+vsPqjJsaeoitA3NJRtNftgjhW3T14mE9g9+jxDHgb19eswcHM3u7JGEALp3x9xgQ60Q8k4eqzAGkZ4IUuLu4Eg= X-Received: by 2002:ab0:4aca:: with SMTP id t10mr2694006uae.89.1583355527731; Wed, 04 Mar 2020 12:58:47 -0800 (PST) MIME-Version: 1.0 References: In-Reply-To: From: Matthew Ahrens Date: Wed, 4 Mar 2020 12:58:36 -0800 Message-ID: Subject: Re: March OpenZFS Leadership Meeting To: developer , illumos-zfs , zfs-devel@freebsd.org, freebsd-fs X-Rspamd-Queue-Id: 48XmSn5zkgz3LXX X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=delphix.com header.s=google header.b=a9W1R1qN; dmarc=pass (policy=none) header.from=delphix.com; spf=pass (mx1.freebsd.org: domain of matthew.ahrens@delphix.com designates 2607:f8b0:4864:20::930 as permitted sender) smtp.mailfrom=matthew.ahrens@delphix.com X-Spamd-Result: default: False [-5.30 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[delphix.com:s=google]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36:c]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[zfs-devel@freebsd.org]; TO_DN_SOME(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[delphix.com:+]; DMARC_POLICY_ALLOW(-0.50)[delphix.com,none]; RCVD_IN_DNSWL_NONE(0.00)[0.3.9.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; FORGED_SENDER(0.30)[mahrens@delphix.com,matthew.ahrens@delphix.com]; MIME_TRACE(0.00)[0:+,1:+,2:~]; IP_SCORE(-2.60)[ip: (-9.44), ipnet: 2607:f8b0::/32(-1.86), asn: 15169(-1.66), country: US(-0.05)]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[mahrens@delphix.com,matthew.ahrens@delphix.com]; RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2] Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 04 Mar 2020 20:58:51 -0000 Thanks to everyone who attended this week's meeting. We discussed: - ZFS github repo move to openzfs organization - Progress on FreeBSD support in openzfs repo - directio PR The video recording is now available: https://www.youtube.com/watch?v=3DFVwYAwrKZCU&feature=3Dyoutu.be Meeting notes: - Zfs repo move (Matt) - Which repo should they use: - If FreeBSD / ZoL, you can continue to use the same URL. There should be link forwarding via GitHub, and you shouldn=E2=80=99t ha= ve to do any local updates. - Code coverage links point to the wrong place, so they don=E2=80=99t r= esolve. At least for PRs that were open at the time when the move happened. Probably this won=E2=80=99t come up for new PRs. - FreeBSD progress (Brian) - Over the past few weeks, they=E2=80=99ve been getting all of the test= s running. That=E2=80=99s mostly done. - There is one more big PR open to push the code (new files). Hopefully that will be merged this week or next. - Who is reviewing the code? Are reviewers needed? - FreeBSD folks have done reviews. Not waiting for any reviewers, but speak up if you have an issue. - Won=E2=80=99t turn on CI by default until we know the tests are passi= ng. You may have to manually initiate. - Illumos is watching the progress of FreeBSD closely and will then look at doing something similar with illumos. - Directio (Mark M) - Motivation - Cray wanting to improve performance on NVMe drives - Sequential large-block read/write - Concerned with cost of bcopy to/from userland - With prototype, can saturate up to 12 NVMe drives (up from ~4 without directio) - implications of directly reading/writing - Less bcopy() of buffers - Less memory is allocated (e.g. no dirty data in dbuf cache) - Data is not added to ARC cache - Write happens before call returns (but data is not necessarily persistent if the system crashes) - Are directio reads truly zero-copy? How to compute checksum consistently? A: It isn=E2=80=99t consistent. You could get a =E2=80=9Cfalse=E2=80=9D checksum error (hardware is fine but software caused checksum to be wrong). - Maybe add a flag to the block that indicates that the checksum may be faulty because it was a directio write and userland may = have modified the block while it was being written, causing a checksum error. Then handle checksum errors on these blocks differently. - User interface - Modes - Some mode that=E2=80=99s the same as the current behavior (igno= re DIRECTIO request, do it the old way) - Some mode that will be the new default, which does (at least some) DIRECTIO-requested operations directly. - (perhaps) some mode that does i/o directly even when DIRECTIO is not requested. - What happens to partial-block reads that are DIRECTIO-requested? - Fail? - Fall back to normal behavior, adding block to the ARC? - Do directly, discarding non-requested data? - performance impact to sequential partial-block reads - What happens to partial-block writes that are DIRECTIO-requested? - Fail? - Fall back to normal behavior, adding block to the ARC? - Fall back to normal behavior, but don=E2=80=99t add block to th= e ARC? - I.e. write is buffered until sync context, potentially accumulating more changes, but after it=E2=80=99s written in sync context, we discard it from the ARC/dbuf caches - Do directly (read-modify-write before returning)? - performance impact to sequential partial-block writes - (probably) implementation complexity - Nobody wants to argue against failing partial block writes - How does directio and regular i/o interact? - Inconsistent? - Matt says no - Consistent? - May be complicated to implement On Mon, Mar 2, 2020 at 8:47 PM Matthew Ahrens wrote: > The next OpenZFS Leadership meeting will be held tomorrow, March 3, > 1pm-2pm Pacific time. The agenda is pretty light this month, so if you > have any topics you'd like to discuss with the group, tomorrow will be a > good opportunity :-) > > Everyone is welcome to attend and participate, and we will try to keep th= e > meeting on agenda and on time. The meetings will be held online via Zoom= , > and recorded and posted to the website and YouTube after the meeting. > > The agenda for the meeting will be a discussion of the projects listed in > the agenda doc. > > For more information and details on how to attend, as well as notes and > video from the previous meeting, please see the agenda document: > > > https://docs.google.com/document/d/1w2jv2XVYFmBVvG1EGf-9A5HBVsjAYoLIFZAnW= HhV-BM/edit > > --matt > From owner-zfs-devel@freebsd.org Thu Mar 5 09:22:14 2020 Return-Path: Delivered-To: zfs-devel@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 76FF7263814; Thu, 5 Mar 2020 09:22:14 +0000 (UTC) (envelope-from rlaager@wiktel.com) Received: from smtp1.wiktel.com (smtp1.wiktel.com [69.89.207.151]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 48Y4yX3RNGz40kN; Thu, 5 Mar 2020 09:22:12 +0000 (UTC) (envelope-from rlaager@wiktel.com) Received: from [172.16.0.114] (thief-pool-116-15.mncable.net [24.225.116.15]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp1.wiktel.com (Postfix) with ESMTPSA id 48Y4yT2NJHzxF5; Thu, 5 Mar 2020 03:22:09 -0600 (CST) Subject: ZFS Direct IO (Was: Re: March OpenZFS Leadership Meeting) To: openzfs-developer , developer , illumos-zfs , zfs-devel@freebsd.org, freebsd-fs References: From: Richard Laager Autocrypt: addr=rlaager@wiktel.com; prefer-encrypt=mutual; keydata= xsFNBF2mXIMBEADD6qOWoPZwMsV9XijFhHh+IeeKS6FG0xTe+aNny8ZjNsiDPPlJVIZz+udg p0FwGnB/QjTl1k/BhfUuuFbO8ZQii9eHbFrpqApnKYmRXh64TNApdYAAim5gmwdw0IGEfsp2 gjJHi4ems+PPbikf9xtPjiz1EjVSaLsNF3vo9cADpgUa62Ba9kxmZ2PBw5rWttbwlXOhpHlh rv96pGRXvsP/h2ih+bOsASkx1sV/7gykNyJlDkKK6sFCSBG7hIClj5g8c+Myx+cxYC9qDUa1 mZtpjfU+fy4xQ0X6x0b22pOdKJGNfYBSslRiAa5WYSidU0MQ+s6YNftYbW8/0qSGu6OF4DzN 9lMMxIT3bzny+lb7YQPDl00XkuCZyCfk1k5VLi1VPAq4DJAybNgxBXx5yD7Sl916P1L9xy7Z yqYrTLC/Te3kOc3WuGDCyY3Qv7f4ZblIWNscG13fHKLD+UNJMJObAoqBs5Ovc+tFpqUuacsl OI+dKr6Sg+3izFIVIwaAOKyKHeynaEIviJe6r2cc4ZCSlff+jH9pA/jZ+bq+SrWMm4OwWR8m KNAA8tS9H3bXdVqrD0oy9c1I7tNa2btQxKz7TstTVAsjzuYFN2R4FLNDYjZt5RMSmItspW5d 7mEfDp1cQAEn1nNdmVsR3ETaV/78tpaFeaV9mJV7zPchkgM9NwARAQABzSJSaWNoYXJkIExh YWdlciA8cmxhYWdlckBwaWRnaW4uaW0+wsGrBBMBCgA+FiEE1Ot9lOeOTujs4H+U+HlhmcBF hs4FAl2mXN4CGwMFCRLMAwAFCwkIBwMFFQoJCAsFFgIDAQACHgECF4AAIQkQ+HlhmcBFhs4W IQTU632U545O6Ozgf5T4eWGZwEWGzsDgEAClg2zJLcEO4aNO77NSJmdQ+rNa/87+A4v3TXEx 1OqyJ90rZfk/qzvS67jCF9qXMR7ic54zQ5fHf3+NuL0ZOXzu9CA+7HYmnq6UHqWYl4YHARj6 MEpbhGWW5I7IuCxNWTxNndst2lxwhDDQEOg4SkWAf+WtcrewFeYn7A3xTrvZxyRLcoQRPWwI i6CKFgHM27GYIbvChDr7J9EKwuY3Rsqke6Xs4wwUE4XAHn88OtxPrU5Zz6UyI8soDLk8QrTb zB3s6rjcNsn2bWOWaOscHoEi0Nkm9VJqBlYdPOYar4zm1odc9cf75UF3Hq6dYhiQV7LGY8Qj zUpQANetcCDBaG+fOdAES/6l1ArewRgqFcIvEkoMudvJNwy1oCB7pmYmn3SaJsaN9rk30k8c lh2LmFudrBl+SFZPpLbSQCZvTDakJPdSqABb6o/H1vbIkqQCjilA1PGqj2JJNHdsihEwn+qx ZUMTScjXlxXw8JdhMWCR0C6QOyiWNxqMKyFCBbnF++/rdPRdYqNtW+yHwIt+NpjjuWtguyNC e1UXAjISBLKKcGL2MfG02nLhaOq7YxnjUMDHqDyJ+fXa8FOdttvspVB6TURFcGW2OHI0oO9L 5NINXqUAREu3zk4LCswI63bMFUYTi5DNeCbNGB3bmxUXoQZt9aDOlJGnJzzhsre+a+Mc6s7B TQRdplyDARAAuISKrKfmcRa1O0ChF18fUAAc9OEf26PjuHq76eWCGPn52FQiQFH5myMcV23e oHqfJvobDUQp+VMdiwd90FAYLTi5kiqdFHk29h3OEk2DQZzJulD5szRzvBdrbTZyxKUG4bIv VWQuhWxzJr758iVxNRFgJn7VJy+n5AXOvpVJ8MQY3sxuWT/GhbScm07UOJr3S6XRCezJ1/So YmCZvMh9VyKlTro/nSLIA08nzJJYfovD/iLCIOhwjW3gjoDQl8OZntHccu09g1+6QY6664Y5 2VY+yGjjkFtvrW6FwT3ZzFYGHWP4kXmXnyBtd9xKsrSyh2+4FUfsRYKu1piSR89AgyGkeNgv vSNM8mGJH3FEBLavb6/woIc5ZJk/+FM1RdOZ2cMRfu8AL1OPGXeWp5tChBYEiTAO4Qp61JH6 VM3ngFb2i7ho+O861QXUodII+N4jDVeHVqhwcNeRD2WNbfCpqC24YNGP4WexLPyGIW5BNr34 3hfg61Sc23NAAv9LGb1ux7+m1pEg+2BBFKPo0PBw8LtKsFHU93elWX78qD88uhhrsV+R17NZ LwJvehFMftOyY1FzLCFVOjxMlMT8Ki0kC8MxhaC4r60Y3049mFuCYyyQ/kmA4ytv5nbBNxSu 6ElbmAoYnS+MGqFJg6JYiFNg0uEQ2WWENSHZmG6Qq+ywhKsAEQEAAcLBkwQYAQoAJhYhBNTr fZTnjk7o7OB/lPh5YZnARYbOBQJdplyDAhsMBQkSzAMAACEJEPh5YZnARYbOFiEE1Ot9lOeO Tujs4H+U+HlhmcBFhs5JYg//ZXJ6ZUxGdIHF+OruI+ShN6vcncQv3a4LUqhoBo11vdbrI29k UXib6GU0IkhA/2YF2p9wIWCkN3d2BhqV45Gh13x7IlcLYRdFHuw8iJ0aGcpYM/viwOD0wd5b ezJ/fl7G7PdxOi3Yb/dTpICAdz9JxJuI6bHDob46ZVispS8hiHPnxxpTDk3mmF7mSSZesu6U KEw3UnA2nXa0ccvERjSsYm+/HVkwIGi6jT4IRZunsJpmicdK0MbR6dz0VuR79I2hb79XCqoM RxwaS4CIeAK11mE9cn9qhjx8XOh8XCq6xBQrxwb1fNJxKTQ2ToxLcdpy9wYSe20IKhkt2JkG RpYLRRDWZ4S2waBhmLNWTTa18AUMz0NxVxlu2RGG5fmGEZ5qwgLX2e4GzJWQFY23eCH8eg+w 7vz8DLNqgomm5x/CqyRENjf/Vqx0vaq4nthp9Y55uhzS49N9hTlmZ7Ob3OCzkIW8Q4q+lL/s 9b4PxIHXAvw9DZa9Jf3CDO9ODNnF2IMXw/bTvV4DD4DbQ270TE3jeaXL/Bp8DcLOgTltE8aK yiqOT4QmgbCjjarzVT0EZQQ8MHG7DyiRqWiVOfjOELfiRyF3nJH7ns0nY5uK4518aDZP7YR6 hC6135at3RVvx3XSdROy85FhP86Es+nJuIhzwQ7u1/X6gCNPfjNuZ55TYGE= Message-ID: <94570a94-d64e-6ebd-e314-a97d66b9116e@wiktel.com> Date: Thu, 5 Mar 2020 03:22:03 -0600 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.4.1 MIME-Version: 1.0 In-Reply-To: Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="GCUetWhX8ucNpWXz6cf5fscUr5kb2Pwfz" X-bounce-key: wiktel.com-1; rlaager@wiktel.com; 1583400129; 77B2/wFMpw1qMmO/w/GwTpsZde8; DKIM-Signature: v=1; a=rsa-sha256; c=relaxed; d=wiktel.com; h=subject:to :references:from:message-id:date:mime-version:in-reply-to :content-type; s=wiktel1; bh=tvjyzHh7tHL2/doxWHi4DrFi6tT61ZWgWRV tH3WWOyE=; b=yPGF0lCZcdIo00YWsNN5lETEVsv8pEAgXufo9kXH8mIl18FVBvM uoYmhmsl3sMqNs9JeYgiFEMMyCUg9544SAR0D2aDByAi+5xchAyiQWdsJCy3WxjV PpGTS7zwux7DVTX++utdn6S4St2p/4o96wnmC90PsyTeY4lXQheEsC6HgSJY353l wrilq9u+KjYatXjbgrF09L8PpkHEV0FXxW+nowb6DuNw2nIDSpsTT8hXeVajRgvb ZI7fh+BcYJodjGYxpJLrWC5j172b1EnaJGefLnJsuHe0IcL5Q3s3gN7lPSZnkUkP 0xY4k4ES7dccFGuYMXMLJG2/oNuVX9JCEMQ== X-Spam-Level: X-Rspamd-Queue-Id: 48Y4yX3RNGz40kN X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=wiktel.com header.s=wiktel1 header.b=yPGF0lCZ; dmarc=none; spf=pass (mx1.freebsd.org: domain of rlaager@wiktel.com designates 69.89.207.151 as permitted sender) smtp.mailfrom=rlaager@wiktel.com X-Spamd-Result: default: False [-4.61 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_DKIM_ALLOW(-0.20)[wiktel.com:s=wiktel1]; NEURAL_HAM_MEDIUM(-0.90)[-0.903,0]; FROM_HAS_DN(0.00)[]; RWL_MAILSPIKE_GOOD(0.00)[151.207.89.69.rep.mailspike.net : 127.0.0.18]; R_SPF_ALLOW(-0.20)[+ip4:69.89.207.0/24]; IP_SCORE(-0.01)[country: US(-0.05)]; HAS_ATTACHMENT(0.00)[]; MIME_GOOD(-0.20)[multipart/signed,multipart/mixed,multipart/alternative,text/plain]; DMARC_NA(0.00)[wiktel.com]; TO_DN_SOME(0.00)[]; RCPT_COUNT_FIVE(0.00)[5]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; DWL_DNSWL_NONE(0.00)[wiktel.com.dwl.dnswl.org : 127.0.5.0]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[wiktel.com:+]; SIGNED_PGP(-2.00)[]; RCVD_IN_DNSWL_LOW(-0.10)[151.207.89.69.list.dnswl.org : 127.0.5.1]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:+,3:+,4:~,5:~]; ASN(0.00)[asn:33362, ipnet:69.89.192.0/20, country:US]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 05 Mar 2020 09:22:14 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --GCUetWhX8ucNpWXz6cf5fscUr5kb2Pwfz Content-Type: multipart/mixed; boundary="Dw0TqhjoSpK0jP2jrgHIe3xhax5e4mbT9" --Dw0TqhjoSpK0jP2jrgHIe3xhax5e4mbT9 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable On 3/4/20 2:58 PM, Matthew Ahrens wrote: > > * > > Directio (Mark M) > > o > > User interface > > + > > What happens to partial-block writes that are > DIRECTIO-requested? > > # > > Nobody wants to argue against failing partial block wri= tes > qemu is an application which uses O_DIRECT when configured in its cache=3Dnone mode (and also cache=3Ddirectsync which also uses O_DSYNC): https://documentation.suse.com/sles/11-SP4/html/SLES-kvm4zseries/cha-qemu= -cachemodes.html In this use case (qemu/virtualization), the key desired benefit of O_DIRECT is that it bypasses the host's cache. I think the ideal mapping in this use case is that O_DIRECT should have the same effect as primarycache=3Dmetadata and ZFS should /not/ require recordsize-sized writes (/not/ "fail[] partial block writes"). VMs are generally writing in 512 B or 4 KiB blocks. It is almost certainly not feasible to get the VM to write in e.g. 128 KiB blocks. If the application is required to write in recordsize blocks (and assuming the application does not want to take on the read-modify-write, which I think is the case here), this would force the administrator to set the recordsize to 512 B or 4 KiB. Using a small recordsize like that is detrimental in terms of compression ratios, metadata overhead, raidz space overhead (if you use raidz), etc. A reasonable counter-argument would be that for this use case, the administrator could set the proposed option to use the current direct IO behavior (i.e. just ignore O_DIRECT) and then set primarycache=3Dmetadata= =2E If it turns out that requiring recordsize-sized blocks is enough of a win in other use cases, at least we have a decent fallback for this use case. A further question was raised about what downsides caching has. For the virtualization use case, I've always had a particular concern. Fundamentally, caching here is using the host's RAM to speed up disk IO. For virtualization use cases, the vast majority of host RAM is allocated to guests. This leads to a capacity planning/predictability concern. Imagine I have a host with e.g. 64 GiB of RAM, I've allocated 32 GiB of RAM to guests, and everything is working fine. Can I start another guest that uses 16 GB of RAM? It seems that I should be able to, and if I'm using uncached (on the host) IO, I definitely can. However, if I'm using the host's RAM for caching, taking 16 GB away from the cache could have detrimental performance effects. In other words, I might be (inadvertently) relying upon the caching to provide acceptable IO performance. Eliminating the host cache ensures that all IO caching is occurring in the guest's RAM, which makes RAM allocation easier to understand / more predictable. --=20 Richard --Dw0TqhjoSpK0jP2jrgHIe3xhax5e4mbT9-- --GCUetWhX8ucNpWXz6cf5fscUr5kb2Pwfz Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAEBCgAdFiEE1Ot9lOeOTujs4H+U+HlhmcBFhs4FAl5gxLsACgkQ+HlhmcBF hs7ung/+Ls9r5jCGbPtz2tDOCnqkfAUTiWC0goxKUT1GQiAPqEJvaQvIFNFCo5Gg gY/2nv3kpjdFqn4rFT6VWmNMac1sCFwbeGJYeu9XxQibCQ7GE8J1mA+yCu/STGLm kmcZQzy2shxYftCtfLPQthPNPMtz61OAcY6bI+yMkXwWDFlzu+yHIvXgJ4PnOaR2 dfRivxD9Mj/t+ngvc5d54VT7RE+xCNGzjZDTlxMxH5oHULwSKu8vCap7qxHrJe0y hTO6EFfd55mCmhA1UWTwazgDq/Uz20aH80rGkTGIX6XTOTV6VNxk8y+EHz25vrwE 8acSwGYfwj+6087R8QzW/J4SimAI52jADm895cjJ/RoBnzgkw54LSuWnII6MU1q8 Wj24mvt19h43t0KUtbEN3sh4pgeztgZz0GlkCCKSyLo6yz09xZncCB5QJTi7tgAo UbaK6hgZG2hLzjvzY/9gxvkUR5MRcy0QUFXZ2TKsfsWAW9eN73BhxaqQzYEX63n3 5Rf2s9Bk8dNOlX0k+htBNnUfTa3ft63MO1WNjS/ggmbU1G7fS/RDHV5VJnq3KUuI eS1TcuuZ9r4eJSG8M6oSobiuMU1m5REYy0k9PmHSsA49EosTgZsUu/oV5+/BXMAA gWlkHx0ulw9MWojVH6L3O1H1unO26KdEIBSwGklu5Pl6FKkjWUw= =Ae9L -----END PGP SIGNATURE----- --GCUetWhX8ucNpWXz6cf5fscUr5kb2Pwfz-- From owner-zfs-devel@freebsd.org Mon Mar 30 16:10:17 2020 Return-Path: Delivered-To: zfs-devel@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id CBE862A70EF for ; Mon, 30 Mar 2020 16:10:17 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: from mail-vs1-xe33.google.com (mail-vs1-xe33.google.com [IPv6:2607:f8b0:4864:20::e33]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 48rcqd49CWz4Bc1 for ; Mon, 30 Mar 2020 16:10:05 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: by mail-vs1-xe33.google.com with SMTP id 184so10616842vsu.3 for ; Mon, 30 Mar 2020 09:10:05 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=J0CT2ZxD0VdNs7kl2V57sMtKDjtyx4L3oWm7gJW7KFY=; b=tT80vTruEohHxv4nOHCtHYrmCrODvlCWxaQ591m5Qk/gGofF2xqiZnifcBGhrbVpEW cWwMlOybj45TpKG8YNitJzyKnR1IRSnnhW1oiGOkmLMeZx734Gv+wnW5NutGZmt3RIXY yMvEVsHJ0jEoOiUL0Bu5FkEII1MJgk0vHbqnLPnK7YiIcx0buXzRkuhsxI2oI79d62X0 uAs4inW6rlvVhlQ3ICFBb9GGkD7KxPLaoFacUAW15IgS7YCDQxhBJN2pEsXbpZeaYlbt TcCXr9iakGAF2uVlO3Kw2VCRRRyro3W5awMTtloERfgJn7QHbZ2vzqggis3PESRP9N2K KIsA== X-Gm-Message-State: AGi0PuaceoE0srt/xhztWptwGHd7mppP49LmgsStGXGX1eAt1awv3ugb Jiq65UNF273kc8Kh1Wf2o0FoFdaJth9DuYbB/vpJAxvk9m0= X-Google-Smtp-Source: APiQypKjqOMJ/q08G496KRz7HurXPC1RVp223gr03kee8cO+Be3Sv8uDQTFVrCZ1ACwMnt+sHuMbyq6Yb3nKJv/b3WQ= X-Received: by 2002:a67:5ec6:: with SMTP id s189mr9039956vsb.56.1585584594704; Mon, 30 Mar 2020 09:09:54 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Matthew Ahrens Date: Mon, 30 Mar 2020 09:09:43 -0700 Message-ID: Subject: Second March OpenZFS Leadership Meeting To: developer , illumos-zfs , zfs-devel@freebsd.org, freebsd-fs X-Rspamd-Queue-Id: 48rcqd49CWz4Bc1 X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.76 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[delphix.com:s=google]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[zfs-devel@freebsd.org]; TO_DN_SOME(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[delphix.com:+]; DMARC_POLICY_ALLOW(-0.50)[delphix.com,quarantine]; RCVD_IN_DNSWL_NONE(0.00)[3.3.e.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; FORGED_SENDER(0.30)[mahrens@delphix.com,matthew.ahrens@delphix.com]; MIME_TRACE(0.00)[0:+,1:+,2:~]; IP_SCORE(-2.06)[ip: (-9.45), ipnet: 2607:f8b0::/32(-0.35), asn: 15169(-0.46), country: US(-0.05)]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[mahrens@delphix.com,matthew.ahrens@delphix.com]; RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2] Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Mar 2020 16:10:17 -0000 The next OpenZFS Leadership meeting will be held tomorrow, March 31, 9am-10am Pacific time. We have several interesting topics on the agenda for tomorrow's meeting: - Add =E2=80=9Czstream redup = =E2=80=9D utility; remove =E2=80=9Czfs send --dedup =E2=80=9D (Matt) - Add O_DIRECT support: design update (Matt) - New API, higher-level than libzfs - Changes that need reviewers: - Persistent L2ARC - https://github.com/openzfs/zfs/pull/9582 - Introduction of ZSTD compr. to ZFS - https://github.com/openzfs/zfs/pull/9735 - Dedup DDT load https://github.com/zfsonlinux/zfs/pull/9464/ Everyone is welcome to attend and participate, and we will try to keep the meeting on agenda and on time. The meetings will be held online via Zoom, and recorded and posted to the website and YouTube after the meeting. The agenda for the meeting will be a discussion of the projects listed in the agenda doc. For more information and details on how to attend, as well as notes and video from the previous meeting, please see the agenda document: https://docs.google.com/document/d/1w2jv2XVYFmBVvG1EGf-9A5HBVsjAYoLIFZAnWHh= V-BM/edit --matt From owner-zfs-devel@freebsd.org Mon Apr 27 17:29:30 2020 Return-Path: Delivered-To: zfs-devel@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 4E42D2C0A4C for ; Mon, 27 Apr 2020 17:29:30 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: from mail-ed1-x532.google.com (mail-ed1-x532.google.com [IPv6:2a00:1450:4864:20::532]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 499sGK1PgGz4N9W for ; Mon, 27 Apr 2020 17:29:29 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: by mail-ed1-x532.google.com with SMTP id w2so14092407edx.4 for ; Mon, 27 Apr 2020 10:29:28 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=B54AYtfBfD5WuviIc9xZHeMtHaKJL1Yv4Xw10I9zZ2o=; b=pYYetcS+ZwyNTz1F6CHquqbqBwHwmKUFAQHJpnTXvdqv40FK/Wk9xEG4FRRkirWBbi 1HRi4q+NoBCjf67yfQvaDecIm8VdsCMT4SU9OM3LIFay1d6fNZUXxAZyl18isz4PfEuy epQLtCfeuFMJTtuLtNgoRa671pr1M3Myyxht5QB0GebbgMMUuMsIanNGyspcbE67xlpQ LUp9NuyK/LjhY20rFWdI02AZKkpZtzwWXqyScmDdSoouc9+VgZ3wrCbiur4PUj/GrVE7 dIOvWBRr+b/8N5tpvEO4XlPe+54daH34UX/6ajTVrd0eDyPGevhQK8dEZWtEMsIUr8Rf 5GWA== X-Gm-Message-State: AGi0PuaUOmWP2CrMyRW4L2TRkkMfT2zA6MsA/NQEkHHzweefoJyF8GQC z9O5mgy29Chr1XMWLL/yW8RONl4AknxIaX963WbuopviGc0= X-Google-Smtp-Source: APiQypI7JC7/7o5KOmNaTGyQYB8Hzn3fJycjk/pN0ghjUMY8KfKUFp8XP0sS5q47kBrb8YkVGs2K0SNCc4H9hIPYiFg= X-Received: by 2002:a50:8d4a:: with SMTP id t10mr20036769edt.63.1588008566869; Mon, 27 Apr 2020 10:29:26 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Matthew Ahrens Date: Mon, 27 Apr 2020 10:29:15 -0700 Message-ID: Subject: April OpenZFS Leadership Meeting To: developer , illumos-zfs , zfs-devel@freebsd.org, freebsd-fs X-Rspamd-Queue-Id: 499sGK1PgGz4N9W X-Spamd-Bar: ----- X-Spamd-Result: default: False [-5.16 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[delphix.com:s=google]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[zfs-devel@freebsd.org]; TO_DN_SOME(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[delphix.com:+]; DMARC_POLICY_ALLOW(-0.50)[delphix.com,quarantine]; RCVD_IN_DNSWL_NONE(0.00)[2.3.5.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.5.4.1.0.0.a.2.list.dnswl.org : 127.0.5.0]; FORGED_SENDER(0.30)[mahrens@delphix.com,matthew.ahrens@delphix.com]; MIME_TRACE(0.00)[0:+,1:+,2:~]; IP_SCORE(-2.46)[ip: (-9.51), ipnet: 2a00:1450::/32(-2.33), asn: 15169(-0.43), country: US(-0.05)]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[mahrens@delphix.com,matthew.ahrens@delphix.com]; RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 27 Apr 2020 17:29:30 -0000 The next OpenZFS Leadership meeting will be held tomorrow, April 28, 1pm-2pm Pacific time. We have several topics on the agenda for tomorrow's meeting, let me know if you'd like to add anything: > - > > Libshare changes (George) > - > > Status on Panzura dedup (Josh P) > - > > Status on dedup-log & DDT limit (Allan) > - > > Persistent L2ARC merged > - > > Changes that need reviewers: > - > > Introduction of ZSTD compr. to ZFS - > https://github.com/openzfs/zfs/pull/9735 > - > > Dedup DDT load https://github.com/zfsonlinux/zfs/pull/9464/ > - > > Dedup Ceiling https://github.com/openzfs/zfs/pull/10169 > > Everyone is welcome to attend and participate, and we will try to keep the meeting on agenda and on time. The meetings will be held online via Zoom, and recorded and posted to the website and YouTube after the meeting. The agenda for the meeting will be a discussion of the projects listed in the agenda doc. For more information and details on how to attend, as well as notes and video from the previous meeting, please see the agenda document: https://docs.google.com/document/d/1w2jv2XVYFmBVvG1EGf-9A5HBVsjAYoLIFZAnWHhV-BM/edit --matt From owner-zfs-devel@freebsd.org Thu Apr 30 18:19:17 2020 Return-Path: Delivered-To: zfs-devel@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 66C182C4CF8 for ; Thu, 30 Apr 2020 18:19:17 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: from mail-ej1-x633.google.com (mail-ej1-x633.google.com [IPv6:2a00:1450:4864:20::633]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 49CkDM6Nj5z3Nq9 for ; Thu, 30 Apr 2020 18:19:15 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: by mail-ej1-x633.google.com with SMTP id s3so5478406eji.6 for ; Thu, 30 Apr 2020 11:19:15 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=sRR5KnvGZ6OLpYRZABgXQtMp+MJI5dNL6pYel2fJNP4=; b=hpLvEK/8aKNcPrToNAKcCyyfwkEx5oaaQkYLx39a++24Y+643HLUiQtuKn+8Es+/JN 6jsWyXwe30p4w3sV2NByTR2dW3u88TL6oINf3D0tV8OyTajN1UbvFMoAE2N0/qbCnnvp sbTGmxdfFT9/EoDxDWTFFHEAceGGgxPt0fDmhP3PtBXB904CQJmiIc8h/7cKspqvsOb3 lZpzgaW12lzOH0TQJQWOh1N3Zrvs84INT9DVkQeAuKAFHNWJD65GmAmM9T4pS2tHJ/0/ 6fmMu663wZawajt/CxwbBACMJvlyi6sBuoAo6PPyhDixsC4SzYlpRNJkuz3HMCb3hvPv 52Yw== X-Gm-Message-State: AGi0PubFc21I4fNNGltT8Tf1T2w3d4EjdZ6Nrv/PmClPXKKHPc4a3EKQ Qzk/jJ0gw7pa5UGyt0a+Lmx6uCnYFYMoV+coQeOe9A== X-Google-Smtp-Source: APiQypL737bvHDHZj6EzW+Kneqj/tFRn8+I03esflDGwD+D0aaMSWDSL25sLvPro9lTmFhACET3dgUpvLsOgbVtsGSY= X-Received: by 2002:a17:906:31d7:: with SMTP id f23mr4124696ejf.59.1588270753726; Thu, 30 Apr 2020 11:19:13 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Matthew Ahrens Date: Thu, 30 Apr 2020 11:19:02 -0700 Message-ID: Subject: Re: April OpenZFS Leadership Meeting To: developer , illumos-zfs , zfs-devel@freebsd.org, freebsd-fs X-Rspamd-Queue-Id: 49CkDM6Nj5z3Nq9 X-Spamd-Bar: --- X-Spamd-Result: default: False [-3.26 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[delphix.com:s=google]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[zfs-devel@freebsd.org]; TO_DN_SOME(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[delphix.com:+]; DMARC_POLICY_ALLOW(-0.50)[delphix.com,quarantine]; RCVD_IN_DNSWL_NONE(0.00)[3.3.6.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.5.4.1.0.0.a.2.list.dnswl.org : 127.0.5.0]; FORGED_SENDER(0.30)[mahrens@delphix.com,matthew.ahrens@delphix.com]; MIME_TRACE(0.00)[0:+,1:+,2:~]; IP_SCORE(-0.56)[ipnet: 2a00:1450::/32(-2.32), asn: 15169(-0.43), country: US(-0.05)]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[mahrens@delphix.com,matthew.ahrens@delphix.com]; RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2] Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 30 Apr 2020 18:19:17 -0000 Thanks to everyone who participated at this month's meeting. The video recording is now posted: https://www.youtube.com/watch?v=3DAfvBY8du8OE Thanks to Serapheim for taking notes: - Libshare changes (George Wilson) - George has been doing work revamping the sharenfs property codepaths. The reason is performance and consistency between multiple consumers = that need to look at the state of the system. - Work mostly combines what exists in FreeBSD while at the same time removing a lot of the logic around sharetab. The changes affect the FreeBSD code too where libzfsfsshare is renamed/moved to libshare. - Preliminary Performance Testing (having ~1000 filesystems and multiple threads that share/unshare them) shows up to 99% performance improvements, on the scale of going from minutes to seconds. - During questions: - (Michael Dexter) Is iSCSI and/or FiberChannel on the radar? - There is some existing code to handle these subsystems but it hasn=E2=80=99t been touched(used?) in a long time. It would be = an interesting project if someone is using them and wants to pick up the work. - Status on Panzura dedup (Josh P) - Initially thought to be self-contained but was discovered that this is not the case. - Internally within Panzura the effort has been temporarily deprioritized. - Status on dedup-log & DDT limit (Allan) - Both coming along nicely. DDT is on the last round of feedback. - REMINDER: For the DDT limit the semantics have changed - it is not an explicit memory limit but more of an on-disk one. The memory-limit wa= s causing usability problems and its behavior was hard to model. - Persistent L2ARC merged - Feature doesn=E2=80=99t require a feature flag. - There are still a couple of follow-up patches for that change that could use reviewers. - NOTE: The version merged is a slightly modified version of what initially had started from Nexenta. - Dedup=E2=80=99d send/recv removed - NOTE: This feature has actually nothing to do with Dedup and is a specific feature of Send/Recv (zfs send --dedup/-D) - The feature has been removed from master. From 0.8.4 users should see a deprecation notice if they try to use it. - OpenZFS for OSX common repo - Since FreeBSD was merged Jorgen is looking into a fresh new port for OSX to be merged into the OpenZFS repo. The new port gives us a chanc= e to redo somethings that were not optimal in terms of implementation. The ability to migrate from old OSX pools to use the new implementation is not a requirement but it would be good to have. - Once everything is green with existing tests Jorgen will start pinging people for feedback, the potential of adding OSX tests in the= CI pipeline, and to discuss issues found in the common code used by all = the operating systems. - Zpool user properties - Allan hopes to open a PR to upstream this soon. - Still looking to hear from people who already have use cases for this and overall any feedback. - Changes that need reviewers: - Introduction of ZSTD compr. to ZFS - https://github.com/openzfs/zfs/pull/9735 - Needs to be updated now that persistent L2ARC has been merged. There is also some leftover feedback to incorporate. - Allan is looking for feedback on how to break this change down to logical parts so it becomes easier to review, and potentially make= it smaller. - There are also a few old ZSTD PRs laying around in the PR list of OpenZFS on Github, we should take one last look before closing the= m to ensure we haven=E2=80=99t missed anything. - Dedup DDT load https://github.com/zfsonlinux/zfs/pull/9464/ - Has been out for a while - we need to solicit the mailing list so we: - [1] Give a heads up that this is coming - [2] Get any high or low-level feedback - [3] Ensure that conflicts with other dedup patches that are still downstream are minimized. - Dedup Ceiling https://github.com/openzfs/zfs/pull/10169 - Feedback from Matt is almost done and change should be ready to go soon. On Mon, Apr 27, 2020 at 10:29 AM Matthew Ahrens wrote= : > The next OpenZFS Leadership meeting will be held tomorrow, April 28, > 1pm-2pm Pacific time. We have several topics on the agenda for tomorrow'= s > meeting, let me know if you'd like to add anything: > > >> - >> >> Libshare changes (George) >> - >> >> Status on Panzura dedup (Josh P) >> - >> >> Status on dedup-log & DDT limit (Allan) >> - >> >> Persistent L2ARC merged >> - >> >> Changes that need reviewers: >> - >> >> Introduction of ZSTD compr. to ZFS - >> https://github.com/openzfs/zfs/pull/9735 >> - >> >> Dedup DDT load https://github.com/zfsonlinux/zfs/pull/9464/ >> - >> >> Dedup Ceiling https://github.com/openzfs/zfs/pull/10169 >> >> Everyone is welcome to attend and participate, and we will try to keep > the meeting on agenda and on time. The meetings will be held online via > Zoom, and recorded and posted to the website and YouTube after the meetin= g. > > The agenda for the meeting will be a discussion of the projects listed in > the agenda doc. > > For more information and details on how to attend, as well as notes and > video from the previous meeting, please see the agenda document: > > > https://docs.google.com/document/d/1w2jv2XVYFmBVvG1EGf-9A5HBVsjAYoLIFZAnW= HhV-BM/edit > > --matt > From owner-zfs-devel@freebsd.org Mon May 25 22:45:08 2020 Return-Path: Delivered-To: zfs-devel@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 281EC2CDB92 for ; Mon, 25 May 2020 22:45:08 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: from mail-ej1-x644.google.com (mail-ej1-x644.google.com [IPv6:2a00:1450:4864:20::644]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 49WBxb3J6pz3Xq5 for ; Mon, 25 May 2020 22:45:07 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: by mail-ej1-x644.google.com with SMTP id se13so21803091ejb.9 for ; Mon, 25 May 2020 15:45:07 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=un31iLMYR/G0E0p/C5sgeiHo+kUInMwVbV49uD9rGus=; b=SPN+elAHhMp9wW0ldFnj+LoB+mdh2vhhJvfmuu6Ay1iq1RmvQJWVHWPLOZ3PV0TMab Kf2kPSChjRNtOpjxNCVw/vrmqfk2fzGmSickzSWk8bSWdKCOf4K64BhnKEkwnhey6JoG nfbDcD/Ti7KBj5C7kHRYwUPm4Ksy7/5IsVR1hR0mKLchVSeeCtQHSla1H2XG9pCalQhH 6eWuTnkW+GnmqRnYHyNtegl7G2byEiRE8ZI9cP94e+YcnuJCfwd4NUfpIX6R0kxa+rhh xsAsVN5GtwjD39Z5ioBZmGi6v+tWgb8GpLVogdNBoDLTPBay9Ug9qanNPIGQwH9Y2jJm ZF2g== X-Gm-Message-State: AOAM533aAqkHpij3QEXKCtYg2IIRdl8ouui3owSrqj3NFdCxt0UcHTfu iZ6toTeLCOQDcsfhh0asJtU0kJOo3uy8Y/o3sMNOQntMpnM= X-Google-Smtp-Source: ABdhPJzxdkGsX2bZOi1vxouxRcJIQClsgstRKujussE6k2h1wBWGGXk8kbtggRVnvVVT7kCecegeBpHRJxD5lZ5kZW4= X-Received: by 2002:a17:906:a0c2:: with SMTP id bh2mr21398205ejb.458.1590446705596; Mon, 25 May 2020 15:45:05 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Matthew Ahrens Date: Mon, 25 May 2020 15:44:54 -0700 Message-ID: Subject: May OpenZFS Leadership Meeting To: developer , illumos-zfs , zfs-devel@freebsd.org, freebsd-fs X-Rspamd-Queue-Id: 49WBxb3J6pz3Xq5 X-Spamd-Bar: --- X-Spamd-Result: default: False [-3.81 / 15.00]; RCVD_TLS_ALL(0.00)[]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[delphix.com:s=google]; NEURAL_HAM_MEDIUM(-0.99)[-0.986]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[zfs-devel@freebsd.org]; NEURAL_HAM_LONG(-1.02)[-1.025]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[delphix.com:+]; DMARC_POLICY_ALLOW(-0.50)[delphix.com,quarantine]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::644:from]; NEURAL_HAM_SHORT(-1.10)[-1.104]; FORGED_SENDER(0.30)[mahrens@delphix.com,matthew.ahrens@delphix.com]; MIME_TRACE(0.00)[0:+,1:+,2:~]; RCVD_COUNT_TWO(0.00)[2]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[mahrens@delphix.com,matthew.ahrens@delphix.com] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.33 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 25 May 2020 22:45:08 -0000 The next OpenZFS Leadership meeting will be held tomorrow, May 26, 1pm-2pm Pacific time. We have several topics on the agenda for tomorrow's meeting, let me know if you'd like to add anything: - Documentation system - POSIX AIO for async dmu? - Formalizing pad2, bootonce/nextboot/rescue counter (Allan) - Remove uncompressed ARC? (Matt) - See previous discussion - Rebuild PR (Brian) - Send toggle -L fix (Matt) Everyone is welcome to attend and participate, and we will try to keep the meeting on agenda and on time. The meetings will be held online via Zoom, and recorded and posted to the website and YouTube after the meeting. The agenda for the meeting will be a discussion of the projects listed in the agenda doc. For more information and details on how to attend, as well as notes and video from the previous meeting, please see the agenda document: https://docs.google.com/document/d/1w2jv2XVYFmBVvG1EGf-9A5HBVsjAYoLIFZAnWHhV-BM/edit --matt From owner-zfs-devel@freebsd.org Wed May 27 21:10:03 2020 Return-Path: Delivered-To: zfs-devel@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id DACDE2CF569 for ; Wed, 27 May 2020 21:10:03 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: from mail-ed1-x544.google.com (mail-ed1-x544.google.com [IPv6:2a00:1450:4864:20::544]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 49XNky6hYLz48gP for ; Wed, 27 May 2020 21:10:02 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: by mail-ed1-x544.google.com with SMTP id bs4so21488931edb.6 for ; Wed, 27 May 2020 14:10:02 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=8kypp3zbo3sh30B5fLxuvOmPZmvFnb4s6ee2hDDWzlk=; b=S8qIpKmKjFP0MaRQu+vgI4INdVYwHAXmE8ZFwPieeR0L2Bjfbgm1jOUPr4JpmDYc/d ucSMuR7+aGi52Y6LXoWqJjkJrMMBkr5Nke2LLR/bspjaz2r0VjdSm2HkSD+nK+FjWvy3 9lPj+XlDdvtfFVYWS1/9mbBpwbsMKQNKX1phdBxPfAutllsJqYESF3eRAK+6LWWP8xSy 5dYs1hvZZf8touREav9gGXzeUg3T0iasaULkGMqC/YuhkBNaXw8icf/880xdVf0eGj+N kIO7vykttTYOQJQttfAPOL/X7KkLm67kr2sFJeoStn15zioGk3oo151ZBwIlBDj0iLkf yiVg== X-Gm-Message-State: AOAM5314c8XGEwGa9G7i8TJjAzOLVlCj+HnzaFbgDqN2VT6DEa7+OGoK lxok0Nw3fO8170lzla9VY8FCFzU4NDAEXAt6AFa+GFlziGc= X-Google-Smtp-Source: ABdhPJxi5FxgWxYWmiA7D5tCdtT+tfvewC1FB0RLnCe5w39BE6DEwQqWdyiTPct4ATxtf3JHeuNJAc9kPRdAUNMDAmU= X-Received: by 2002:a50:f983:: with SMTP id q3mr37022edn.259.1590613800259; Wed, 27 May 2020 14:10:00 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Matthew Ahrens Date: Wed, 27 May 2020 14:09:48 -0700 Message-ID: Subject: Re: May OpenZFS Leadership Meeting To: developer , illumos-zfs , zfs-devel@freebsd.org, freebsd-fs X-Rspamd-Queue-Id: 49XNky6hYLz48gP X-Spamd-Bar: -- X-Spamd-Result: default: False [-2.37 / 15.00]; RCVD_TLS_ALL(0.00)[]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[delphix.com:s=google]; NEURAL_HAM_MEDIUM(-0.98)[-0.982]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[zfs-devel@freebsd.org]; NEURAL_HAM_LONG(-0.99)[-0.985]; URI_COUNT_ODD(1.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[delphix.com:+]; DMARC_POLICY_ALLOW(-0.50)[delphix.com,quarantine]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::544:from]; NEURAL_HAM_SHORT(-0.71)[-0.705]; FORGED_SENDER(0.30)[mahrens@delphix.com,matthew.ahrens@delphix.com]; MIME_TRACE(0.00)[0:+,1:+,2:~]; RCVD_COUNT_TWO(0.00)[2]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[mahrens@delphix.com,matthew.ahrens@delphix.com] Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.33 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 27 May 2020 21:10:03 -0000 Thanks to everyone who participated at this month's meeting. The video recording is now available: https://www.youtube.com/watch?v=3DSyNn26YbRZY Meeting notes (thanks Serapheim): - OpenZFS conference - Will be held virtually/online this year - Currently planned at the end of September - there is flexibility as people won=E2=80=99t be travelling - higher attendance is expected for the same reason - Talk Submissions is opening next week - Format will be roughly the same as other years - We still haven=E2=80=99t decided on the platform to host it - We want to hear from others: what conference tools and formats worked on other conferences (or didn=E2=80=99t) - contact Karyn or= Matt - Rebuild PR (Brian) - Work came out as preparatory work for the DRAID projects but for mirrors - The PR as it stands introduces a new option (-R) for the zpool replace and zpool attach commands that performs the rebuild and provi= des redundancy - The caveat is that no checksums are verified but for that same reason it can be a lot faster (especially for fragmented pools and/or small block sizes) - Each vdev can be rebuilt independently, so you can have multiple operations going on, unlike resilver which is pool-wide. - The code used to be part of resilver but then it was reworked to have its own standalone infrastructure. - Looking for feedback: https://github.com/openzfs/zfs/pull/10349 - Documentation system (George Melikov) - https://openzfs.github.io/openzfs-docs/ - Short-term goal: Move Linux Wiki to the above repo and add automation - Long-term goal: Have all documentation at one place (e.g. man pages, content from the media wiki, etc..) - For folks interested in migrating the media wiki content feel free to ping Karyn, Michael Dexter, or Matt to see what content is still rele= vant and what is stale. - Open to feedback on what format and tools to use or other ideas - Several concerns with using RST as the source for manpages. - POSIX AIO for async dmu? (Matt Macy) - =E2=80=9Cwondering if it's worth integrating async dmu with [POSIX AI= O] as opposed to just using [async dmu] for zvol=E2=80=9D - Some work was started by SpectraLogic years ago - The goal is to expose the asynchronous ZIO interfaces at the dmu level and integrate it to ZVOLs or the POSIX AIO interface (standard syscalls like POSIX reads cannot take advantage of this). - Regarding POSIX AIO the questions are: - Is anyone already using these interfaces? If so, what=E2=80=99s th= eir experience? - Would people be interested in using it if it existed? If so, for what use-case? - Upstreaming =E2=80=9Cforced export=E2=80=9D feature (Allan) - Work for this project will be starting soon - Use-Case: Problems in connectivity with remote vdevs would suspend the pool (and potentially block other pools when higher-level locks a= re grabbed) - Proposal: zpool export -F - force the export of that pool so everything goes back to normal. - Formalizing pad2, bootonce/nextboot/rescue counter (Allan) - An interface introduced recently regarding this functionality only accepts a character array as an argument (in libZFS_Core). The propos= al here is to either change this existing call or add a new one that acc= epts an nvlist instead to provide more flexibility for new features. - OS-specific properties (Sean) - PR is out: https://github.com/openzfs/zfs/pull/10111 - Deals with the issue of os-specific properties varying wildly in their behavior between platforms while there is room for normalizatio= n. - Needs Reviewers/Feedback On Mon, May 25, 2020 at 3:44 PM Matthew Ahrens wrote: > The next OpenZFS Leadership meeting will be held tomorrow, May 26, 1pm-2p= m > Pacific time. We have several topics on the agenda for tomorrow's meetin= g, > let me know if you'd like to add anything: > > > - > > Documentation system > - > > POSIX AIO for async dmu? > - > > Formalizing pad2, bootonce/nextboot/rescue counter (Allan) > - > > Remove uncompressed ARC? (Matt) > - > > See previous discussion > > - > > Rebuild PR (Brian) > - > > Send toggle -L fix (Matt) > > > Everyone is welcome to attend and participate, and we will try to keep th= e > meeting on agenda and on time. The meetings will be held online via Zoom= , > and recorded and posted to the website and YouTube after the meeting. > > The agenda for the meeting will be a discussion of the projects listed in > the agenda doc. > > For more information and details on how to attend, as well as notes and > video from the previous meeting, please see the agenda document: > > > https://docs.google.com/document/d/1w2jv2XVYFmBVvG1EGf-9A5HBVsjAYoLIFZAnW= HhV-BM/edit > > --matt > From owner-zfs-devel@freebsd.org Sat May 30 07:10:27 2020 Return-Path: Delivered-To: zfs-devel@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id B60F332863E for ; Sat, 30 May 2020 07:10:27 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 49Ysyq4W2Yz4LD0 for ; Sat, 30 May 2020 07:10:27 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 9623D250F1 for ; Sat, 30 May 2020 07:10:27 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 04U7AR1p036084 for ; Sat, 30 May 2020 07:10:27 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 04U7AR5a036083 for zfs-devel@FreeBSD.org; Sat, 30 May 2020 07:10:27 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: zfs-devel@FreeBSD.org Subject: [Bug 222007] Deadlock at shutdown time between zfs and usb when using usb stick as cache device Date: Sat, 30 May 2020 07:10:27 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 10.3-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: bsdpr@phoe.frmug.org X-Bugzilla-Status: Closed X-Bugzilla-Resolution: Overcome By Events X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: bugs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: bug_status resolution Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 30 May 2020 07:10:27 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D222007 Bertrand Petit changed: What |Removed |Added ---------------------------------------------------------------------------- Status|New |Closed Resolution|--- |Overcome By Events --- Comment #4 from Bertrand Petit --- I'm closing this old bug report which is now useless, the reported behavior= is no longer exhibited. --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-zfs-devel@freebsd.org Sat May 30 07:28:44 2020 Return-Path: Delivered-To: zfs-devel@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 32E4A328E85 for ; Sat, 30 May 2020 07:28:44 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 49YtMw08VWz4Mwd for ; Sat, 30 May 2020 07:28:44 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id F3DDC25340 for ; Sat, 30 May 2020 07:28:43 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 04U7ShhH050619 for ; Sat, 30 May 2020 07:28:43 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 04U7Sh2q050599 for zfs-devel@FreeBSD.org; Sat, 30 May 2020 07:28:43 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: zfs-devel@FreeBSD.org Subject: [Bug 246856] [zfs] ZCP feature request: permit the creation and alteration of dataset properties Date: Sat, 30 May 2020 07:28:44 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 12.1-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: bsdpr@phoe.frmug.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: bugs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 30 May 2020 07:28:44 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D246856 Bertrand Petit changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |bsdpr@phoe.frmug.org, | |zfs-devel@FreeBSD.org --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-zfs-devel@freebsd.org Sat May 30 08:58:19 2020 Return-Path: Delivered-To: zfs-devel@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 5C6D032B230 for ; Sat, 30 May 2020 08:58:19 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 49YwMH1rkHz4VkL for ; Sat, 30 May 2020 08:58:19 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 3AE8825E77 for ; Sat, 30 May 2020 08:58:19 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 04U8wJjE084934 for ; Sat, 30 May 2020 08:58:19 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 04U8wJc3084933 for zfs-devel@FreeBSD.org; Sat, 30 May 2020 08:58:19 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: zfs-devel@FreeBSD.org Subject: [Bug 246856] [zfs] ZCP feature request: permit the creation and alteration of dataset properties Date: Sat, 30 May 2020 08:58:18 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 12.1-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: avg@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: bugs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 30 May 2020 08:58:19 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D246856 Andriy Gapon changed: What |Removed |Added ---------------------------------------------------------------------------- CC|zfs-devel@FreeBSD.org | --- Comment #1 from Andriy Gapon --- zfs-devel@ is not actively used anymore. fs@ is the list. --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-zfs-devel@freebsd.org Sat May 30 08:59:41 2020 Return-Path: Delivered-To: zfs-devel@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 1BC2932B258 for ; Sat, 30 May 2020 08:59:41 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 49YwNr70t9z4W6D for ; Sat, 30 May 2020 08:59:40 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id EBC83260DE for ; Sat, 30 May 2020 08:59:40 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 04U8xePP086486 for ; Sat, 30 May 2020 08:59:40 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 04U8xe8J086485 for zfs-devel@FreeBSD.org; Sat, 30 May 2020 08:59:40 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: zfs-devel@FreeBSD.org Subject: [Bug 222007] Deadlock at shutdown time between zfs and usb when using usb stick as cache device Date: Sat, 30 May 2020 08:59:41 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 10.3-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: avg@FreeBSD.org X-Bugzilla-Status: Closed X-Bugzilla-Resolution: Overcome By Events X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: bugs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 30 May 2020 08:59:41 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D222007 Andriy Gapon changed: What |Removed |Added ---------------------------------------------------------------------------- CC|zfs-devel@FreeBSD.org | --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-zfs-devel@freebsd.org Mon Jun 8 20:42:37 2020 Return-Path: Delivered-To: zfs-devel@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id BD89333AEBA for ; Mon, 8 Jun 2020 20:42:37 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 49glYn4hPTz4Vmc for ; Mon, 8 Jun 2020 20:42:37 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 9C831A98B for ; Mon, 8 Jun 2020 20:42:37 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 058Kgb4j082771 for ; Mon, 8 Jun 2020 20:42:37 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 058KgbBL082770 for zfs-devel@FreeBSD.org; Mon, 8 Jun 2020 20:42:37 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: zfs-devel@FreeBSD.org Subject: [Bug 180001] [zfs] [panic] Solaris(panic): zfs: allocating allocated segment Date: Mon, 08 Jun 2020 20:42:35 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: Unspecified X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: madpilot@FreeBSD.org X-Bugzilla-Status: In Progress X-Bugzilla-Resolution: X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: zfs-devel@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Jun 2020 20:42:37 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D180001 Guido Falsi changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |madpilot@FreeBSD.org --- Comment #5 from Guido Falsi --- Posting a followup since I'm experiencing a very similar error. Machine is a laptop with only one disk. FreeBSD 13-head, r361728 The machine is configures with geli and zfs boot, as created by the FreeBSD installer. At some point the machine has had a panic+reboot while shutting down [1] an= d on reboot, after inputting the geli password, the machine panicked. With a deb= ug kernel I discovered the reason: panic: Solaris(panic): zfs: allocating allocated segment(offset=3D433818959872 size=3D4096) of (offset=3D433818021200 size= =3D32768) The panic appears to happen in range_tree_add_impl, called from space_map_load_callback. After this I was able to make the machine boot by setting vfs.zfs.recover= =3D1 in loader.conf. At boot it now gives these error messages: Solaris: WARNING: zfs: allocating allocated segment(offset=3D433818959872 size=3D4096) of (offset=3D433818931200 size=3D32768) Solaris: WARNING: zfs: allocating allocated segment(offset=3D433815740416 size=3D4096) of (offset=3D433815736320 size=3D8192) Once booted the machine works fine, for the little use I've put it up after this event. I also tried to run some more diagnostic zdb gave me this: # zdb -c zroot Traversing all blocks to verify metadata checksums and verify nothing leaked ... loading concrete vdev 0, metaslab 101 of 113 ...Assertion failed: rs->rs_st= art <=3D start (0x65019be000 <=3D 0x65019bd000), file /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/range_tree.c, line = 410. Abort (core dumped) Anny other diagnostic I could try to run? [1] this is actually speculation since it happened when I commanded a shut= down from another PC in another room, and found the machine at thew geli password prompt instead of turned off. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-zfs-devel@freebsd.org Sun Jun 14 08:39:11 2020 Return-Path: Delivered-To: zfs-devel@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 22305333B5C for ; Sun, 14 Jun 2020 08:39:11 +0000 (UTC) (envelope-from info@hwinfotech.net) Received: from mata.hwinfotech.net (mata.hwinfotech.net [51.79.44.1]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 49l7DG1kkqz47KS for ; Sun, 14 Jun 2020 08:39:10 +0000 (UTC) (envelope-from info@hwinfotech.net) DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; s=dkim; d=hwinfotech.net; h=Message-ID:Date:Subject:From:Reply-To:To:MIME-Version:Content-Type:List-Unsubscribe:List-Id; i=info@hwinfotech.net; bh=B+AkF8n5CXfqwvIpWevysNVNIHM=; b=qUhh4ZpukzYWhfmYUoHMW744eSzpDSMroIdAaPJ0vPR/YjoZlZXiPlvqD8bG40Us2EBi8VWSd5uq PuY/skTskninfyVF7V14dNNnjFpWPodKWv+nRlfc8X6CSOhAu8lMDuhZNonxb+Qwtbh7Rc+pC4nh pb7NDMMRKtjBrm9v/+I= DomainKey-Signature: a=rsa-sha1; c=nofws; q=dns; s=dkim; d=hwinfotech.net; b=ZrssGJUMUkZ5JllXSEaFl9vuogKY53UEr19WqMA7yXAtHJoO/AkF9mbOnlVVN950DThkEUYnE525 kvaKgL3IhXPu1MfiIEUQX2DX38bQboygwh/jLM6o9um+oSLwtFHPSkoJn1YnEeE/nCJTe83ynAgK k0LLknmiWSIkoSbdExc=; Message-ID: <30bfe16df0c836cc00b129ccd859ab95@hwinfotech.net> Date: Sun, 14 Jun 2020 08:39:08 +0000 Subject: Website and Mobile App Development From: HW Infotech Reply-To: HW Infotech To: "zfs-devel@freebsd.org" MIME-Version: 1.0 X-Sender: info@hwinfotech.net X-Report-Abuse: Please report abuse for this campaign here: http://app.hwinfotech.net/index.php/campaigns/ok072mq8cc88f/report-abuse/dd296hm0jzd67/wd423m5jsx768 X-Receiver: zfs-devel@freebsd.org X-Fxyn-Tracking-Did: 0 X-Fxyn-Subscriber-Uid: wd423m5jsx768 X-Fxyn-Mailer: SwiftMailer - 5.4.x X-Fxyn-EBS: http://app.hwinfotech.net/index.php/lists/block-address X-Fxyn-Delivery-Sid: 3 X-Fxyn-Customer-Uid: pp359780cbfd3 X-Fxyn-Customer-Gid: 0 X-Fxyn-Campaign-Uid: ok072mq8cc88f Precedence: bulk Feedback-ID: ok072mq8cc88f:wd423m5jsx768:dd296hm0jzd67:pp359780cbfd3 X-Rspamd-Queue-Id: 49l7DG1kkqz47KS X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=hwinfotech.net header.s=dkim header.b=qUhh4Zpu; dmarc=pass (policy=none) header.from=hwinfotech.net; spf=pass (mx1.freebsd.org: domain of info@hwinfotech.net designates 51.79.44.1 as permitted sender) smtp.mailfrom=info@hwinfotech.net X-Spamd-Result: default: False [-4.22 / 15.00]; ARC_NA(0.00)[]; HAS_REPLYTO(0.00)[info@hwinfotech.net]; R_DKIM_ALLOW(-0.20)[hwinfotech.net:s=dkim]; REPLYTO_EQ_FROM(0.00)[]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:51.79.44.1:c]; PRECEDENCE_BULK(0.00)[]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-0.97)[-0.972]; HAS_LIST_UNSUB(-0.01)[]; RCPT_COUNT_ONE(0.00)[1]; MANY_INVISIBLE_PARTS(0.05)[1]; NEURAL_HAM_MEDIUM(-0.93)[-0.928]; DKIM_TRACE(0.00)[hwinfotech.net:+]; DMARC_POLICY_ALLOW(-0.50)[hwinfotech.net,none]; NEURAL_HAM_SHORT(-1.36)[-1.360]; TO_DN_EQ_ADDR_ALL(0.00)[]; RCVD_COUNT_ZERO(0.00)[0]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; ASN(0.00)[asn:16276, ipnet:51.79.0.0/17, country:FR]; MID_RHS_MATCH_FROM(0.00)[] Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.33 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.33 List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 14 Jun 2020 08:39:11 -0000 We have =E2=9C=85 500+ clone ScriptHi, I am Waseem From HW Infotech. = =C2=A0 Please share your phone number or Skype id, Whatsapp if you are = looking to develop a clone of any website. Work in the following area := =E2=9C=85 Clone Script for any Website and Mobile APP -Development and= Digital Marketing for App, Web, Software Clone Scripts =E2=9C=85Readym= ade Script for useful product -Shop Website Templates, Clone Scripts, & M= arketplace Software | HW Infotech =E2=9C=85Software Requirement Specifica= tion Documentation =E2=9C=85A wireframe design for Website, ERP, Product = &Mobile app =E2=9C=85PHP Framework & MYSQL, HTML5, CSS3, JavaScript, jQue= ry, XML, JSON. =E2=9C=85eCommerce, WordPress,Magento,Open Cart, Joomla, M= agento 2, Prestashop =E2=9C=85Mobile APP- Android, IOS, Phone Gap and F= lutter =E2=9C=85SEO, SMO, and PPC =E2=9C=85Install and config the proje= ct =E2=9C=85Plugin: Customize and create a new plugin =E2=9C=85Migrate = & Upgrade: sustainable development and long-term =E2=9C=85WHM Panel serve= r configuration =E2=9C=85Server Optimization =E2=9C=85Website and Serve= r Penetration testing =E2=9C=85Load Balancing and Failover clustering Ser= ver =E2=9C=85Mail Server configuration for bulk emailing Contact us - H= ire A Developer | Web and App Development Services | HW Infotech Regard= s Waseem www.hwinfotech.com=C2=A0=C2=A0Gurugram | California |London|Va= ncouver|Abuja | Melbourne http://app.hwinfotech.net/index.php/lists/dd2= 96hm0jzd67/unsubscribe/wd423m5jsx768/ok072mq8cc88f From owner-zfs-devel@freebsd.org Mon Jun 22 19:16:26 2020 Return-Path: Delivered-To: zfs-devel@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 396B6338D23 for ; Mon, 22 Jun 2020 19:16:26 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: from mail-ej1-x636.google.com (mail-ej1-x636.google.com [IPv6:2a00:1450:4864:20::636]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 49rJzs2Mmzz4FMG for ; Mon, 22 Jun 2020 19:16:25 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: by mail-ej1-x636.google.com with SMTP id i14so740003ejr.9 for ; Mon, 22 Jun 2020 12:16:25 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=WVYv5yEh5JksOG3rhw3USkwjx+yb7cYx+KbFQJGg9ak=; b=sQCl+rb5iv9xkMprV9M+NBYtWQWvVPnc8JVMu8X0tY7qkhFGQCj9bMlj3Fso/Cb81e PKnR6t7ShHp1l9ZB03R3kZgNJ3ppS4d1qqPJwWOcYYLaBKG+JRXpOGqCKeJZkn//Vq35 mLQ5sAVPNWPLx6hA6l6+QR2VMUmsw4yLvIUqgEjLAucQBGmtw4b6EBmcWgjXx1bAaPCt fgZMfYVD+zuSaSDHZkPSRLUpNFtWPCV8AegTPI9/Y7OsqHckJZL3W42DtZk/hsNlzHm/ 5t5cJeRdHTS7ac6C4QEqmCRl0bi3or9XbEctcllb+qUngcP7GtlCVN1nUITsSMFd0rho e+pA== X-Gm-Message-State: AOAM5333CXFXs0P0uRb+okR+P8ia1Lf2oY1uc07aGXMrYvC6OrE1l+Cd KY1FSBX/tqf5r56ENgeitOGunfhPGDBBOW7RqXKQqeNZ X-Google-Smtp-Source: ABdhPJzyRzwrr0guuyPKg8qdo3Em+8u6axvW06He9jKC23FC/trAzgWFK2O3KvO9UTU07op8hx+VnWiGR13ddvXEqq0= X-Received: by 2002:a17:906:6b8e:: with SMTP id l14mr17642811ejr.32.1592853383176; Mon, 22 Jun 2020 12:16:23 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Matthew Ahrens Date: Mon, 22 Jun 2020 12:16:12 -0700 Message-ID: Subject: June OpenZFS Leadership Meeting To: developer , illumos-zfs , zfs-devel@freebsd.org, freebsd-fs X-Rspamd-Queue-Id: 49rJzs2Mmzz4FMG X-Spamd-Bar: --- X-Spamd-Result: default: False [-3.71 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.96)[-0.963]; R_DKIM_ALLOW(-0.20)[delphix.com:s=google]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36:c]; NEURAL_HAM_LONG(-1.03)[-1.034]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[zfs-devel@freebsd.org]; TO_DN_SOME(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[delphix.com:+]; DMARC_POLICY_ALLOW(-0.50)[delphix.com,quarantine]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::636:from]; NEURAL_HAM_SHORT(-1.01)[-1.008]; FORGED_SENDER(0.30)[mahrens@delphix.com,matthew.ahrens@delphix.com]; MIME_TRACE(0.00)[0:+,1:+,2:~]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[mahrens@delphix.com,matthew.ahrens@delphix.com]; RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.33 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Jun 2020 19:16:26 -0000 The next OpenZFS Leadership meeting will be held tomorrow, June 22, 9am-10am Pacific time. We don't have many topics on the agenda for tomorrow's meeting, so let me know if you'd like to add anything. Everyone is welcome to attend and participate, and we will try to keep the meeting on agenda and on time. The meetings will be held online via Zoom, and recorded and posted to the website and YouTube after the meeting. The agenda for the meeting will be a discussion of the projects listed in the agenda doc. For more information and details on how to attend, as well as notes and video from the previous meeting, please see the agenda document: https://docs.google.com/document/d/1w2jv2XVYFmBVvG1EGf-9A5HBVsjAYoLIFZAnWHhV-BM/edit --matt From owner-zfs-devel@freebsd.org Tue Jun 23 15:47:28 2020 Return-Path: Delivered-To: zfs-devel@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id B2A2333096C for ; Tue, 23 Jun 2020 15:47:28 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: from mail-ed1-x52b.google.com (mail-ed1-x52b.google.com [IPv6:2a00:1450:4864:20::52b]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 49rrJH4v8Zz3TTv for ; Tue, 23 Jun 2020 15:47:27 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: by mail-ed1-x52b.google.com with SMTP id t21so16710197edr.12 for ; Tue, 23 Jun 2020 08:47:27 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=ahBVndu6GT5nK2I9dalmfR2xwzC9c1Tu4Xa9V2aH+wY=; b=W8UbbuCDJky1cK9cKXi15FatB1Dkp9l3AfwvcRawOvkIUm8yeWl2bWKWd6DeR0OfeW pyPiFEyks/rvqTttmVpx+iD4vOHES1+FgTVA4G8NFiM1cWdbhz4XyySnJm02UbCS9i00 dW0KjWGe1fmZtKLaaa9/kbKpkLyLDPTiwtWoCPHTAoMalJrzXitapO5zgMvxibuHGb2J 1YeDauONoAeP5fqibgFkaody2/uq85ku/Rq153p7H2lYEG7KoBpfiqnds0CV0PH+EuGy Gbl9eZUJiHbUaTvJMKucFC6hTrgu/qrvl8bgqJaogl5S3OzU6QhLYiochathGTeHTbZF +JSg== X-Gm-Message-State: AOAM532WEYHFcFKVCwiDDPUJ86uHe5MqJ1lLRi/+PhgV1ycWE+SaoJig qaESoLSKN/bujgDqAkjt2jUwjgvfhc3t3gVvoTOTRvz1G2c= X-Google-Smtp-Source: ABdhPJxBBwlQX2S0n6HXPwc91eSsPJE7eYguIN2LkMFHy76HxlcOC6ASLmzHeg6lWvwXea1LTBMefJutiEAWRQfOOHQ= X-Received: by 2002:a05:6402:1fc:: with SMTP id i28mr23262339edy.63.1592927245949; Tue, 23 Jun 2020 08:47:25 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Matthew Ahrens Date: Tue, 23 Jun 2020 08:47:15 -0700 Message-ID: Subject: OpenZFS Developer Summit 2020 - announcement & CFP To: developer , illumos-zfs , Devel , zfs-devel@freebsd.org, freebsd-fs , zfs-discuss X-Rspamd-Queue-Id: 49rrJH4v8Zz3TTv X-Spamd-Bar: -- X-Spamd-Result: default: False [-2.81 / 15.00]; RCVD_TLS_ALL(0.00)[]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[delphix.com:s=google]; NEURAL_HAM_MEDIUM(-1.00)[-1.004]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36]; NEURAL_HAM_LONG(-1.00)[-0.998]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[zfs-devel@freebsd.org]; URI_COUNT_ODD(1.00)[1]; RCPT_COUNT_FIVE(0.00)[6]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[delphix.com:+]; DMARC_POLICY_ALLOW(-0.50)[delphix.com,quarantine]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::52b:from]; NEURAL_HAM_SHORT(-1.11)[-1.108]; FORGED_SENDER(0.30)[mahrens@delphix.com,matthew.ahrens@delphix.com]; MIME_TRACE(0.00)[0:+,1:+,2:~]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; RCVD_COUNT_TWO(0.00)[2]; FROM_NEQ_ENVFROM(0.00)[mahrens@delphix.com,matthew.ahrens@delphix.com] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.33 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Jun 2020 15:47:28 -0000 The eighth annual OpenZFS Developer Summit will be held online, *September 29-30, 2020*. All OpenZFS developers are invited to participate, and (free!) registration is now open. See http://www.open-zfs.org/wiki/OpenZFS_Developer_Summit for all of the details, including slides and videos from previous years' conferences. The goal of the event is to foster cross-community discussions of OpenZFS work and to make progress on some of the projects we have proposed. The first day of the conference will be set aside for presentations. *If you would like to give a talk, submit a proposal via email to matt@mahrens.org , including a 1-2 paragraph abstract.* Talks may be recorded prior to the event, and speakers' (virtual) attendance the day of the event will allow for everyone to experience the talks together, and for Q&A with attendees. We welcome proposals for any kinds of talks related to ZFS, and we are particularly interested in fostering a diverse group of speakers. In the past, the most popular talks have featured internal details of new or upcoming features in OpenZFS, and explanations of how existing subsystems work. If you're new to the OpenZFS community or new to giving talks at conferences, mentorship is available to help make your presentation the best it can be. The deadlines are as follows: *July 20, 2020 All abstracts/proposals submitted* to matt@mahrens.org July 31, 2020 Proposal submitters notified September 29-30, 2019 OpenZFS Developer Summit The second day of the event will be a hackathon, time set aside to work with OpenZFS developers, with the goal of having something, no matter how insignificant, to demo at the end of the day. This event is only possible because of the efforts and contributions of the community of developers and companies that sponsor the event. Although our costs are lower this year with an online conference, sponsorship opportunities are available. Please see the website for details and send email to victoria@vgfevents.com if you have any questions. --matt > > From owner-zfs-devel@freebsd.org Thu Jun 25 03:38:29 2020 Return-Path: Delivered-To: zfs-devel@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 8EB8034D551 for ; Thu, 25 Jun 2020 03:38:29 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: from mail-ej1-x62a.google.com (mail-ej1-x62a.google.com [IPv6:2a00:1450:4864:20::62a]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 49sm2D4Pg6z4TLr for ; Thu, 25 Jun 2020 03:38:28 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: by mail-ej1-x62a.google.com with SMTP id ga4so4469275ejb.11 for ; Wed, 24 Jun 2020 20:38:28 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=8rXGmcm+CEixvIKsdHACGIx5kBgk+6vEJrQ5pjXRWk4=; b=QlDqc6HhFQMXL6bPnFkiQ3VvBz2Y4Yd/zJYBdCCyGY9xmFiy6NVGAgZa3+sguXx9bK axhQ48bMFpQRb7IuPy/y2TkfhidGnK2QSJ64tBXLOXR1HJUNwAlJYeFZDS6ouLaeV1Uk fkkSdB8rYdsuFgR7BKe+0lN2rByqVishiupsTtbA78f3OpZxcM6hQrD44oNKe6tenQi5 fYGgJmMwmWJfCAQ7LJkZvdiQoBjb6aPD+VadKFKeYRh7ZITws0OZIvCno2HppKjjs3aS 64qWPQU3AmtxsYfBIgGOMqoCUhdhzvhgeu9orLBSS39VECHxrXLnpcqJDpzMX/AmC3xT 3ceA== X-Gm-Message-State: AOAM531/zLi0djQaX52BEJr/6YyGVIlz7Euwt/3S1WERG7inFr6WRXqw E8MFz23jVHzIg5jUJ/2cDtxjGuuMdjue3Gi9bOGQd+rA6ns= X-Google-Smtp-Source: ABdhPJys1QOdnPadSJCm1uTmoaRUoGXTLDcARosCnzrR3DO1m/h6qHJQaUDAbNG5lXcaIRMAMudcnd1T2SxrYvNR8m0= X-Received: by 2002:a17:906:abc9:: with SMTP id kq9mr18291353ejb.493.1593056306798; Wed, 24 Jun 2020 20:38:26 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Matthew Ahrens Date: Wed, 24 Jun 2020 20:38:15 -0700 Message-ID: Subject: Re: June OpenZFS Leadership Meeting To: developer , illumos-zfs , zfs-devel@freebsd.org, freebsd-fs X-Rspamd-Queue-Id: 49sm2D4Pg6z4TLr X-Spamd-Bar: --- X-Spamd-Result: default: False [-3.89 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.04)[-1.038]; R_DKIM_ALLOW(-0.20)[delphix.com:s=google]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36:c]; NEURAL_HAM_LONG(-0.99)[-0.995]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[zfs-devel@freebsd.org]; TO_DN_SOME(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[delphix.com:+]; DMARC_POLICY_ALLOW(-0.50)[delphix.com,quarantine]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::62a:from]; NEURAL_HAM_SHORT(-1.16)[-1.160]; FORGED_SENDER(0.30)[mahrens@delphix.com,matthew.ahrens@delphix.com]; MIME_TRACE(0.00)[0:+,1:+,2:~]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[mahrens@delphix.com,matthew.ahrens@delphix.com]; RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2] Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.33 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Jun 2020 03:38:29 -0000 Thanks to everyone who participated. The video recording is now available: https://youtu.be/Y9HQ4RbqIEw Meeting notes (thanks Serapheim): - OpenZFS Developer Summit 2020 (Matt) - Will be virtual due to COVID - dates 29-30th of September - Registration is open and FREE - Call for presentations is open - If you have done interesting work in/with ZFS we=E2=80=99d love to= hear from you. Let us know by July 20th. - Talks will be recorded and will be streamed on the day of the presentation. Presenters will be in a live-stream at the same time to answer questions. - We=E2=80=99ll try to emulate hallway/water-cooler discussions virtual= ly. There will also be a virtual hackathon. - We=E2=80=99re still looking into the technology to do this. If you ha= ve suggestions from other conferences please let us know - There are still sponsorship opportunities - 501c(3) tax-advantaged contributions - Even though the conference will be online doesn=E2=80=99t mean tha= t there are no costs - We are also trying to create different budgets such as paying for testing infrastructure or reduced ticket prices to future in-perso= n conferences for underrepresented groups. - An email was recently sent on the mailing list with all the info - block reference table for file cloning (Pawel) - File-cloning can be thought of as hard-links with copy-on-write properties - Use-cases include cloning VM images and other large files or being able to recover files from snapshots, fast and without additional spa= ce. - Implementation: - Dedup comes to mind as the end-functionality is somewhat similar. That said we would like this to be a generic feature of ZFS and no= t a feature flag. We=E2=80=99d also like to avoid the performance issu= es that come with dedup. - Current idea revolves around a new block-reference table, a mapping of vdev_id+offset to a refcount. The attempt would be to a= void overhead in reads/writes and just put the overhead in when freeing blocks with more than one reference (to make such operations performant w= e=E2=80=99d probably need to maintain most if not all of our structures/metadata in memory). - There are a few implementation questions that we=E2=80=99d need to iron-out. For example, do we expect this to work across datasets? If that=E2=80=99s the case how would it affect send/receive assuming we want to maintain the cloning-property? What about memory? Can=E2=80=99t that table grow too big? We=E2=80=99d probably need to think hard of how the table is managed and how to avoid unpredictable performance. - Renaming nextboot feature to bootonce (Alan) - There is a naming conflict between the nextboot ioctl used by ZFS and the nextboot feature specific to the FreeBSD bootloader. - Proposal: Change nextboot ioctl name to bootonce - Aliased system properties (Sean) - PR is out: https://github.com/openzfs/zfs/pull/10111 - Still looking for reviewers and feedback - Send toggle -L fix (Matt) - The original bug is discussed in the PR - Doing incremental sends where you mix enabling and disabling the -L flag can cause a bug that zeroes data. On Mon, Jun 22, 2020 at 12:16 PM Matthew Ahrens wrote= : > The next OpenZFS Leadership meeting will be held tomorrow, June 22, > 9am-10am Pacific time. We don't have many topics on the agenda for > tomorrow's meeting, so let me know if you'd like to add anything. > > Everyone is welcome to attend and participate, and we will try to keep th= e > meeting on agenda and on time. The meetings will be held online via Zoom= , > and recorded and posted to the website and YouTube after the meeting. > > The agenda for the meeting will be a discussion of the projects listed in > the agenda doc. > > For more information and details on how to attend, as well as notes and > video from the previous meeting, please see the agenda document: > > > https://docs.google.com/document/d/1w2jv2XVYFmBVvG1EGf-9A5HBVsjAYoLIFZAnW= HhV-BM/edit > > --matt > From owner-zfs-devel@freebsd.org Thu Jul 9 06:58:15 2020 Return-Path: Delivered-To: zfs-devel@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 0EC0C362537 for ; Thu, 9 Jul 2020 06:58:15 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4B2RpG6jDrz4c8m for ; Thu, 9 Jul 2020 06:58:14 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id C8916D141 for ; Thu, 9 Jul 2020 06:58:14 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 0696wE2g061827 for ; Thu, 9 Jul 2020 06:58:14 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 0696wExe061826 for zfs-devel@FreeBSD.org; Thu, 9 Jul 2020 06:58:14 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: zfs-devel@FreeBSD.org Subject: [Bug 203877] NFS threads get blocked when writing to ZFS dataset that has reached quota Date: Thu, 09 Jul 2020 06:58:14 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 10.3-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: linimon@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: assigned_to cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 09 Jul 2020 06:58:15 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D203877 Mark Linimon changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|zfs-devel@FreeBSD.org |fs@FreeBSD.org CC| |linimon@FreeBSD.org --- Comment #4 from Mark Linimon --- Reassign. To submitter: is this aging PR still relevant? --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-zfs-devel@freebsd.org Thu Jul 9 06:58:49 2020 Return-Path: Delivered-To: zfs-devel@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id AEA17362333 for ; Thu, 9 Jul 2020 06:58:49 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4B2Rpx4DBhz4c4w for ; Thu, 9 Jul 2020 06:58:49 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 75327D2DB for ; Thu, 9 Jul 2020 06:58:49 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 0696wnAk062289 for ; Thu, 9 Jul 2020 06:58:49 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 0696wndq062287 for zfs-devel@FreeBSD.org; Thu, 9 Jul 2020 06:58:49 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: zfs-devel@FreeBSD.org Subject: [Bug 180001] [zfs] [panic] Solaris(panic): zfs: allocating allocated segment Date: Thu, 09 Jul 2020 06:58:47 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: Unspecified X-Bugzilla-Keywords: panic X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: linimon@FreeBSD.org X-Bugzilla-Status: In Progress X-Bugzilla-Resolution: X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: assigned_to keywords Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 09 Jul 2020 06:58:49 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D180001 Mark Linimon changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|zfs-devel@FreeBSD.org |fs@FreeBSD.org Keywords| |panic --- Comment #6 from Mark Linimon --- Reassign. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-zfs-devel@freebsd.org Mon Jul 20 20:50:48 2020 Return-Path: Delivered-To: zfs-devel@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 573D9369475 for ; Mon, 20 Jul 2020 20:50:48 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: from mail-ej1-x62b.google.com (mail-ej1-x62b.google.com [IPv6:2a00:1450:4864:20::62b]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4B9Ylq1bnHz4YNC for ; Mon, 20 Jul 2020 20:50:47 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: by mail-ej1-x62b.google.com with SMTP id n22so16535186ejy.3 for ; Mon, 20 Jul 2020 13:50:47 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=E2OPbLac3E5UsWAQt9R6Hq8me1QbxyU88PwiMvjN92o=; b=EmLJ6UDM1WRJD4cGJEn2ElMQDkc8SU1ArMYEjU/pQbEgClosKKs1AmXL8PjBgF9AYA 4fLPO7qLcnYlUsxCF4a1ePbZXiAyn9OcGjlWfVX95lT3/pz+wBYkP8wZf64+7XsWuyBx aJZck8uTAcORSirTtlRn5Mg89UxSsM4sxp7U9MAkdFKNgy+Ep0WW4pkezVJc03L7igXl BD0w5Bsq3r3fShQfbhSZ77r0IRDnhpSlAAGyaYS22BTTiQ+MHCg9sDmo18fGQe6cwHCl Pm0AlOWh70L/uxmSF47t12Y4iUsmtoGY5htkq4lLdRULCttRcFYto+iygxdBDbNzS0yK c6Rw== X-Gm-Message-State: AOAM531+CNXImeWTj0el3XbQtpqOoBk96A4ZJMVi6VpG5E5uhmFB8qYG v5VF/ztShjWTzx0Zoo7mbpsZPLuDH0S78YoywjjXcA== X-Google-Smtp-Source: ABdhPJxAOzG4HwkpPM06ipBdOWHvnSLrYkiJvl2QIKtg3R0zcW06+aufOnictblLJD0atVTcrj1JsH+9I3cjK427RrU= X-Received: by 2002:a17:906:c56:: with SMTP id t22mr23514722ejf.50.1595278245113; Mon, 20 Jul 2020 13:50:45 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Matthew Ahrens Date: Mon, 20 Jul 2020 13:50:34 -0700 Message-ID: Subject: NEW DATE for OpenZFS Developer Summit, and call for presentations To: developer , illumos-zfs , Devel , zfs-devel@freebsd.org, freebsd-fs X-Rspamd-Queue-Id: 4B9Ylq1bnHz4YNC X-Spamd-Bar: -- X-Spamd-Result: default: False [-2.94 / 15.00]; RCVD_TLS_ALL(0.00)[]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[delphix.com:s=google]; NEURAL_HAM_MEDIUM(-0.97)[-0.972]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36]; NEURAL_HAM_LONG(-1.01)[-1.006]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[zfs-devel@freebsd.org]; URI_COUNT_ODD(1.00)[1]; RCPT_COUNT_FIVE(0.00)[5]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[delphix.com:+]; DMARC_POLICY_ALLOW(-0.50)[delphix.com,quarantine]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::62b:from]; NEURAL_HAM_SHORT(-1.26)[-1.259]; FORGED_SENDER(0.30)[mahrens@delphix.com,matthew.ahrens@delphix.com]; MIME_TRACE(0.00)[0:+,1:+,2:~]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; RCVD_COUNT_TWO(0.00)[2]; FROM_NEQ_ENVFROM(0.00)[mahrens@delphix.com,matthew.ahrens@delphix.com] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.33 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 20 Jul 2020 20:50:48 -0000 A few of the key dates for the OpenZFS Developer Summit have changed. Due to a scheduling conflict, we are moving the conference date out by 1 week, to *October 6-7, 2020*. We are also extending the deadline for presentation proposals by 2 weeks. *If you would like to give a talk, submit a proposal by August 3rd to matt@mahrens.org , including a 1-2 paragraph abstract.* Speakers will be notified by August 14th. See http://www.open-zfs.org/wiki/OpenZFS_Developer_Summit for all of the details, including slides and videos from previous years' conferences. This event is only possible because of the efforts and contributions of the community of developers and companies that sponsor the event. Although our costs are lower this year with an online conference, sponsorship opportunities are available. Please see the website for details and send email to victoria@vgfevents.com if you have any questions. --matt From owner-zfs-devel@freebsd.org Mon Jul 20 21:27:16 2020 Return-Path: Delivered-To: zfs-devel@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 70621369FF1 for ; Mon, 20 Jul 2020 21:27:16 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: from mail-ej1-x631.google.com (mail-ej1-x631.google.com [IPv6:2a00:1450:4864:20::631]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4B9ZYv2lZ6z4bFH for ; Mon, 20 Jul 2020 21:27:15 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: by mail-ej1-x631.google.com with SMTP id br7so19553482ejb.5 for ; Mon, 20 Jul 2020 14:27:15 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=i9UB7aDujw23ZDwV2RqimZVY2SG79qiBNtdJ1nOPHZk=; b=dVzLw8QUsuVqA+yacQMksLbBSiYNXdb2i9dVh/HwnpUDnrm069VAJ+TUxDvH5Wi0yh IXKDJK0OFLi08ya629dIxFIUyZ4B8IcVqy8JkYgCwxtWivmG1rr/2KKvBtxbq3HGo0Xo yBCyeJTSXGxOEAuDkd4eHxLiNKsQ/4nkcuAs/RVuWxgsLeiLaBM3QTCHnbzuDyJ6AWXp W8wq8gbv6p7VO5mN/TIWRCsZjX2kJNKzoFJcP3eg0TW8HJhRXBKgDYf5Lff1vh9u6Q6u PY/n2Bms7mZmmGZMwef+3fxOeZS8HqVobTisI9SVUKUL0rNRhMeyoGWCEK8Sa4YPQJE3 nPqA== X-Gm-Message-State: AOAM530l+rD9EKAmfkIedkB2oieamRdJF/Se3wjh85oEi6NGP7YZ2eic Ap4QUKMgko2IS5EZO4JaMrudD5uyVeqNwrQNTzFjYQ== X-Google-Smtp-Source: ABdhPJzBrXuibmpTr1fc+jX+F91qx4O6BELMASj0aPwJvUph027OJNTgBxRC5owMQm+zvPpAxTQ7OqHukavruF5awG4= X-Received: by 2002:a17:906:abd6:: with SMTP id kq22mr23509054ejb.458.1595280432782; Mon, 20 Jul 2020 14:27:12 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Matthew Ahrens Date: Mon, 20 Jul 2020 14:27:01 -0700 Message-ID: Subject: July OpenZFS Leadership Meeting To: developer , illumos-zfs , zfs-devel@freebsd.org, freebsd-fs X-Rspamd-Queue-Id: 4B9ZYv2lZ6z4bFH X-Spamd-Bar: --- X-Spamd-Result: default: False [-3.58 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.97)[-0.968]; R_DKIM_ALLOW(-0.20)[delphix.com:s=google]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36:c]; NEURAL_HAM_LONG(-1.03)[-1.031]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[zfs-devel@freebsd.org]; TO_DN_SOME(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[delphix.com:+]; DMARC_POLICY_ALLOW(-0.50)[delphix.com,quarantine]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::631:from]; NEURAL_HAM_SHORT(-0.88)[-0.882]; FORGED_SENDER(0.30)[mahrens@delphix.com,matthew.ahrens@delphix.com]; MIME_TRACE(0.00)[0:+,1:+,2:~]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[mahrens@delphix.com,matthew.ahrens@delphix.com]; RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.33 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 20 Jul 2020 21:27:16 -0000 The next OpenZFS Leadership meeting will be held tomorrow, July 20, 1pm-2pm Pacific time. We don't have many topics on the agenda for tomorrow's meeting, so let me know if you'd like to add anything. Everyone is welcome to attend and participate, and we will try to keep the meeting on agenda and on time. The meetings will be held online via Zoom, and recorded and posted to the website and YouTube after the meeting. The agenda for the meeting will be a discussion of the projects listed in the agenda doc. For more information and details on how to attend, as well as notes and video from the previous meeting, please see the agenda document: https://docs.google.com/document/d/1w2jv2XVYFmBVvG1EGf-9A5HBVsjAYoLIFZAnWHhV-BM/edit --matt From owner-zfs-devel@freebsd.org Thu Jul 23 16:28:17 2020 Return-Path: Delivered-To: zfs-devel@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id C6F2E35B8CD for ; Thu, 23 Jul 2020 16:28:17 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: from mail-ej1-x643.google.com (mail-ej1-x643.google.com [IPv6:2a00:1450:4864:20::643]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4BCHnY2JP5z3X1r for ; Thu, 23 Jul 2020 16:28:17 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: by mail-ej1-x643.google.com with SMTP id rk21so7067027ejb.2 for ; Thu, 23 Jul 2020 09:28:17 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=kM2vipw6loav5ZDNcbKPiMD24+KiFHpEH2trZPbpaQw=; b=EqdjUMwIEnQyGtii8MTvinlmM+p9Kg+InoZs5Y2cTJ5vRcyVd8YTOqvDifOf3ODYUS JR1T/ChYxXQ5J1Uf52WzOFxpmVSKtKziiDuELdFCEVrFYkwpgZWjc/GcSs6P3p1IPHBE VSOpJ2UBVMDVYWocqa/YudJjObk/CTQFmaN+SHRkbQIq5rzWaX6tox1yRrf1+jP5Qjwc TUApJWgrp5oHgx9PNWwSKi1PIU8YIhJ5PE3l+ikhUWfItxXyxZ98qeWe80EjJSe8S1kc D2buJ615Esm1yskOkv4J2n3waukTVfIMqWHqQSAxkgByjWGU104C8mYT+3MnD0XYWfoR YYTg== X-Gm-Message-State: AOAM533usaj2GN0whESjl61YpvdoeQYZGinq+s+mevMHv/t/Fz9o+Rip y66Q93aSCMS4fTEgygJHSHPiy0/o0ImaEVLLU7+/RA== X-Google-Smtp-Source: ABdhPJzyV551d3cMhUiJby5sAOrbMeBrpNoebsYJNYKZVvYLMBgjEtyxx1h58BekzPO4Uklqov7JlsjWdOqmS4XXF+g= X-Received: by 2002:a17:906:abc9:: with SMTP id kq9mr5159466ejb.493.1595521695512; Thu, 23 Jul 2020 09:28:15 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Matthew Ahrens Date: Thu, 23 Jul 2020 09:28:04 -0700 Message-ID: Subject: Re: July OpenZFS Leadership Meeting To: developer , illumos-zfs , zfs-devel@freebsd.org, freebsd-fs X-Rspamd-Queue-Id: 4BCHnY2JP5z3X1r X-Spamd-Bar: --- X-Spamd-Result: default: False [-3.60 / 15.00]; RCVD_TLS_ALL(0.00)[]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[delphix.com:s=google]; NEURAL_HAM_MEDIUM(-1.00)[-0.999]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[zfs-devel@freebsd.org]; NEURAL_HAM_LONG(-1.04)[-1.041]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[delphix.com:+]; DMARC_POLICY_ALLOW(-0.50)[delphix.com,quarantine]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::643:from]; NEURAL_HAM_SHORT(-0.86)[-0.857]; FORGED_SENDER(0.30)[mahrens@delphix.com,matthew.ahrens@delphix.com]; MIME_TRACE(0.00)[0:+,1:+,2:~]; RCVD_COUNT_TWO(0.00)[2]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[mahrens@delphix.com,matthew.ahrens@delphix.com] Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.33 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Jul 2020 16:28:17 -0000 Thanks to everyone who participated at this week's meeting. The video recording is now available: https://youtu.be/rKK1pfNt66g Meeting notes (thanks Serapheim): - OpenZFS Developer Summit (Matt) - Event pushed 1 week later than the initial date - to the beginning of October - Talk Submission deadline extended by 2 weeks - If you have things you want to hear about feel free to ping Matt and he can bug the right people - ZFS send compatibility for ZSTD (Allan) - Problem: Creating a stream that the receiving side can=E2=80=99t unde= rstand (e.g. the receiving machine doesn=E2=80=99t know about ZSTD because i= t uses an older ZFS version) - Multiple ways to go about it - there was common agreement that whatever the solution it should probably follow the following princip= les: - The receiving side should be able to tell the user whether it supports receiving the given stream, hopefully with a friendly error message - The default behavior in general should be to use the latest and most common thing but also give the option for the user to explici= tly downrev/configure it to their needs. - E.g. people rarely do plain `zfs send` without -c - The above principles are not expected to be enforced/implemented for the ZSTD PR but they should guide the future direction of dealing with this issue. The ZSTD PR will proceed as discussed. - Activating feature flags on =E2=80=98zfs set=E2=80=99 instead of first b= lock birth (ZSTD) (Allan) - Problem: Activating a feature flag for a dataset, but not enabling it (e.g. haven=E2=80=99t written any blocks yet), can cause panics when = the dataset is opened by a ZFS version that doesn=E2=80=99t know about that feature. - This issue has been brought up in the past, the next steps would be to have someone implement the fix. - What are the plans for OpenZFS release? (Alexander Motin) - There are no features officially holding the release. - Brian will create the 2.0 version branch mid-August and probably make the release at some point before the end of the year. - The master branch will remain open for development during that time. - Any bug fixes applicable to the 2.0 branch will be =E2=80=9Cbackporte= d=E2=80=9D from master. - Forceful export - zpool export -F (Allan Jude) - Functionality: A pool=E2=80=99s suspension doesn=E2=80=99t cause ZFS = operations to hang for other pools (e.g. no holding the namespace lock. etc..). In addition the suspended pool can be exported. On Mon, Jul 20, 2020 at 2:27 PM Matthew Ahrens wrote: > The next OpenZFS Leadership meeting will be held tomorrow, July 20, > 1pm-2pm Pacific time. We don't have many topics on the agenda for > tomorrow's meeting, so let me know if you'd like to add anything. > > Everyone is welcome to attend and participate, and we will try to keep th= e > meeting on agenda and on time. The meetings will be held online via Zoom= , > and recorded and posted to the website and YouTube after the meeting. > > The agenda for the meeting will be a discussion of the projects listed in > the agenda doc. > > For more information and details on how to attend, as well as notes and > video from the previous meeting, please see the agenda document: > > > https://docs.google.com/document/d/1w2jv2XVYFmBVvG1EGf-9A5HBVsjAYoLIFZAnW= HhV-BM/edit > > --matt > From owner-zfs-devel@freebsd.org Tue Aug 18 03:47:31 2020 Return-Path: Delivered-To: zfs-devel@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id A4A803ACF87 for ; Tue, 18 Aug 2020 03:47:31 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: from mail-ed1-x52a.google.com (mail-ed1-x52a.google.com [IPv6:2a00:1450:4864:20::52a]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4BVxgk3zvBz4fFp for ; Tue, 18 Aug 2020 03:47:29 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: by mail-ed1-x52a.google.com with SMTP id w17so14088468edt.8 for ; Mon, 17 Aug 2020 20:47:29 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=olXzkdtJh/e6oVqR2Q/LbnFBbVqtlyidrcDxU+uqTKU=; b=ubq0vDkUfFcRuYSvXC7Mbf7BMklXZp4Bt0hrDLzhJc5jIrr7QK2jOCUEwaxXP4xrLO PQJr0qlMkDUoetBrDWu/sad3uJkd271f6GPWczeVU77AH23U+sqvsE/5zVDOJC5toy+m tw/hLTSrNxScIMseVp4yn2+JvPoTE9Pir33oxhqlqQe7595YO1DQi8D0Mzpj9Oy8g31l 0gxqVMsTekRdBrR2Ri2Z1ahgGa7sQtH+yEHOAlXzQUeklklQfk31yj/GX1qjaB3rggYo IrkeSg4eFbWGMDGqnqnWZe+Q6faAmvoZCjUbBBNoZ3FTNzVNQmbFz1klvdjVL7pBoKFu bHoQ== X-Gm-Message-State: AOAM532NmpYTceYLZ45FRr/faWxhrxdf0JXk8aEYfWc0Q2GrJkkokdWp aPlLafvY1pQ1azWgb1Y+UB0cpapBHBLccV3Zvty2Ow== X-Google-Smtp-Source: ABdhPJyWHfXrnfSPNWivToubU3RM2dlUiGNv4CTJQhc1+bvFluimafwoSVTf+jWXWrHmkAzJ1BFD+YjqbUT8UC/80jg= X-Received: by 2002:a50:ef04:: with SMTP id m4mr18559079eds.63.1597722448068; Mon, 17 Aug 2020 20:47:28 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Matthew Ahrens Date: Mon, 17 Aug 2020 20:47:16 -0700 Message-ID: Subject: August OpenZFS Leadership Meeting To: developer , illumos-zfs , zfs-devel@freebsd.org, freebsd-fs ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1597722451; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references:dkim-signature; bh=olXzkdtJh/e6oVqR2Q/LbnFBbVqtlyidrcDxU+uqTKU=; b=FpcFEJsbCwoKLiqab5gEyIISwk94p9hdq4cY7kMuYeCOfT9Xu83l2h3E/s+yyw79AYl/QT cVRcV5u81EcgRuZOD6PBuRpnwbKm1TqenIziNVSnPWDxDG1Opvkbp0F16+LfRu7RHjaqlY jifpDF2UQbiG47VZrQ07zCnJTubqyQrU/3qkVj6KUTvUbIMHMj3vFkNqKioMO0FIf53+Ao M7KbhoYPsuamZHIgIn6A97FexvKrkFCBylOnPrIMgevPdGn4xRXUcVJNCtN8JHJeQySoAP H0RTIdncAAm0lrm71iRxpNxm0mVB8gAhU7rdV3EuBUSdLetgizJ/RORqUGipxA== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1597722451; a=rsa-sha256; cv=none; b=Ce3FdOmyeJeQEwacAwvIhjdSdfhyLNPjUoyPaJXVwBvlK3R9HftYCOjQz3M4hjG4HFkvDE eX2UBWVca2jmchkKrDDONZHk/1uuIkiAZe5QXjr3fscaEjh1PAqy5KQ3lFiYq78Nu51SnJ U4WNpN4O81C36qvpH3IcjwZJ9hRSFtMjy3whscYZsbWSh5tY43ATf+NydR3Uv2wXhLdQs4 VFBd2oIQUhoiK2FFljt93/J3tpXYYovCdKxd0G1UBK034k7Qfos60LuXIOOj227ogYjXJD Q+EADRibIartw5gbKwZg+45XacW/Xu/fqqENE/F6UIpuAdj/u5CjcQbmtfbhXg== ARC-Authentication-Results: i=1; mx1.freebsd.org; dkim=pass header.d=delphix.com header.s=google header.b=FZvUDcXT; spf=pass (mx1.freebsd.org: domain of matthew.ahrens@delphix.com designates 2a00:1450:4864:20::52a as permitted sender) smtp.mailfrom=matthew.ahrens@delphix.com X-Rspamd-Queue-Id: 4BVxgk3zvBz4fFp X-Spamd-Bar: --- X-Spamd-Result: default: False [-3.29 / 15.00]; RCVD_TLS_ALL(0.00)[]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[delphix.com:s=google]; NEURAL_HAM_MEDIUM(-1.00)[-0.996]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[zfs-devel@freebsd.org]; ARC_SIGNED(0.00)[i=1]; NEURAL_HAM_LONG(-0.99)[-0.989]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[delphix.com:+]; DMARC_POLICY_ALLOW(-0.50)[delphix.com,quarantine]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::52a:from]; NEURAL_HAM_SHORT(-0.61)[-0.608]; FORGED_SENDER(0.30)[mahrens@delphix.com,matthew.ahrens@delphix.com]; MIME_TRACE(0.00)[0:+,1:+,2:~]; RCVD_COUNT_TWO(0.00)[2]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[mahrens@delphix.com,matthew.ahrens@delphix.com] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.33 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Aug 2020 03:47:31 -0000 The next OpenZFS Leadership meeting will be held tomorrow, August 18, 1pm-2pm Pacific time. We don't have many topics on the agenda for tomorrow's meeting, so let me know if you'd like to add anything. FYI, the talks for this year's OpenZFS Developer Summit have been posted on the website . Everyone is welcome to attend and participate, and we will try to keep the meeting on agenda and on time. The meetings will be held online via Zoom, and recorded and posted to the website and YouTube after the meeting. The agenda for the meeting will be a discussion of the projects listed in the agenda doc. For more information and details on how to attend, as well as notes and video from the previous meeting, please see the agenda document: https://docs.google.com/document/d/1w2jv2XVYFmBVvG1EGf-9A5HBVsjAYoLIFZAnWHhV-BM/edit --matt From owner-zfs-devel@freebsd.org Tue Aug 18 15:40:57 2020 Return-Path: Delivered-To: zfs-devel@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id F0CC33BFED9 for ; Tue, 18 Aug 2020 15:40:57 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: from mail-ej1-x62e.google.com (mail-ej1-x62e.google.com [IPv6:2a00:1450:4864:20::62e]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4BWFVx1WpWz4GNK for ; Tue, 18 Aug 2020 15:40:57 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: by mail-ej1-x62e.google.com with SMTP id g19so22597155ejc.9 for ; Tue, 18 Aug 2020 08:40:56 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=NPfhLmsE8rQzNVU25E9Fvk735KzzGpjJ8G+Rqtw9IH4=; b=d3q+ikoINOgN0cNcAWLd1XnRxRMy/bb5f2urU0EZNtj4eRm2Dcm3ufU6B1EAvdkoSp hIFsmi9oYvt9H2AMbHyh0MQvoJwVE7GQlKhHr2GGkoLV9BS98l/wbeWzf3Fwx+K/E52u oVtuvYZHtGIEda+cMGz+AlBvRfuu6eHN4ViJQpHaIXhH+VYpVM0GBU5APkswbykfhu9x f5X0tjG6xGA47AFfeKpfV2frxpSBJpFJLlA2P7Ad6/hq6zHp4uS5gkTsH5xauDP3xEbR GxKEt+HZt2c0v+6MynGEXZ9uIWcjU/RMdBZpW8ATQDcuXdlUw5b1QptLegUufLBLRR6P hQ5w== X-Gm-Message-State: AOAM532M9CTH8PsEULY420ZOTULPGAkhfdrcSFnd1fQZhufhhtdP5JRS piI2UaPli2sL//lGXh3W0h/0TNVLGxqJ+cLAvbAhv6/YKa+Vlg== X-Google-Smtp-Source: ABdhPJxHFXdGORNIjENoGe09YYVkwZQuSvygZQfTFT4mZ3gOZE6sao3Su+nAroAGQYqSemXKaY8GUF6ElnFbFKMI8bY= X-Received: by 2002:a17:906:d159:: with SMTP id br25mr20062649ejb.16.1597765255440; Tue, 18 Aug 2020 08:40:55 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Matthew Ahrens Date: Tue, 18 Aug 2020 08:40:44 -0700 Message-ID: Subject: OpenZFS Developer Summit speakers & talks announcement To: developer , illumos-zfs , zfs-devel@freebsd.org, freebsd-fs X-Rspamd-Queue-Id: 4BWFVx1WpWz4GNK X-Spamd-Bar: / X-Spamd-Result: default: False [-0.96 / 15.00]; RCVD_TLS_ALL(0.00)[]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[delphix.com:s=google]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[zfs-devel@freebsd.org]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[delphix.com:+]; DMARC_POLICY_ALLOW(-0.50)[delphix.com,quarantine]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::62e:from]; NEURAL_HAM_SHORT(-0.26)[-0.263]; FORGED_SENDER(0.30)[mahrens@delphix.com,matthew.ahrens@delphix.com]; MIME_TRACE(0.00)[0:+,1:+,2:~]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[mahrens@delphix.com,matthew.ahrens@delphix.com]; MAILMAN_DEST(0.00)[zfs-devel]; RCVD_COUNT_TWO(0.00)[2] Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.33 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Aug 2020 15:40:58 -0000 I'm pleased to announce the eleven great talks that we have lined up for the OpenZFS Developer Summit , which will be held October 6-7 this year. The conference will be live-streamed, and you can register (for free!) to participate in Q&A with the speakers, breakout discussions with other attendees, and the hackathon! TitleSpeakerCompany State of OpenZFS Matt Ahrens Delphix ZIL Performance Improvements for Fast Media Saji Nair Nutanix Sequential Reconstruction Mark Maybee Cray dRAID, Finally! Mark Maybee Cray Persistent L2ARC George Amanakis Independent ZFS Caching: How Big Is the ARC? George Wilson Delphix Improving =E2=80=9Czfs diff=E2=80=9D performance with reverse-name lookup S= anjeev Bagewadi & David Chen Nutanix Send/Receive Performance Enhancements Matt Ahrens Delphix Performance Troubleshooting Tools Gaurav Kumar Nutanix Default-Compatible Pool Features Josh Paetzel Panzura File Cloning with Block Reference Table Pawel Dawidek Fudo Security --matt > From owner-zfs-devel@freebsd.org Wed Aug 19 17:30:58 2020 Return-Path: Delivered-To: zfs-devel@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 453983C5440 for ; Wed, 19 Aug 2020 17:30:58 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: from mail-ej1-x634.google.com (mail-ej1-x634.google.com [IPv6:2a00:1450:4864:20::634]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4BWvvP1WM5z4Kbq for ; Wed, 19 Aug 2020 17:30:56 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: by mail-ej1-x634.google.com with SMTP id jp10so27296824ejb.0 for ; Wed, 19 Aug 2020 10:30:56 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=/3X1GrTBtQNAGa/arEUIoKQbRfvVeB7S46OiK/46cIU=; b=tWj2IskRYm5TodbhYd7OEbN2gJdqiLnXg4lpH5khbRZHCGQvt14nEYNbnAuTdePJJ+ wK+oT6Uxr0rSWHeZqH6ARdg1Vb5YrIi9khtMSNKBSTtOWMaMXp37BLtfB71uj043o1dE ltAHxH8GLnOetrFiV722eH2Y84BMdWDzouDkdTP+knXrx5Cx6KB2ckBnr4/GAFxRCxp2 5pTguPhCFhR4tzMpA3JVWv3PBr6Dwg7bd1JXNGN/mGI6zBJK/+r3Zr0sbxZqmGPBshDs vJyheTXeHf8CgNQVSBK0cYGn07MKUWbjcVYZPjmMavuRcyxBK5QoakkyzKgvj1FStwfL x9/w== X-Gm-Message-State: AOAM530m2xZMnmOJm3YrMspqq/LveANDHu9XxvGZLWIq6whvqpbo2lDi 8cpNgpRFkV6YOy0kwtv3CebyO8SvlYFDlzTWRhlMuQ== X-Google-Smtp-Source: ABdhPJyQJVgelqnGUrl8ss/E3HnisxPjYbiq1s7sHDIRL32FIXv+AvOmYH8mn8ycP4zhtI16eXksYe9eoqsq/djAX8U= X-Received: by 2002:a17:906:364e:: with SMTP id r14mr24973453ejb.295.1597858255252; Wed, 19 Aug 2020 10:30:55 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Matthew Ahrens Date: Wed, 19 Aug 2020 10:30:44 -0700 Message-ID: Subject: Re: August OpenZFS Leadership Meeting To: developer , illumos-zfs , zfs-devel@freebsd.org, freebsd-fs X-Rspamd-Queue-Id: 4BWvvP1WM5z4Kbq X-Spamd-Bar: / X-Spamd-Result: default: False [-0.88 / 15.00]; ARC_NA(0.00)[]; MAILMAN_DEST(0.00)[zfs-devel]; R_DKIM_ALLOW(-0.20)[delphix.com:s=google]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[zfs-devel@freebsd.org]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[delphix.com:+]; DMARC_POLICY_ALLOW(-0.50)[delphix.com,quarantine]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::634:from]; NEURAL_HAM_SHORT(-0.18)[-0.182]; FORGED_SENDER(0.30)[mahrens@delphix.com,matthew.ahrens@delphix.com]; MIME_TRACE(0.00)[0:+,1:+,2:~]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; FROM_NEQ_ENVFROM(0.00)[mahrens@delphix.com,matthew.ahrens@delphix.com]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US] Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.33 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Aug 2020 17:30:58 -0000 Thanks to everyone who participated in this month's meeting The video recording is now available: https://www.youtube.com/watch?v=3DOpvyKVjHEFE Notes (thanks Serapheim): - OpenZFS Developer Summit (Matt) - Speakers have been announced and registration is open - First day will be talks, second day will be the hackathon - The hackathon this year will be slightly more organized, as participation is virtual. There will be a small idea pitching session= at first before people break out to per-project sessions. - We are looking for people to lead the hackathon project sessions. If you are working towards something as part of the hackathon you = can probably create a session for it and others can join to help. - Boot once API changes (extend Delphix nextboot with nvlist support, want to get the signatures stable before 2.0) (Allan) - There is a working prototype which has undergone some preliminary testing - Allan and Toomas hope to have the API change in before the 2.0 release so it becomes part of stable. - dRAID (Brian) - Distributed parity/spare implementation for ZFS. - The feature has been wrapping up - there is a PR that needs more reviewers. Users are also welcome to clone that branch and test the c= ode. - Mark Maybee will give a talk about this. - 2.0 branching (Brian) - Plan was originally to branch mid-August, we are close to that - the timeline has been stretched by a couple of weeks. - There is a possibility that DRAID will be part of it too. - meta: move to or explicitly endorse semantic versioning for release versions =C2=B7 Issue #10334 (Gabriel Devenyi) - Discussion around having concrete information/communication for the patch/release branches of the code. E.g. What do patch releases conta= in? What branches do we maintain in parallel and for how long? - So far the policy has been that for major release branches we don=E2= =80=99t add new features or on-disk format changes, we provide fixes for kern= el bugs and compatibility for new kernels, we sometimes include performa= nce improvements if they are not very invasive. The cadence of those rele= ases and their updates is roughly 3 months in general. - The point raised by Gabriel is that users may need some of these kernel/compatibility fixes earlier in some branches. - L2ARC cache policy (Georgios= ) - Discussion started up on Github for the policy; MFU vs MRU, data vs metadata, etc.. - the main question raised was how do we move forward from here to allow users to get the most performance. Should we allow a ke= rnel module parameter? A full-fledged knob from the userlard that is per-dataset? Should we just go ahead and change the default and leave it at that? - There are two guiding principles for answering the above: - [1] Is to incorporate more observability into the policy so we can make points through data-driven observations. - [2] If we are to provide a user-knob - we should be able to easily answer the question of what the knob does and when to turn it. - dnode_sync is careless with range tree =C2=B7 Issue #10708 =C2=B7 openzf= s/zfs (jclulow/pmooney; we would like someone to take a look!) - George and/or Matt will be taking a look at this On Mon, Aug 17, 2020 at 8:47 PM Matthew Ahrens wrote: > The next OpenZFS Leadership meeting will be held tomorrow, August 18, > 1pm-2pm Pacific time. We don't have many topics on the agenda for > tomorrow's meeting, so let me know if you'd like to add anything. > > FYI, the talks for this year's OpenZFS Developer Summit have been posted > on the website > . > > Everyone is welcome to attend and participate, and we will try to keep th= e > meeting on agenda and on time. The meetings will be held online via Zoom= , > and recorded and posted to the website and YouTube after the meeting. > > The agenda for the meeting will be a discussion of the projects listed in > the agenda doc. > > For more information and details on how to attend, as well as notes and > video from the previous meeting, please see the agenda document: > > > https://docs.google.com/document/d/1w2jv2XVYFmBVvG1EGf-9A5HBVsjAYoLIFZAnW= HhV-BM/edit > > --matt > From owner-zfs-devel@freebsd.org Mon Sep 7 14:34:31 2020 Return-Path: Delivered-To: zfs-devel@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 666E13CB134 for ; Mon, 7 Sep 2020 14:34:31 +0000 (UTC) (envelope-from noreply@tovarioseni8.fun) Received: from tovarioseni8.fun (tovarioseni8.fun [62.4.16.55]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4BlW526ld0z49vx for ; Mon, 7 Sep 2020 14:34:30 +0000 (UTC) (envelope-from noreply@tovarioseni8.fun) Received: by tovarioseni8.fun for ; Mon, 7 Sep 2020 16:34:30 +0200 (envelope-from ) Message-ID: <04929d8672a251795aa452010adee5b99a625b77eb@tovarioseni8.fun> Reply-To: Woodward Stoops From: Woodward Stoops To: zfs-devel@freebsd.org Subject: Receipt 2137555 Date: Mon, 7 Sep 2020 07:34:28 -0700 MIME-Version: 1.0 Precedence: bulk X-Rspamd-Queue-Id: 4BlW526ld0z49vx X-Spamd-Bar: - X-Spamd-Result: default: False [-1.37 / 15.00]; HAS_REPLYTO(0.00)[noreply@tovarioseni8.fun]; R_SPF_ALLOW(-0.20)[+ip4:62.4.16.55:c]; TO_DN_NONE(0.00)[]; URI_COUNT_ODD(1.00)[1]; DKIM_TRACE(0.00)[tovarioseni8.fun:+]; DMARC_POLICY_ALLOW(-0.50)[tovarioseni8.fun,reject]; RCVD_COUNT_ZERO(0.00)[0]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; ASN(0.00)[asn:12876, ipnet:62.4.0.0/19, country:FR]; MID_RHS_MATCH_FROM(0.00)[]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.97)[-0.969]; R_DKIM_ALLOW(-0.20)[tovarioseni8.fun:s=mail]; REPLYTO_EQ_FROM(0.00)[]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; PRECEDENCE_BULK(0.00)[]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[zfs-devel@freebsd.org]; NEURAL_HAM_LONG(-0.97)[-0.971]; HAS_LIST_UNSUB(-0.01)[]; RCPT_COUNT_ONE(0.00)[1]; SUBJECT_ENDS_SPACES(0.50)[]; NEURAL_SPAM_SHORT(0.08)[0.083]; MAILMAN_DEST(0.00)[zfs-devel] Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.33 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.33 List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Sep 2020 14:34:31 -0000 Hello Dear client, Legal receipt #92375 Click here for download doc Best Regards From owner-zfs-devel@freebsd.org Mon Sep 14 18:58:27 2020 Return-Path: Delivered-To: zfs-devel@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id BF8393DF76A for ; Mon, 14 Sep 2020 18:58:27 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: from mail-ej1-x635.google.com (mail-ej1-x635.google.com [IPv6:2a00:1450:4864:20::635]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4BqwcL3Xdxz4Csq for ; Mon, 14 Sep 2020 18:58:26 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: by mail-ej1-x635.google.com with SMTP id i22so1483996eja.5 for ; Mon, 14 Sep 2020 11:58:26 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=kh+Ls8ILjUC5YOWoPRpI6Xe3PzjIoRq6ihIlnvmKXsQ=; b=FAU6TCRHtGrCv8LQcyE/yRyzRHVvWakoVDkT85z7HHljVT+uSGZuNMtVlIv1IJ7/x+ 6+ukxXXduYiq1zqygAHb8FYTApQ49ziRTYCem70rgYDN8+SPhWwnT8scS2wmVf46i0Ao 0rJpTRgQGiIiwzd9x9mkXh7sp7UmG+Yq0jGQRJGYEEnuN3gUNcIUDvK7QkXYl34CF6BQ EpUt6rK55LP2YZNl77JG1FrYwa/p98WrjJS5b01M7Tn2/JxIeVcU8S/XExQA8ew9GtSu NRG7ktQ9Y165rKBGUK96ZUnU4hKZf3bX/o7XkrkOkTHBelmUuW3dDSLhiIwu8fkWRSaE 1vtg== X-Gm-Message-State: AOAM532FVNuLU8bTXOLC7CpsGGhiRuJH7shAlj9YPytR4Wly9ilZfiMA c2JSPjVujuVYKGUgIs1Pgn8L4/XcvWLAFeD0KKrjurRVlu8= X-Google-Smtp-Source: ABdhPJw+T0MPQOdL8/UZhJJ6dZOU6W/UUaKNmx2OmWtdV4ENF7HMiOPyvQe1rpm9FES99AS8uFT6ke+lm86bltPuj+8= X-Received: by 2002:a17:906:1691:: with SMTP id s17mr16856368ejd.458.1600109904590; Mon, 14 Sep 2020 11:58:24 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Matthew Ahrens Date: Mon, 14 Sep 2020 11:58:13 -0700 Message-ID: Subject: September OpenZFS Leadership Meeting To: developer , illumos-zfs , zfs-devel@freebsd.org, freebsd-fs X-Rspamd-Queue-Id: 4BqwcL3Xdxz4Csq X-Spamd-Bar: --- X-Spamd-Result: default: False [-3.34 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.99)[-0.991]; R_DKIM_ALLOW(-0.20)[delphix.com:s=google]; RCVD_TLS_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36:c]; NEURAL_HAM_LONG(-1.02)[-1.021]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[zfs-devel@freebsd.org]; TO_DN_SOME(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[delphix.com:+]; DMARC_POLICY_ALLOW(-0.50)[delphix.com,quarantine]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::635:from]; NEURAL_HAM_SHORT(-0.63)[-0.632]; FORGED_SENDER(0.30)[mahrens@delphix.com,matthew.ahrens@delphix.com]; MIME_TRACE(0.00)[0:+,1:+,2:~]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[mahrens@delphix.com,matthew.ahrens@delphix.com]; MAILMAN_DEST(0.00)[zfs-devel]; RCVD_COUNT_TWO(0.00)[2] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.33 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 14 Sep 2020 18:58:27 -0000 The next OpenZFS Leadership meeting will be held tomorrow, September 15, 1pm-2pm Pacific time. We don't have many topics on the agenda for tomorrow's meeting, so let me know if you'd like to add anything. FYI, it's 3 short weeks until the 2020 OpenZFS Developer Summit ! Everyone is welcome to attend and participate, and we will try to keep the meeting on agenda and on time. The meetings will be held online via Zoom, and recorded and posted to the website and YouTube after the meeting. For more information and details on how to attend, as well as notes and video from the previous meeting, please see the agenda document: https://docs.google.com/document/d/1w2jv2XVYFmBVvG1EGf-9A5HBVsjAYoLIFZAnWHhV-BM/edit --matt From owner-zfs-devel@freebsd.org Wed Sep 16 20:06:05 2020 Return-Path: Delivered-To: zfs-devel@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 0988D3EFE43 for ; Wed, 16 Sep 2020 20:06:05 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: from mail-ed1-x543.google.com (mail-ed1-x543.google.com [IPv6:2a00:1450:4864:20::543]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4BsB1R157Mz4CsP for ; Wed, 16 Sep 2020 20:06:02 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: by mail-ed1-x543.google.com with SMTP id k14so7979265edo.1 for ; Wed, 16 Sep 2020 13:06:02 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=/gYtUxcxZ/0NCnwCque2i15I9j83o0nh5OS3r26csLI=; b=NwrtVzihYSmrOaUfczZzW8wEKCM8S2t8ZUaIrngKqzxpuzFzrCBOP2vjnS3hLjJbue 3+FRtkM6APXOqRkTuY11hG/m52R8uVm5UA0ErFVwr6tsN+Ae38+xicjNhh9Hg8WQnCav nhGXGi0j0otxO/XnbywZMrgGDy2x0VfEhdBvfW89O958Hm7JfnPK09i428McrPB8l+YG N7vbBklsMvYxlaiFD+w70jTt0fmWVOVklAP4n1bjyqoattMFbSMHvGp+kV8Im1IXaVR+ TuGRIbNf9qEzg7/j2yPjGRfJ+DYxdnGMAHVPwb7RJJUwV1CaHbL4s4L+dqrKhLeAji+V R2Cw== X-Gm-Message-State: AOAM530zX8ooFkT+X6UzJBwt5ScYoGVARwK7SMBTXnrfaMVlpRMxP3ey QfRNqNgqkmT00PNq+VfW+jaAmyMXtMO2UxMATaZKlw== X-Google-Smtp-Source: ABdhPJzpzhLqyHzymgpZObKw8rXfMTGSV64oNtoCTv/guDMKxWIxbc/4oPL8et29lKKW/0DpDS6v5tiGBJpmbWVV9No= X-Received: by 2002:a50:e685:: with SMTP id z5mr20586551edm.259.1600286761347; Wed, 16 Sep 2020 13:06:01 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Matthew Ahrens Date: Wed, 16 Sep 2020 13:05:50 -0700 Message-ID: Subject: Re: September OpenZFS Leadership Meeting To: developer , illumos-zfs , zfs-devel@freebsd.org, freebsd-fs X-Rspamd-Queue-Id: 4BsB1R157Mz4CsP X-Spamd-Bar: -- X-Spamd-Result: default: False [-2.15 / 15.00]; RCVD_TLS_ALL(0.00)[]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[delphix.com:s=google]; NEURAL_HAM_MEDIUM(-0.94)[-0.943]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[zfs-devel@freebsd.org]; NEURAL_HAM_LONG(-0.98)[-0.980]; URI_COUNT_ODD(1.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[delphix.com:+]; DMARC_POLICY_ALLOW(-0.50)[delphix.com,quarantine]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::543:from]; NEURAL_HAM_SHORT(-0.53)[-0.528]; FORGED_SENDER(0.30)[mahrens@delphix.com,matthew.ahrens@delphix.com]; MIME_TRACE(0.00)[0:+,1:+,2:~]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[mahrens@delphix.com,matthew.ahrens@delphix.com]; MAILMAN_DEST(0.00)[zfs-devel]; RCVD_COUNT_TWO(0.00)[2] Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.33 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 Sep 2020 20:06:05 -0000 The video recording has been posted: https://www.youtube.com/watch?v=3DfVcNnFHDwHY meeting notes: - OpenZFS hackathon (Matt) - Spreadsheet with Hackathon Ideas/Breakout Sessions: https://docs.google.com/spreadsheets/d/1qH-qST3uSYVh7fzWHWIB1eM1y5rAY= CQs6tRDjOhVYx0/edit#gid=3D0 - Feel free to put your ideas in the doc even if you are just working by yourself! - Since it will be held virtually, the hackathon will be slightly more structured this year. Each idea proposer will be given a couple of minutes in the beginning of the day to pitch their idea. Then a Zoom Room (and/or a Slack channel) will be created for each team project. - Per-pool ARC STATS (Richard Elling) - Unlike the L2ARC, the ARC statistics are not broken down to per-pool statistics. It would be nice to be able to break them down into that fashion when dealing with issues. - For the short-term we could just add a few trace points but in the long term we may want something more integrated to ZFS. - This is still in the idea phase and the most straightforward solution so far seems to be adding more specific arcstats under the SPL kstat = code (the upside of this approach being that there is no need for a userla= nd change whatsoever). - There were also a couple of tangential issues brought up to the discussion: - The idea to potentially attempt to break this info even further to a per-dataset level - Existing aggsum implementation being slightly more heavy-weight and not as scalable as existing percpu counter implementations in FreeBSD and Linux. We could either learn from these platform-specific solutions and improve the aggsum code=E2=80=99s design, or just get rid of its existing code and make it a thin wrapper on top of the existing platform-specific solutions. - ZED all-syslog.sh (Don Brady) - Illumos=E2=80=99 FMA logs were retained after a crash but Linux=E2=80= =99s zpool events are not. There exists all-syslog.sh in ZED but its output is t= erse and misses any useful information. - Don has prototyped a change in all-syslogs.sh to provide more useful information for some events (e.g. the vdev, and offset fields for checksum error events). - Even though changing that code may result in the breakage of some existing stuff, all agreed that this is a step into the right directi= on. - Don expects to open a PR for this soon. - OpenZFS 2.0RC (Brian) - The first 2.0 release candidate was created 2 weeks ago and a new one is expected to open this week. - People who want to try it out are encouraged to do so while the code is soaking - There is no strict schedule for these RC releases but for now we expect to see one every two weeks approximately. - As an overall status, the project is still on track for the official 2.0 release before the end of the year. On Mon, Sep 14, 2020 at 11:58 AM Matthew Ahrens wrote= : > The next OpenZFS Leadership meeting will be held tomorrow, September 15, > 1pm-2pm Pacific time. We don't have many topics on the agenda for > tomorrow's meeting, so let me know if you'd like to add anything. > > FYI, it's 3 short weeks until the 2020 OpenZFS Developer Summit > ! > > Everyone is welcome to attend and participate, and we will try to keep th= e > meeting on agenda and on time. The meetings will be held online via Zoom= , > and recorded and posted to the website and YouTube after the meeting. > > For more information and details on how to attend, as well as notes and > video from the previous meeting, please see the agenda document: > > > https://docs.google.com/document/d/1w2jv2XVYFmBVvG1EGf-9A5HBVsjAYoLIFZAnW= HhV-BM/edit > > --matt > From owner-zfs-devel@freebsd.org Tue Sep 22 14:29:42 2020 Return-Path: Delivered-To: zfs-devel@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id E8D5A3EAC98 for ; Tue, 22 Sep 2020 14:29:42 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4BwkGZ5wbTz45NB for ; Tue, 22 Sep 2020 14:29:42 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id B00DB20CEA for ; Tue, 22 Sep 2020 14:29:42 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 08METgki084071 for ; Tue, 22 Sep 2020 14:29:42 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 08METgnO084070 for zfs-devel@FreeBSD.org; Tue, 22 Sep 2020 14:29:42 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: zfs-devel@FreeBSD.org Subject: [Bug 249115] The zfs command in the installer fails to log a set command Date: Tue, 22 Sep 2020 14:29:42 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: bin X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: swills@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: zfs-devel@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: assigned_to Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 22 Sep 2020 14:29:43 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D249115 Steve Wills changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|sysinstall@FreeBSD.org |zfs-devel@FreeBSD.org --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-zfs-devel@freebsd.org Tue Sep 22 14:56:59 2020 Return-Path: Delivered-To: zfs-devel@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id A66F23EB859 for ; Tue, 22 Sep 2020 14:56:59 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Bwkt33zmTz473f for ; Tue, 22 Sep 2020 14:56:59 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 6823C2121E for ; Tue, 22 Sep 2020 14:56:59 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 08MEux54014818 for ; Tue, 22 Sep 2020 14:56:59 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 08MEuxg1014817 for zfs-devel@FreeBSD.org; Tue, 22 Sep 2020 14:56:59 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: zfs-devel@FreeBSD.org Subject: [Bug 249115] The zfs command in the installer fails to log a set command Date: Tue, 22 Sep 2020 14:56:59 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: bin X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: avg@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: zfs-devel@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 22 Sep 2020 14:56:59 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D249115 --- Comment #3 from Andriy Gapon --- (In reply to Steve Wills from comment #2) zfs-devel@ is effectively dead, please use fs@. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-zfs-devel@freebsd.org Mon Oct 12 22:04:57 2020 Return-Path: Delivered-To: zfs-devel@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 224B642984C for ; Mon, 12 Oct 2020 22:04:57 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: from mail-ed1-x533.google.com (mail-ed1-x533.google.com [IPv6:2a00:1450:4864:20::533]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4C9CQc2WTrz3V4y for ; Mon, 12 Oct 2020 22:04:56 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: by mail-ed1-x533.google.com with SMTP id l24so18701949edj.8 for ; Mon, 12 Oct 2020 15:04:56 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=ISmQGvmifII9TYarXisVol32KRN8IqGmKBmQc/qlG9w=; b=PIKrykRN30a8jIJu7eV/Nl1+m1bb7RQY4vQv32oNom4wFZww0hjByOr/GCt1agfB7B /nroRzT3esqxU1SQPcfVOcc7pRvaFX/jJmpAOlzEAL/M2DN6jhyL/y9yKFc+B5irsP3C Ax7L+JDPm7PaG7hYsofmOFm54eaqNBOYtoY9ju/4Rt3K3o+aCw/LZREPqTO4z9cpl6iX MP5unVNVcAvEp3umE540MF0ZpDxR4f+0dxPeI8o++e22UYcdHoTBf83gg8gZCdBLLlO8 9oRV1rwuLFeIgt/SmhIy7SsuH0isRPIJyQ7HVe3W1R2nVwwRIa00z/j32TZLjxH9TnuF +oBQ== X-Gm-Message-State: AOAM532MyEyXVt9+rVL/A+RUwTS+4A8/Yg9hFmwO3jRZRzYka62r1fXD nUzY1Foqaw757xHlicmKpZXwJQy3QwVVIyzZSo9YNdg1hpw= X-Google-Smtp-Source: ABdhPJxvlYvRBcRkIv3acw3imX/yS1xbfuCSAOO0EdcaiHtDE9RDOwDOxp6E8JN7gDnEncW3EXRyvDyOeVoV3R0iHLk= X-Received: by 2002:aa7:c1d3:: with SMTP id d19mr17212571edp.293.1602540294281; Mon, 12 Oct 2020 15:04:54 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Matthew Ahrens Date: Mon, 12 Oct 2020 15:04:43 -0700 Message-ID: Subject: Re: September OpenZFS Leadership Meeting To: developer , illumos-zfs , zfs-devel@freebsd.org, freebsd-fs X-Rspamd-Queue-Id: 4C9CQc2WTrz3V4y X-Spamd-Bar: -- X-Spamd-Result: default: False [-2.73 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.96)[-0.955]; R_DKIM_ALLOW(-0.20)[delphix.com:s=google]; RCVD_TLS_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36:c]; NEURAL_HAM_LONG(-0.99)[-0.989]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[zfs-devel@freebsd.org]; TO_DN_SOME(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[delphix.com:+]; DMARC_POLICY_ALLOW(-0.50)[delphix.com,quarantine]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::533:from]; NEURAL_HAM_SHORT(-0.09)[-0.089]; FORGED_SENDER(0.30)[mahrens@delphix.com,matthew.ahrens@delphix.com]; MIME_TRACE(0.00)[0:+,1:+,2:~]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[mahrens@delphix.com,matthew.ahrens@delphix.com]; MAILMAN_DEST(0.00)[zfs-devel]; RCVD_COUNT_TWO(0.00)[2] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.33 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Oct 2020 22:04:57 -0000 The next OpenZFS Leadership meeting will be held tomorrow, October 12, 9am-10am Pacific time. We don't have many topics on the agenda for tomorrow's meeting, so let me know if you'd like to add anything. FYI, we held the 2020 OpenZFS Developer Summit last week, and the videos for it are now posted. If you have questions about any of the material presented at the conference, tomorrow's meeting would be a good time to discuss them. Everyone is welcome to attend and participate, and we will try to keep the meeting on agenda and on time. The meetings will be held online via Zoom, and recorded and posted to the website and YouTube after the meeting. For more information and details on how to attend, as well as notes and video from the previous meeting, please see the agenda document: https://docs.google.com/document/d/1w2jv2XVYFmBVvG1EGf-9A5HBVsjAYoLIFZAnWHhV-BM/edit --matt From owner-zfs-devel@freebsd.org Thu Oct 15 15:55:12 2020 Return-Path: Delivered-To: zfs-devel@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 3D6844418E4 for ; Thu, 15 Oct 2020 15:55:12 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: from mail-ej1-x62d.google.com (mail-ej1-x62d.google.com [IPv6:2a00:1450:4864:20::62d]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4CBv4b2SSNz4dwt for ; Thu, 15 Oct 2020 15:55:11 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: by mail-ej1-x62d.google.com with SMTP id p15so4342218ejm.7 for ; Thu, 15 Oct 2020 08:55:11 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=Pq2cqsVJgg0E7Z9HPaO1Wx2A7Y3p8J2+YKGTc+yNUUU=; b=krCUEHhJbMc7/9Rl0ssrzHu8sK3i/hvVvg3u3DOBSRM/r2gNGaVAeKtB5vJ6hXzJNT l/erH7qNdt1pG5aMAPaogN/QWAiaSm7abOWo0aXke9Pgt5F3FYQvXLkTRGOPEeV8+1rS upTsX6/p5GhboRlYxJJBh03uX157Y/yNnsyLNJMiAzYQC7y/2c3BNINz6jaJgoZ2Eibr NgUwfSO+2ysxizSqtLQo5JBJAj9z6x2L/B4wA5zg8rHjcwIm1c8H6hE6pTB02Y7DvTRB yrCLgClf5VS10n2JtPhWl+4FjaP7Q2TB/80D1OtkNQo8HSt9LgNWFFm2UGBM7+dGEsj9 WD8Q== X-Gm-Message-State: AOAM530shyxOEBMBRn6I4lo5hcprr1CXBWcjdBOkGBvB7Pg/lzM16H9I E5gAkBYik83idL3Rm7xosu+jSHWR79U4/JWyOa++snHOjc4Kiw== X-Google-Smtp-Source: ABdhPJze2QA1V8aO848GyyZLMx4gexhj4W9HWLUZO5sN+C5I8dAjh6ixXbGqIInBZ6wtblvQfM8YomvN4kCGzrVbjok= X-Received: by 2002:a17:906:38d8:: with SMTP id r24mr2658175ejd.32.1602777309233; Thu, 15 Oct 2020 08:55:09 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Matthew Ahrens Date: Thu, 15 Oct 2020 08:54:58 -0700 Message-ID: Subject: October OpenZFS Leadership Meeting To: developer , illumos-zfs , zfs-devel@freebsd.org, freebsd-fs X-Rspamd-Queue-Id: 4CBv4b2SSNz4dwt X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.08 / 15.00]; RCVD_TLS_ALL(0.00)[]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[delphix.com:s=google]; NEURAL_HAM_MEDIUM(-1.05)[-1.052]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[zfs-devel@freebsd.org]; NEURAL_HAM_LONG(-1.02)[-1.018]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[delphix.com:+]; DMARC_POLICY_ALLOW(-0.50)[delphix.com,quarantine]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::62d:from]; NEURAL_HAM_SHORT(-1.31)[-1.306]; FORGED_SENDER(0.30)[mahrens@delphix.com,matthew.ahrens@delphix.com]; MIME_TRACE(0.00)[0:+,1:+,2:~]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[mahrens@delphix.com,matthew.ahrens@delphix.com]; MAILMAN_DEST(0.00)[zfs-devel]; RCVD_COUNT_TWO(0.00)[2] Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.33 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 15 Oct 2020 15:55:12 -0000 Thanks to everyone who participated. The video from this week's meeting is now posted: https://youtu.be/KzdgeBFmLl8 Meeting notes: - Determine AI=E2=80=99s from conference discussions - Code reviews for Add "features" property for zpool feature sets. =C2= =B7 PR #10980 - PR needs reviewers of 2 kinds: - [1] Reviewers of the actual code - [2] Reviewers for the feature sets outlined - Draid code reviews (Brian) - There is a PR out and we are looking for reviewers. - Brian is expected to squash his commits today or tomorrow to make reviews easier - It is a lot of code so the more reviewers the better - if you are interested in how dRaid works, how it interacts with the rest of Z= FS=E2=80=99s features, or just want to read the documentation and see the trade= offs involved in its usage you are more than welcome to take a look at the PR. - Brian is also welcoming people to pull the code and test it - Block reference tracking design doc (Matt/Pawel) - There was some work on writing up an initial design doc in the conference - The goal here is to complete the doc and share it in one of the future meetings. - Terminology review, make sure we use the terms =E2=80=9CHealing resil= ver=E2=80=9D in the docs, and maybe even zpool status etc (Allan) - After Mark Maybee=E2=80=99s sequential rebuild talk during the sum= mit, it would be nice to have distinct terminology for the two resilver options that we provide. - This work entails ensuring that there is no ambiguous terminology used in docs, man pages, and command output. - ZDB --cp upstreaming (Allan) - Initial PR is out - the span of this work does not include the zdb long-opts refactoring - Current status: - Need to add some test cases for the feature and add its option to the help message - Update PR with misc fixes like the dump_all option bug - Potential future work for this would be to use the dnode=E2=80=99s blocksize (instead of the standard 1MB used now) - Forced Export Feature (Allan) - Feature was rebased to latest master with some new improvements - PR should be out soon - Of particular interest may be the code introduced for dataset unmounting in Linux to work around some specific VFS cases - zfs list improvements (Allan) - There is some effort in the works of making the usage of zfs-list faster when specific dataset info/properties are requested as some of them are bundled with the information queried at the initial stages of dataset lookup. On Mon, Oct 12, 2020 at 3:04 PM Matthew Ahrens wrote: > The next OpenZFS Leadership meeting will be held tomorrow, October 12, > 9am-10am Pacific time. We don't have many topics on the agenda for > tomorrow's meeting, so let me know if you'd like to add anything. > > FYI, we held the 2020 OpenZFS Developer Summit > last week, and > the videos for it are now posted. If you have questions about any of the > material presented at the conference, tomorrow's meeting would be a good > time to discuss them. > > Everyone is welcome to attend and participate, and we will try to keep th= e > meeting on agenda and on time. The meetings will be held online via Zoom= , > and recorded and posted to the website and YouTube after the meeting. > > For more information and details on how to attend, as well as notes and > video from the previous meeting, please see the agenda document: > > > https://docs.google.com/document/d/1w2jv2XVYFmBVvG1EGf-9A5HBVsjAYoLIFZAnW= HhV-BM/edit > > --matt > From owner-zfs-devel@freebsd.org Tue Oct 20 08:27:47 2020 Return-Path: Delivered-To: zfs-devel@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id A02E5429A2A for ; Tue, 20 Oct 2020 08:27:47 +0000 (UTC) (envelope-from commerciale@livejob.it) Received: from livejob.it (livejob.it [185.221.173.42]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4CFmw24dzFz403W for ; Tue, 20 Oct 2020 08:27:46 +0000 (UTC) (envelope-from commerciale@livejob.it) Received: from [93.37.216.147] (port=39051 helo=SendBlaster) by bonolaoffice.serverkeliweb.it with esmtpa (Exim 4.93) (envelope-from ) id 1kUmze-0004do-F0 for zfs-devel@freebsd.org; Tue, 20 Oct 2020 10:27:42 +0200 MIME-Version: 1.0 From: "TermoScanner Anti-Covid" Reply-To: commerciale@livejob.it To: zfs-devel@freebsd.org Subject: TermoScanner Anti-Covid in pronta consegna. Prezzi dimezzati e modelli a partire da soli Euro 499, 00. Non Abbassiamo la guardia!!! X-Mailer: Smart_Send_4_4_2 Date: Tue, 20 Oct 2020 10:26:55 +0200 Message-ID: <616047793683227725793@SEND-SRV> X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - bonolaoffice.serverkeliweb.it X-AntiAbuse: Original Domain - freebsd.org X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - livejob.it X-Get-Message-Sender-Via: bonolaoffice.serverkeliweb.it: authenticated_id: commerciale@livejob.it X-Authenticated-Sender: bonolaoffice.serverkeliweb.it: commerciale@livejob.it X-Source: X-Source-Args: X-Source-Dir: X-Rspamd-Queue-Id: 4CFmw24dzFz403W X-Spamd-Bar: + X-Spamd-Result: default: False [1.29 / 15.00]; HAS_REPLYTO(0.00)[commerciale@livejob.it]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_SPF_ALLOW(-0.20)[+a]; HAS_X_SOURCE(0.00)[]; REPLYTO_ADDR_EQ_FROM(0.00)[]; TO_DN_NONE(0.00)[]; DKIM_TRACE(0.00)[livejob.it:+]; DMARC_POLICY_ALLOW(-0.50)[livejob.it,reject]; SUBJECT_ENDS_EXCLAIM(0.00)[]; HAS_X_ANTIABUSE(0.00)[]; RECEIVED_SPAMHAUS_PBL(0.00)[93.37.216.147:received]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; ASN(0.00)[asn:202675, ipnet:185.221.173.0/24, country:IT]; RCVD_TLS_LAST(0.00)[]; HAS_X_AS(0.00)[commerciale@livejob.it]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.60)[-0.597]; R_DKIM_ALLOW(-0.20)[livejob.it:s=default]; FROM_HAS_DN(0.00)[]; EXT_CSS(1.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; HTML_SHORT_LINK_IMG_1(2.00)[]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; NEURAL_SPAM_SHORT(0.39)[0.392]; NEURAL_HAM_LONG(-0.99)[-0.991]; HAS_LIST_UNSUB(-0.01)[]; RCPT_COUNT_ONE(0.00)[1]; HAS_X_GMSV(0.00)[commerciale@livejob.it]; MID_RHS_NOT_FQDN(0.50)[]; RCVD_COUNT_TWO(0.00)[2]; MAILMAN_DEST(0.00)[zfs-devel] Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.33 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Oct 2020 08:27:47 -0000 Il Tuo Partner Tecnologico TERMOSCANNER ANTI COVID-19=20 Misurazione della temperatura corporean in tempo reale!=20 nostri prodotti sono marchiati CE e coperti da regolare Garanzia di Legge. Garantiamo il supporto tecnico gratuito da remoto, per la messa in funzione degli strumenti, Termoscanner con case in alluminio pressofuso e display 8'' munito di inte= lligenza artificiale atta alla rilevazione automatica e a distanza della te= mperatura corporea con riconoscimento facciale. Oltre alla temperatura rile= va il corretto o mancato utilizzo della mascherina permettendo il riconosci= mento dei volti per il controllo degli accessi automatizzati con tornelli o= porte automatiche. Il software fornito con il dispositivo, se voluto, perm= ette di creare un=E2=80=99anagrafica delle persone autorizzate all=E2=80=99= ingresso nella struttura, consentendo l'esportazione delle letture o l=E2= =80=99importazione di anagrafiche da fonti esterne ed evitando onerosi cari= camenti manuali. E' possibile salvare tutti gli eventi o non registrare alcun dato a seconda= delle proprie policy sulla privacy e nel rispetto del GDPR applicato. Dispositivo standalone per la prevenzione epidemica che consente la rilevaz= ione della temperatura corporea con avviso sonoro e luminoso al superamento= di una soglia di temperatura. Alimentatore esterno in dotazione. Connettor= e USB per collegamento con batteria esterna opzionale. Possibilit=C3=A0 di = montaggio a muro o su piedistallo treppiedi.=20 From owner-zfs-devel@freebsd.org Sat Oct 24 17:23:24 2020 Return-Path: Delivered-To: zfs-devel@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 5950544F78A for ; Sat, 24 Oct 2020 17:23:24 +0000 (UTC) (envelope-from commerciale@bonolaoffice.com) Received: from vps.bonolaoffice.com (vps.bonolaoffice.com [185.221.173.235]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4CJScC1j4Tz3bRd for ; Sat, 24 Oct 2020 17:23:22 +0000 (UTC) (envelope-from commerciale@bonolaoffice.com) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=bonolaoffice.com; s=default; h=List-Unsubscribe:Message-ID:Date: Content-Type:Subject:To:Reply-To:From:MIME-Version:Sender:Cc: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To: References:List-Id:List-Help:List-Subscribe:List-Post:List-Owner:List-Archive ; bh=flYR+UwP+6BcTbY5xHu/qjhKihklPbFL1L8U8gr7jYk=; b=mziQqTw8Dm/aQFtOtf+YKUpf Ik/TnVPFvShWOdKLDsBo9XxAitL7QjZUZaZ+8LgliJrY3feDJpFUdbcaDHj/o95wZi8OpETIOvaVi ZvTRUjU4JNzFVK+1vY8LC8GaJRRyUZI/HsqIGI2CQGD1HraDydJYQ1u08ZzhnLVxxYXYK9F1V+SeF fvKtMHQwHDlIlIgjnXvuEYPto0HLo0cnLuVRYl3ibrrq/yUVVXIG4REycLVuFBjgf02z6LAf/JeMU dmJqEHnJrQFs00Bu083XPNJYE8dWutzDNsiP7zfD55jRodsBYn15muVYU1Yy9RdYcG5XUDHy12Ajq FXgkLZj5oQ==; Received: from 93-50-165-11.ip153.fastwebnet.it ([93.50.165.11]:41441 helo=SendBlaster) by bonolaoffice.serverkeliweb.it with esmtpa (Exim 4.93) (envelope-from ) id 1kWNGB-0006uQ-0j for zfs-devel@freebsd.org; Sat, 24 Oct 2020 19:23:19 +0200 MIME-Version: 1.0 From: "Business Center Milano" Reply-To: commerciale@bonolaoffice.com To: zfs-devel@freebsd.org Subject: Uffici temporanei a Milano. Provali per un mese con la formula soddisfatti o rimborsati al 100%. Non esitare, i posti liberi non sono infiniti. X-Mailer: Smart_Send_4_4_2 Date: Sat, 24 Oct 2020 19:10:58 +0200 Message-ID: <558441522277612527171@PP04> X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - bonolaoffice.serverkeliweb.it X-AntiAbuse: Original Domain - freebsd.org X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - bonolaoffice.com X-Get-Message-Sender-Via: bonolaoffice.serverkeliweb.it: authenticated_id: commerciale@bonolaoffice.com X-Authenticated-Sender: bonolaoffice.serverkeliweb.it: commerciale@bonolaoffice.com X-Source: X-Source-Args: X-Source-Dir: X-Rspamd-Queue-Id: 4CJScC1j4Tz3bRd X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=bonolaoffice.com header.s=default header.b=mziQqTw8; dmarc=pass (policy=none) header.from=bonolaoffice.com; spf=pass (mx1.freebsd.org: domain of commerciale@bonolaoffice.com designates 185.221.173.235 as permitted sender) smtp.mailfrom=commerciale@bonolaoffice.com X-Spamd-Result: default: False [-2.11 / 15.00]; HAS_REPLYTO(0.00)[commerciale@bonolaoffice.com]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:185.221.173.235]; ZERO_FONT(0.10)[1]; HAS_X_SOURCE(0.00)[]; REPLYTO_ADDR_EQ_FROM(0.00)[]; TO_DN_NONE(0.00)[]; DKIM_TRACE(0.00)[bonolaoffice.com:+]; DMARC_POLICY_ALLOW(-0.50)[bonolaoffice.com,none]; NEURAL_HAM_SHORT(-0.24)[-0.244]; HAS_X_ANTIABUSE(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:202675, ipnet:185.221.173.0/24, country:IT]; R_PARTS_DIFFER(0.52)[75.8%]; HAS_X_AS(0.00)[commerciale@bonolaoffice.com]; RWL_MAILSPIKE_NEUTRAL(0.00)[185.221.173.235:from]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[bonolaoffice.com:s=default]; NEURAL_HAM_MEDIUM(-1.05)[-1.050]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-0.98)[-0.977]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; HAS_LIST_UNSUB(-0.01)[]; RCPT_COUNT_ONE(0.00)[1]; MANY_INVISIBLE_PARTS(0.05)[1]; HAS_X_GMSV(0.00)[commerciale@bonolaoffice.com]; MID_RHS_NOT_FQDN(0.50)[]; RCVD_COUNT_TWO(0.00)[2]; MAILMAN_DEST(0.00)[zfs-devel] Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.33 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 24 Oct 2020 17:23:24 -0000 BONOLAOFFICE LO SPAZIO CHE TI SERVE QUANDO TI SERVE. Anche se fai fatica a crederci, ti confermiamo che la prova e' totalmente GRATUITA. Vivi l'esperienza di lavorare in un Business Center con tutti i servizi co= mpresi senza spendere un euro per 15 giorni. Nessun vincolo di acquisto di = servizi accessori ne di sottoscrizione di contratti. Siamo cosi=E2=80=99 si= curi dei servizi proposti che siamo disposti a investire per farteli vivere= nel quotidiano.=20 Non lasciarti sfuggire l'occasione!!! UFFICI E UTENZE COMPLETAMENTE GRATUITI!* Fai ripartire la tua azienda grazie a BonolaOffice.com Perche' non approfittare dell'attuale emergenza sanitaria per riorganizzare la tua azienda=3F Passa da un ufficio classico ad uno dinamico!=09 La nostra struttura riorganizzata con uffici e spazi ad uso esclusivo, garantisce oltre alla privacy, la sicurezza di poter operare in spazi dedi= cati senza la necessita' di dover interagire con il personale di altre aziende. In questo periodo di emergenza, il giusto distanziamento e le giuste precauzioni sono diventate indispensabili per poter ripartire con la massima sicurezza! Gli uffici BonolaOffice.com sono ideali sia per aziende che per liberi professionisti, le diverse conformazioni e metrature semindipendenti con=20 soluzioni da 20 a 600 mq, sono in grado di soddisfare ogni tipo di esigenz= a. =20 From owner-zfs-devel@freebsd.org Wed Nov 11 07:39:39 2020 Return-Path: Delivered-To: zfs-devel@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 31DD82D7C54 for ; Wed, 11 Nov 2020 07:39:39 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4CWGpM0sNLz4Rbn for ; Wed, 11 Nov 2020 07:39:39 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 106C91138B for ; Wed, 11 Nov 2020 07:39:39 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 0AB7dcEG091101 for ; Wed, 11 Nov 2020 07:39:38 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 0AB7dcBC091100 for zfs-devel@FreeBSD.org; Wed, 11 Nov 2020 07:39:38 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: zfs-devel@FreeBSD.org Subject: [Bug 251035] [zfs] Allow 64 bit ZFS to support 32 bit ioctls (Wine) Date: Wed, 11 Nov 2020 07:39:39 +0000 X-Bugzilla-Reason: AssignedTo CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 12.2-RELEASE X-Bugzilla-Keywords: needs-qa X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: koobs@FreeBSD.org X-Bugzilla-Status: Open X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: zfs-devel@FreeBSD.org X-Bugzilla-Flags: mfc-stable12? mfc-stable11? X-Bugzilla-Changed-Fields: assigned_to flagtypes.name short_desc keywords bug_status cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 Nov 2020 07:39:39 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D251035 Kubilay Kocak changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|bugs@FreeBSD.org |zfs-devel@FreeBSD.org Flags| |mfc-stable12?, | |mfc-stable11? Summary|[zfs][patch] allow 64 bit |[zfs] Allow 64 bit ZFS to |zfs to support 32 bit |support 32 bit ioctls |ioctls (Wine) |(Wine) Keywords| |needs-qa Status|New |Open CC| |zfs-devel@FreeBSD.org --=20 You are receiving this mail because: You are the assignee for the bug. You are on the CC list for the bug.= From owner-zfs-devel@freebsd.org Wed Nov 11 08:09:44 2020 Return-Path: Delivered-To: zfs-devel@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 793302E0B83 for ; Wed, 11 Nov 2020 08:09:44 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4CWHT42RcCz4Svh for ; Wed, 11 Nov 2020 08:09:44 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 42F87115C0 for ; Wed, 11 Nov 2020 08:09:44 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 0AB89iOR006980 for ; Wed, 11 Nov 2020 08:09:44 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 0AB89iVH006979 for zfs-devel@FreeBSD.org; Wed, 11 Nov 2020 08:09:44 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: zfs-devel@FreeBSD.org Subject: [Bug 251035] [zfs] Allow 64 bit ZFS to support 32 bit ioctls (Wine) Date: Wed, 11 Nov 2020 08:09:44 +0000 X-Bugzilla-Reason: CC AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 12.2-RELEASE X-Bugzilla-Keywords: needs-qa X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: avg@FreeBSD.org X-Bugzilla-Status: Open X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: fs@FreeBSD.org X-Bugzilla-Flags: mfc-stable12? mfc-stable11? X-Bugzilla-Changed-Fields: assigned_to cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 Nov 2020 08:09:44 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D251035 Andriy Gapon changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|zfs-devel@FreeBSD.org |fs@FreeBSD.org CC|zfs-devel@FreeBSD.org | --- Comment #1 from Andriy Gapon --- zfs-devel@ is disused. --=20 You are receiving this mail because: You are on the CC list for the bug. You are the assignee for the bug.= From owner-zfs-devel@freebsd.org Sat Nov 21 09:05:47 2020 Return-Path: Delivered-To: zfs-devel@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 82C3647F3AE for ; Sat, 21 Nov 2020 09:05:47 +0000 (UTC) (envelope-from commerciale@livejob.info) Received: from livejob.info (livejob.info [185.17.107.91]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4CdSF531SCz3nPC for ; Sat, 21 Nov 2020 09:05:45 +0000 (UTC) (envelope-from commerciale@livejob.info) Received: from [93.37.216.147] (port=36537 helo=SendBlaster) by bonolaoffice.serverkeliweb.it with esmtpa (Exim 4.93) (envelope-from ) id 1kgOpr-00009Y-P3 for zfs-devel@freebsd.org; Sat, 21 Nov 2020 10:05:35 +0100 MIME-Version: 1.0 From: "Smaltimento Toner Esausti" Reply-To: commerciale@livejob.info To: zfs-devel@freebsd.org Subject: Smaltisci i Toner esausti con un click. Approfitta del ritiro in sede A COSTO ZERO e mettiti a riparo delle sanzioni penali. X-Mailer: Smart_Send_4_4_2 Date: Sat, 21 Nov 2020 10:05:36 +0100 Message-ID: <161646875458435915886@WIN10-Client01> X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - bonolaoffice.serverkeliweb.it X-AntiAbuse: Original Domain - freebsd.org X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - livejob.info X-Get-Message-Sender-Via: bonolaoffice.serverkeliweb.it: authenticated_id: commerciale@livejob.info X-Authenticated-Sender: bonolaoffice.serverkeliweb.it: commerciale@livejob.info X-Source: X-Source-Args: X-Source-Dir: X-Rspamd-Queue-Id: 4CdSF531SCz3nPC X-Spamd-Bar: - X-Spamd-Result: default: False [-1.01 / 15.00]; HAS_REPLYTO(0.00)[commerciale@livejob.info]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:185.17.107.91]; HAS_X_SOURCE(0.00)[]; REPLYTO_ADDR_EQ_FROM(0.00)[]; TO_DN_NONE(0.00)[]; DKIM_TRACE(0.00)[livejob.info:+]; DMARC_POLICY_ALLOW(-0.50)[livejob.info,reject]; NEURAL_HAM_SHORT(-1.00)[-1.000]; HAS_X_ANTIABUSE(0.00)[]; RECEIVED_SPAMHAUS_PBL(0.00)[93.37.216.147:received]; RCVD_TLS_LAST(0.00)[]; RBL_DBL_DONT_QUERY_IPS(0.00)[185.17.107.91:from]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; HAS_X_AS(0.00)[commerciale@livejob.info]; ASN(0.00)[asn:202675, ipnet:185.17.107.0/24, country:IT]; ARC_NA(0.00)[]; SUBJECT_ENDS_SPACES(0.50)[]; R_DKIM_ALLOW(-0.20)[livejob.info:s=default]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; HTML_SHORT_LINK_IMG_1(2.00)[]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; NEURAL_HAM_LONG(-1.00)[-1.000]; HAS_LIST_UNSUB(-0.01)[]; RCPT_COUNT_ONE(0.00)[1]; SPAMHAUS_ZRD(0.00)[185.17.107.91:from:127.0.2.255]; HAS_X_GMSV(0.00)[commerciale@livejob.info]; MID_RHS_NOT_FQDN(0.50)[]; RCVD_COUNT_TWO(0.00)[2]; MAILMAN_DEST(0.00)[zfs-devel] Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.34 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 21 Nov 2020 09:05:47 -0000 l problema Aziendale del Trattamento dei Rifiuti di Stampa LA SOLUZIONe La Programs& Projects Consulting attraverso il = servizio per la Manutenzione dei Consumabili, quali =20 i TONER DI =20 STAMPA offre l'unico recupero che trasferisce la = Paternita' del rifiuto. LE SANZIONI=20 Le aziende che nonrispettano in tutto o in parte lalegge 152, sono suscettibili di SANZIONIPENALI oltre che di SANZIONIAMMINISTRATIVE DA = EURO 1.032,00 A EURO92.962,00 From owner-zfs-devel@freebsd.org Fri Dec 11 14:55:50 2020 Return-Path: Delivered-To: zfs-devel@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 3CCD74B1CD8 for ; Fri, 11 Dec 2020 14:55:50 +0000 (UTC) (envelope-from commerciale@bonolaoffice.com) Received: from vps.bonolaoffice.com (vps.bonolaoffice.com [185.221.173.235]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Csv3n0GMLz4Rm6 for ; Fri, 11 Dec 2020 14:55:48 +0000 (UTC) (envelope-from commerciale@bonolaoffice.com) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=bonolaoffice.com; s=default; h=List-Unsubscribe:Message-ID:Date: Content-Type:Subject:To:Reply-To:From:MIME-Version:Sender:Cc: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To: References:List-Id:List-Help:List-Subscribe:List-Post:List-Owner:List-Archive ; bh=P7wwUs86J2A3tD2LEpXfVYcBAz5Rvi40+pkTMAcQQ54=; b=EKwPYMQSctFxxo6K8IO8U50Z HAL3bgcMmSvuWYrwLB3fAxyCwbeV+OHrt0dNZWm3rCfX/Om94bNoVbApz69fBc80SVVJHaUB4aqCg erjuCWOIPeqkxmeNzGT3rf6IOK8M4Zyd+A5eZEhNDdSg/DopRho8a6FnC9SBv4iL1yowC3Y7gr1n4 3EWcdfUvNbedTatNjyoDRrAQB0yXiBnV0tdPm7bHl6WU4O96EoCGBIKaQZB1ND8pps0+y81dIGVCM re5vt4TZsamSd9C+t0AI/CyQ5dRwPVi0o/KwuccSHYJZzo9U6Rke2b84PDok+agPFVXOBiDPnBJO3 0aDo7DqKoA==; Received: from [84.33.91.25] (port=40872 helo=SendBlaster) by bonolaoffice.serverkeliweb.it with esmtpa (Exim 4.93) (envelope-from ) id 1knjpg-0003P3-S8 for zfs-devel@freebsd.org; Fri, 11 Dec 2020 15:55:45 +0100 MIME-Version: 1.0 From: "Business Center - Coworking" Reply-To: commerciale@bonolaoffice.com To: zfs-devel@freebsd.org Subject: Lavora in Smartworking e domicilia la tua azienda a meno di 2 euro al giorno presso il Ns Business Center. Non aspettare, riduci i costi fissi. X-Mailer: Smart_Send_4_4_2 Date: Fri, 11 Dec 2020 15:55:44 +0100 Message-ID: <600447759619217918121@WIN10-Client01> X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - bonolaoffice.serverkeliweb.it X-AntiAbuse: Original Domain - freebsd.org X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - bonolaoffice.com X-Get-Message-Sender-Via: bonolaoffice.serverkeliweb.it: authenticated_id: commerciale@bonolaoffice.com X-Authenticated-Sender: bonolaoffice.serverkeliweb.it: commerciale@bonolaoffice.com X-Source: X-Source-Args: X-Source-Dir: X-Rspamd-Queue-Id: 4Csv3n0GMLz4Rm6 X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; dkim=pass header.d=bonolaoffice.com header.s=default header.b=EKwPYMQS; dmarc=pass (policy=none) header.from=bonolaoffice.com; spf=pass (mx1.freebsd.org: domain of commerciale@bonolaoffice.com designates 185.221.173.235 as permitted sender) smtp.mailfrom=commerciale@bonolaoffice.com X-Spamd-Result: default: False [-0.83 / 15.00]; HAS_REPLYTO(0.00)[commerciale@bonolaoffice.com]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:185.221.173.235]; ZERO_FONT(0.10)[1]; HAS_X_SOURCE(0.00)[]; REPLYTO_ADDR_EQ_FROM(0.00)[]; TO_DN_NONE(0.00)[]; DKIM_TRACE(0.00)[bonolaoffice.com:+]; DMARC_POLICY_ALLOW(-0.50)[bonolaoffice.com,none]; NEURAL_HAM_SHORT(-1.00)[-0.995]; HAS_X_ANTIABUSE(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; RBL_DBL_DONT_QUERY_IPS(0.00)[185.221.173.235:from]; ASN(0.00)[asn:202675, ipnet:185.221.173.0/24, country:IT]; MIME_TRACE(0.00)[0:+,1:+,2:~]; R_PARTS_DIFFER(0.52)[76.2%]; HAS_X_AS(0.00)[commerciale@bonolaoffice.com]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[bonolaoffice.com:s=default]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; HAS_LIST_UNSUB(-0.01)[]; RCPT_COUNT_ONE(0.00)[1]; SPAMHAUS_ZRD(0.00)[185.221.173.235:from:127.0.2.255]; MANY_INVISIBLE_PARTS(0.05)[1]; NEURAL_SPAM_LONG(1.00)[1.000]; HAS_X_GMSV(0.00)[commerciale@bonolaoffice.com]; MID_RHS_NOT_FQDN(0.50)[]; RCVD_COUNT_TWO(0.00)[2]; MAILMAN_DEST(0.00)[zfs-devel] Content-Type: text/plain; charset="Windows-1252" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.34 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Dec 2020 14:55:50 -0000 BONOLAOFFICE LO SPAZIO CHE TI SERVE QUANDO TI SERVE. Anche se fai fatica a crederci, ti confermiamo che la prova e' totalmente GRATUITA. Vivi l'esperienza di lavorare in un Business Center con tutti i servizi co= mpresi senza spendere un euro per 15 giorni. Nessun vincolo di acquisto di = servizi accessori ne di sottoscrizione di contratti. Siamo cosi=E2=80=99 si= curi dei servizi proposti che siamo disposti a investire per farteli vivere= nel quotidiano.=20 Non lasciarti sfuggire l'occasione!!! UFFICI E UTENZE COMPLETAMENTE GRATUITI!* Fai ripartire la tua azienda grazie a BonolaOffice.com Perche' non approfittare dell'attuale emergenza sanitaria per riorganizzare la tua azienda=3F Passa da un ufficio classico ad uno dinamico!=09 La nostra struttura riorganizzata con uffici e spazi ad uso esclusivo, garantisce oltre alla privacy, la sicurezza di poter operare in spazi dedi= cati senza la necessita' di dover interagire con il personale di altre aziende. In questo periodo di emergenza, il giusto distanziamento e le giuste precauzioni sono diventate indispensabili per poter ripartire con la massima sicurezza! Gli uffici BonolaOffice.com sono ideali sia per aziende che per liberi professionisti, le diverse conformazioni e metrature semindipendenti con=20 soluzioni da 20 a 600 mq, sono in grado di soddisfare ogni tipo di esigenz= a. =20 From owner-zfs-devel@freebsd.org Tue Jan 26 16:14:47 2021 Return-Path: Delivered-To: zfs-devel@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 3C1524F4676 for ; Tue, 26 Jan 2021 16:14:47 +0000 (UTC) (envelope-from commerciale@bonolaoffice.biz) Received: from bonolaoffice.biz (bonolaoffice.biz [185.221.173.68]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4DQBdf0Bncz4chh for ; Tue, 26 Jan 2021 16:14:45 +0000 (UTC) (envelope-from commerciale@bonolaoffice.biz) Received: from [84.33.91.25] (port=38623 helo=WIN10Client01bonolaofficelocal) by bonolaoffice.serverkeliweb.it with esmtpa (Exim 4.93) (envelope-from ) id 1l4QzJ-0002EX-5q for zfs-devel@freebsd.org; Tue, 26 Jan 2021 17:14:41 +0100 MIME-Version: 1.0 From: "Business Center - Coworking" Reply-To: commerciale@bonolaoffice.biz To: zfs-devel@freebsd.org Subject: Riduci i costi fissi per la tua attivita' anche dell'80% e approfitta inoltre della formula soddisfatti o RIMBORSATI TOTALMENTE. X-Mailer: Smart_Send_4_4_2 Date: Tue, 26 Jan 2021 17:14:39 +0100 Message-ID: <455626679216048516406@WIN10-Client01> X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - bonolaoffice.serverkeliweb.it X-AntiAbuse: Original Domain - freebsd.org X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - bonolaoffice.biz X-Get-Message-Sender-Via: bonolaoffice.serverkeliweb.it: authenticated_id: commerciale@bonolaoffice.biz X-Authenticated-Sender: bonolaoffice.serverkeliweb.it: commerciale@bonolaoffice.biz X-Source: X-Source-Args: X-Source-Dir: X-Rspamd-Queue-Id: 4DQBdf0Bncz4chh X-Spamd-Bar: -- X-Spamd-Result: default: False [-2.63 / 15.00]; HAS_REPLYTO(0.00)[commerciale@bonolaoffice.biz]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_SPF_ALLOW(-0.20)[+a:c]; HAS_X_SOURCE(0.00)[]; REPLYTO_ADDR_EQ_FROM(0.00)[]; TO_DN_NONE(0.00)[]; ZERO_FONT(0.10)[1]; DKIM_TRACE(0.00)[bonolaoffice.biz:+]; DMARC_POLICY_ALLOW(-0.50)[bonolaoffice.biz,reject]; NEURAL_HAM_SHORT(-0.84)[-0.841]; HAS_X_ANTIABUSE(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:202675, ipnet:185.221.173.0/24, country:IT]; R_PARTS_DIFFER(0.57)[78.4%]; HAS_X_AS(0.00)[commerciale@bonolaoffice.biz]; ARC_NA(0.00)[]; RBL_DBL_DONT_QUERY_IPS(0.00)[185.221.173.68:from]; R_DKIM_ALLOW(-0.20)[bonolaoffice.biz:s=default]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; HAS_LIST_UNSUB(-0.01)[]; RCPT_COUNT_ONE(0.00)[1]; SPAMHAUS_ZRD(0.00)[185.221.173.68:from:127.0.2.255]; MANY_INVISIBLE_PARTS(0.05)[1]; HAS_X_GMSV(0.00)[commerciale@bonolaoffice.biz]; MID_RHS_NOT_FQDN(0.50)[]; RCVD_COUNT_TWO(0.00)[2]; MAILMAN_DEST(0.00)[zfs-devel] Content-Type: text/plain; charset="Windows-1252" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.34 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 26 Jan 2021 16:14:47 -0000 BONOLAOFFICE LO SPAZIO CHE TI SERVE QUANDO TI SERVE. Anche se fai fatica a crederci, ti confermiamo che la prova e' totalmente GRATUITA. Vivi l'esperienza di lavorare in un Business Center con tutti i servizi co= mpresi senza spendere un euro per 15 giorni. Nessun vincolo di acquisto di = servizi accessori ne di sottoscrizione di contratti. Siamo cosi=E2=80=99 si= curi dei servizi proposti che siamo disposti a investire per farteli vivere= nel quotidiano.=20 Non lasciarti sfuggire l'occasione!!! UFFICI E UTENZE COMPLETAMENTE GRATUITI!* Fai ripartire la tua azienda grazie a BonolaOffice.com Perche' non approfittare dell'attuale emergenza sanitaria per riorganizzare la tua azienda=3F Passa da un ufficio classico ad uno dinamico!=09 La nostra struttura riorganizzata con uffici e spazi ad uso esclusivo, garantisce oltre alla privacy, la sicurezza di poter operare in spazi dedi= cati senza la necessita' di dover interagire con il personale di altre aziende. In questo periodo di emergenza, il giusto distanziamento e le giuste precauzioni sono diventate indispensabili per poter ripartire con la massima sicurezza! Gli uffici BonolaOffice.com sono ideali sia per aziende che per liberi professionisti, le diverse conformazioni e metrature semindipendenti con=20 soluzioni da 20 a 600 mq, sono in grado di soddisfare ogni tipo di esigenz= a. =20 From owner-zfs-devel@freebsd.org Fri Feb 5 07:56:44 2021 Return-Path: Delivered-To: zfs-devel@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id C8FAC540603 for ; Fri, 5 Feb 2021 07:56:44 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4DX76N5GcMz4fBg for ; Fri, 5 Feb 2021 07:56:44 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id A3FCC1B419 for ; Fri, 5 Feb 2021 07:56:44 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 1157uiu6074294 for ; Fri, 5 Feb 2021 07:56:44 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 1157uitT074293 for zfs-devel@FreeBSD.org; Fri, 5 Feb 2021 07:56:44 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: zfs-devel@FreeBSD.org Subject: [Bug 249115] The zfs command in the installer fails to log a set command Date: Fri, 05 Feb 2021 07:56:44 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: bin X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: dclarke@blastwave.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: zfs-devel@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Feb 2021 07:56:44 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D249115 --- Comment #4 from Dennis Clarke --- I just checked with the latest ALPHA3 installer image for amd64 and the bug is still in place. It is strange that only the first command entered gets ignored and even returns an error to the console but the command succeeds. The change of a property to a ZFS filesystem works as expected but it is not logged in any way in the zpool history.=20=20 Any command entered afterwards is logged.=20 I used FreeBSD-13.0-ALPHA3-amd64-20210129-40cb0344eb2-256214-disc1.iso as the installer disk image. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-zfs-devel@freebsd.org Fri Feb 5 08:22:16 2021 Return-Path: Delivered-To: zfs-devel@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 04F7F541121 for ; Fri, 5 Feb 2021 08:22:16 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4DX7gq6dyQz4gvR for ; Fri, 5 Feb 2021 08:22:15 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id D74541BA87 for ; Fri, 5 Feb 2021 08:22:15 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 1158MFfq090923 for ; Fri, 5 Feb 2021 08:22:15 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 1158MFDw090922 for zfs-devel@FreeBSD.org; Fri, 5 Feb 2021 08:22:15 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: zfs-devel@FreeBSD.org Subject: [Bug 249115] The zfs command in the installer fails to log a set command Date: Fri, 05 Feb 2021 08:22:15 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: bin X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: avg@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: assigned_to Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Feb 2021 08:22:16 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D249115 Andriy Gapon changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|zfs-devel@FreeBSD.org |fs@FreeBSD.org --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-zfs-devel@freebsd.org Thu Feb 11 12:28:09 2021 Return-Path: Delivered-To: zfs-devel@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 3818C54638E for ; Thu, 11 Feb 2021 12:28:09 +0000 (UTC) (envelope-from commerciale@livejob.it) Received: from livejob.it (livejob.it [185.221.173.42]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Dbwrm1z2Qz3QdC for ; Thu, 11 Feb 2021 12:28:08 +0000 (UTC) (envelope-from commerciale@livejob.it) Received: from [93.37.216.147] (port=40591 helo=SendBlaster) by bonolaoffice.serverkeliweb.it with esmtpa (Exim 4.93) (envelope-from ) id 1lAB4l-0002bC-TR for zfs-devel@freebsd.org; Thu, 11 Feb 2021 13:28:04 +0100 MIME-Version: 1.0 From: "TermoScanner - DPI anti Covid19" Reply-To: commerciale@livejob.it To: zfs-devel@freebsd.org Subject: TermoScanner e strumenti DPI Anti-Covid in pronta consegna con sconti oltre il 70% e prezzi a partire da Euro 39, 00 + spedizione gratuita. Non Abbassiamo la guardia!!! X-Mailer: Smart_Send_4_4_2 Date: Thu, 11 Feb 2021 13:28:04 +0100 Message-ID: <13884861799761184725803@WIN10-Client02> X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - bonolaoffice.serverkeliweb.it X-AntiAbuse: Original Domain - freebsd.org X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - livejob.it X-Get-Message-Sender-Via: bonolaoffice.serverkeliweb.it: authenticated_id: commerciale@livejob.it X-Authenticated-Sender: bonolaoffice.serverkeliweb.it: commerciale@livejob.it X-Source: X-Source-Args: X-Source-Dir: X-Rspamd-Queue-Id: 4Dbwrm1z2Qz3QdC X-Spamd-Bar: ++ X-Spamd-Result: default: False [2.49 / 15.00]; HAS_REPLYTO(0.00)[commerciale@livejob.it]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_SPF_ALLOW(-0.20)[+a:c]; HAS_X_SOURCE(0.00)[]; REPLYTO_ADDR_EQ_FROM(0.00)[]; TO_DN_NONE(0.00)[]; URI_COUNT_ODD(1.00)[11]; DKIM_TRACE(0.00)[livejob.it:+]; DMARC_POLICY_ALLOW(-0.50)[livejob.it,reject]; NEURAL_HAM_SHORT(-1.00)[-1.000]; SUBJECT_ENDS_EXCLAIM(0.00)[]; HAS_X_ANTIABUSE(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:202675, ipnet:185.221.173.0/24, country:IT]; HAS_X_AS(0.00)[commerciale@livejob.it]; ARC_NA(0.00)[]; RBL_DBL_DONT_QUERY_IPS(0.00)[185.221.173.42:from]; R_DKIM_ALLOW(-0.20)[livejob.it:s=default]; FROM_HAS_DN(0.00)[]; EXT_CSS(1.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; HTML_SHORT_LINK_IMG_1(2.00)[]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; NEURAL_HAM_LONG(-1.00)[-1.000]; HAS_LIST_UNSUB(-0.01)[]; RCPT_COUNT_ONE(0.00)[1]; SPAMHAUS_ZRD(0.00)[185.221.173.42:from:127.0.2.255]; NEURAL_SPAM_MEDIUM(1.00)[0.999]; HAS_X_GMSV(0.00)[commerciale@livejob.it]; MID_RHS_NOT_FQDN(0.50)[]; RCVD_COUNT_TWO(0.00)[2]; MAILMAN_DEST(0.00)[zfs-devel] Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.34 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 11 Feb 2021 12:28:09 -0000 Il Tuo Partner Tecnologico TERMOSCANNER ANTI COVID-19=20 Misurazione della temperatura corporean in tempo reale!=20 nostri prodotti sono marchiati CE e coperti da regolare Garanzia di Legge. Garantiamo il supporto tecnico gratuito da remoto, per la messa in funzione degli strumenti, Termoscanner con case in alluminio pressofuso e display 8'' munito di inte= lligenza artificiale atta alla rilevazione automatica e a distanza della te= mperatura corporea con riconoscimento facciale. Oltre alla temperatura rile= va il corretto o mancato utilizzo della mascherina permettendo il riconosci= mento dei volti per il controllo degli accessi automatizzati con tornelli o= porte automatiche. Il software fornito con il dispositivo, se voluto, perm= ette di creare un=E2=80=99anagrafica delle persone autorizzate all=E2=80=99= ingresso nella struttura, consentendo l'esportazione delle letture o l=E2= =80=99importazione di anagrafiche da fonti esterne ed evitando onerosi cari= camenti manuali. E' possibile salvare tutti gli eventi o non registrare alcun dato a seconda= delle proprie policy sulla privacy e nel rispetto del GDPR applicato. Dispositivo standalone per la prevenzione epidemica che consente la rilevaz= ione della temperatura corporea con avviso sonoro e luminoso al superamento= di una soglia di temperatura. Alimentatore esterno in dotazione. Connettor= e USB per collegamento con batteria esterna opzionale. Possibilit=C3=A0 di = montaggio a muro o su piedistallo treppiedi.=20 From owner-zfs-devel@freebsd.org Sat Jun 12 03:41:53 2021 Return-Path: Delivered-To: zfs-devel@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 043BB642B9C for ; Sat, 12 Jun 2021 03:41:53 +0000 (UTC) (envelope-from commerciale@livejob.info) Received: from livejob.info (livejob.info [185.17.107.91]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4G23Rg70RGz3hFY for ; Sat, 12 Jun 2021 03:41:51 +0000 (UTC) (envelope-from commerciale@livejob.info) Received: from 93-50-165-11.ip153.fastwebnet.it ([93.50.165.11]:36167 helo=WIN10Client02bonolaofficelocal) by bonolaoffice.serverkeliweb.it with esmtpsa (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1lruWl-0004C7-Gv for zfs-devel@freebsd.org; Sat, 12 Jun 2021 05:41:49 +0200 MIME-Version: 1.0 From: "Risorse Umane" Reply-To: commerciale@livejob.info To: zfs-devel@freebsd.org Subject: Smaltire i Toner esausti in modo green non e' mai stato cosi' semplice. Approfitta inoltre del ritiro in sede GRATUITO in TUTTA Italia e ZERO multe. X-Mailer: Smart_Send_4_4_2 Date: Sat, 12 Jun 2021 05:41:45 +0200 Message-ID: <3680469484608248173485@WIN10-Client02> X-Priority: 1 X-MSMail-Priority: High X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - bonolaoffice.serverkeliweb.it X-AntiAbuse: Original Domain - freebsd.org X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - livejob.info X-Get-Message-Sender-Via: bonolaoffice.serverkeliweb.it: authenticated_id: commerciale@livejob.info X-Authenticated-Sender: bonolaoffice.serverkeliweb.it: commerciale@livejob.info X-Source: X-Source-Args: X-Source-Dir: X-Rspamd-Queue-Id: 4G23Rg70RGz3hFY X-Spamd-Bar: / X-Spamd-Result: default: False [-0.36 / 15.00]; HAS_REPLYTO(0.00)[commerciale@livejob.info]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:185.17.107.91:c]; HAS_X_SOURCE(0.00)[]; REPLYTO_ADDR_EQ_FROM(0.00)[]; TO_DN_NONE(0.00)[]; DKIM_TRACE(0.00)[livejob.info:+]; DMARC_POLICY_ALLOW(-0.50)[livejob.info,reject]; NEURAL_HAM_SHORT(-1.00)[-1.000]; HAS_X_ANTIABUSE(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; HAS_X_PRIO_ONE(0.00)[1]; MISSING_MIMEOLE(2.00)[]; ASN(0.00)[asn:202675, ipnet:185.17.107.0/24, country:IT]; R_PARTS_DIFFER(0.06)[53.2%]; HAS_X_AS(0.00)[commerciale@livejob.info]; ARC_NA(0.00)[]; RBL_DBL_DONT_QUERY_IPS(0.00)[185.17.107.91:from]; R_DKIM_ALLOW(-0.20)[livejob.info:s=default]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-0.91)[-0.909]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; HTML_SHORT_LINK_IMG_2(1.00)[]; HAS_LIST_UNSUB(-0.01)[]; RCPT_COUNT_ONE(0.00)[1]; SPAMHAUS_ZRD(0.00)[185.17.107.91:from:127.0.2.255]; HAS_X_GMSV(0.00)[commerciale@livejob.info]; MID_RHS_NOT_FQDN(0.50)[]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[zfs-devel] Content-Type: text/plain; charset="Windows-1252" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.34 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 12 Jun 2021 03:41:53 -0000 l problema Aziendale del Trattamento dei Rifiuti di Stampa LA SOLUZIONe La Programs& Projects Consulting attraverso il = servizio per la Manutenzione dei Consumabili, quali =20 i TONER DI =20 STAMPA offre l'unico recupero che trasferisce la = Paternita' del rifiuto. LE SANZIONI=20 Le aziende che nonrispettano in tutto o in parte lalegge 152, sono suscettibili di SANZIONIPENALI oltre che di SANZIONIAMMINISTRATIVE DA = EURO 1.032,00 A EURO92.962,00 From owner-zfs-devel@freebsd.org Fri Jul 2 04:27:44 2021 Return-Path: Delivered-To: zfs-devel@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 1228465D654 for ; Fri, 2 Jul 2021 04:27:44 +0000 (UTC) (envelope-from commerciale@livejob.it) Received: from livejob.it (livejob.it [185.221.173.42]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4GGMWL6KGnz4bmB for ; Fri, 2 Jul 2021 04:27:42 +0000 (UTC) (envelope-from commerciale@livejob.it) Received: from [93.50.165.11] (port=37836 helo=WIN10Client02bonolaofficelocal) by bonolaoffice.serverkeliweb.it with esmtpsa (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1lysOH-00088H-DG for zfs-devel@freebsd.org; Thu, 01 Jul 2021 10:49:49 +0200 MIME-Version: 1.0 From: "Specialisti Informatici ICT" Reply-To: commerciale@livejob.it To: zfs-devel@freebsd.org Subject: Se cerchi un software adatto alle tue esigenze aziendali o un tecnico informatico per realizzarlo, contattaci, siamo la soluzione ideale per risolvere i tuoi problemi. X-Mailer: Smart_Send_4_4_2 Date: Thu, 1 Jul 2021 10:49:48 +0200 Message-ID: <42641361776561006717469@WIN10-Client02> X-Priority: 1 X-MSMail-Priority: High X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - bonolaoffice.serverkeliweb.it X-AntiAbuse: Original Domain - freebsd.org X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - livejob.it X-Get-Message-Sender-Via: bonolaoffice.serverkeliweb.it: authenticated_id: commerciale@livejob.it X-Authenticated-Sender: bonolaoffice.serverkeliweb.it: commerciale@livejob.it X-Source: X-Source-Args: X-Source-Dir: X-Rspamd-Queue-Id: 4GGMWL6KGnz4bmB X-Spamd-Bar: ++++ X-Spamd-Result: default: False [4.90 / 15.00]; HAS_REPLYTO(0.00)[commerciale@livejob.it]; RCVD_VIA_SMTP_AUTH(0.00)[]; GREYLIST(0.00)[pass,body]; ZERO_FONT(0.50)[5]; HAS_X_SOURCE(0.00)[]; REPLYTO_ADDR_EQ_FROM(0.00)[]; TO_DN_NONE(0.00)[]; URI_COUNT_ODD(1.00)[17]; R_SPF_ALLOW(-0.20)[+a]; DKIM_TRACE(0.00)[livejob.it:+]; DMARC_POLICY_ALLOW(-0.50)[livejob.it,quarantine]; NEURAL_HAM_SHORT(-1.00)[-0.998]; HAS_X_ANTIABUSE(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; HAS_X_PRIO_ONE(0.00)[1]; MISSING_MIMEOLE(2.00)[]; ASN(0.00)[asn:202675, ipnet:185.221.173.0/24, country:IT]; HAS_X_AS(0.00)[commerciale@livejob.it]; ARC_NA(0.00)[]; RBL_DBL_DONT_QUERY_IPS(0.00)[185.221.173.42:from]; R_DKIM_ALLOW(-0.20)[livejob.it:s=default]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; HTML_SHORT_LINK_IMG_3(0.50)[]; HAS_LIST_UNSUB(-0.01)[]; RCPT_COUNT_ONE(0.00)[1]; SPAMHAUS_ZRD(0.00)[185.221.173.42:from:127.0.2.255]; MANY_INVISIBLE_PARTS(0.60)[7]; NEURAL_SPAM_MEDIUM(1.00)[1.000]; NEURAL_SPAM_LONG(0.81)[0.812]; HAS_X_GMSV(0.00)[commerciale@livejob.it]; MID_RHS_NOT_FQDN(0.50)[]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[zfs-devel] Content-Type: text/plain; charset="Windows-1252" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.34 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 02 Jul 2021 04:27:44 -0000 La Programs &Projects Consulting e' il partner ideale per le aziende. Proponiamo un'offerta di servizi completa ed integrata, dalla consulenza ne= lla progettazione di sistemi informatici custom alla selezione e valorizzaz= ione delle risorse umane, alla vendita di sistemi CRM scalabili completamen= te personalizzabili. Oltre tre lustri di esperienza e professionalita' al vostro servizio. Disponiamodi una vasta rosadi tecnici altamente specializzati distribuiti s= u tutto il territorio nazionale, adatti ad ogni vostra esigenza. Tutti i pr= ofessionisti sonoinquadraticomedipendenti diretti della nostraazienda. La Programs & Projects Consulting e' il tassello che ti manca. Qualsiasisia la vostra attivita', avere un Partneraltamente specializzato c= he possa supportarvi dalla fase di progettazione alla messa inproduzione, e= quivale agettare dellesolidefondamenta per il propriofuturoaziendale. From owner-zfs-devel@freebsd.org Wed Jul 21 03:31:16 2021 Return-Path: Delivered-To: zfs-devel@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 19CEC659A00 for ; Wed, 21 Jul 2021 03:31:16 +0000 (UTC) (envelope-from commerciale@livejob.cloud) Received: from livejob.cloud (livejob.cloud [185.17.107.91]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4GV1MR0WjWz4ps4 for ; Wed, 21 Jul 2021 03:31:14 +0000 (UTC) (envelope-from commerciale@livejob.cloud) Received: from 93-50-165-11.ip153.fastwebnet.it ([93.50.165.11]:65523 helo=WIN10Client02bonolaofficelocal) by bonolaoffice.serverkeliweb.it with esmtpsa (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1m62ws-0001xG-Sw for zfs-devel@freebsd.org; Wed, 21 Jul 2021 05:31:11 +0200 MIME-Version: 1.0 From: "SmartWatch per Ragazzi" Reply-To: commerciale@livejob.cloud To: zfs-devel@freebsd.org Subject: SmartWatch con telefono integrato e geolocalizzazione per bambini e ragazzi. Concediti delle vacanze in totale sicurezza e relax, avendo la certezza di dove sono i tuoi figli in ogni momento. X-Mailer: Smart_Send_4_4_2 Date: Wed, 21 Jul 2021 05:31:08 +0200 Message-ID: <8376257420288344316929@WIN10-Client02> X-Priority: 1 X-MSMail-Priority: High X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - bonolaoffice.serverkeliweb.it X-AntiAbuse: Original Domain - freebsd.org X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - livejob.cloud X-Get-Message-Sender-Via: bonolaoffice.serverkeliweb.it: authenticated_id: commerciale@livejob.cloud X-Authenticated-Sender: bonolaoffice.serverkeliweb.it: commerciale@livejob.cloud X-Source: X-Source-Args: X-Source-Dir: X-Rspamd-Queue-Id: 4GV1MR0WjWz4ps4 X-Spamd-Bar: / X-Spamd-Result: default: False [0.54 / 15.00]; HAS_REPLYTO(0.00)[commerciale@livejob.cloud]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:185.17.107.91]; HAS_X_SOURCE(0.00)[]; REPLYTO_ADDR_EQ_FROM(0.00)[]; TO_DN_NONE(0.00)[]; URI_COUNT_ODD(1.00)[15]; DKIM_TRACE(0.00)[livejob.cloud:+]; DMARC_POLICY_ALLOW(-0.50)[livejob.cloud,reject]; NEURAL_HAM_SHORT(-1.00)[-1.000]; HAS_X_ANTIABUSE(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; HAS_X_PRIO_ONE(0.00)[1]; MISSING_MIMEOLE(2.00)[]; ASN(0.00)[asn:202675, ipnet:185.17.107.0/24, country:IT]; HAS_X_AS(0.00)[commerciale@livejob.cloud]; ARC_NA(0.00)[]; RBL_DBL_DONT_QUERY_IPS(0.00)[185.17.107.91:from]; R_DKIM_ALLOW(-0.20)[livejob.cloud:s=default]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; FROM_HAS_DN(0.00)[]; EXT_CSS(1.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-0.999]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; HAS_LIST_UNSUB(-0.01)[]; RCPT_COUNT_ONE(0.00)[1]; SPAMHAUS_ZRD(0.00)[185.17.107.91:from:127.0.2.255]; MANY_INVISIBLE_PARTS(0.05)[1]; HAS_X_GMSV(0.00)[commerciale@livejob.cloud]; RWL_MAILSPIKE_POSSIBLE(0.00)[185.17.107.91:from]; MID_RHS_NOT_FQDN(0.50)[]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[zfs-devel] Content-Type: text/plain; charset="Windows-1252" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.34 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 Jul 2021 03:31:16 -0000 PP-W07 BABYWATCH e' uno smartwatch di ultima generazione con monitor touch = screen HD da 1,44'', ideale per i vostri bambini, divertente, colorato con = due colori vivaci. Resistente agli urti e all'acqua non teme la vivacita' d= ei vostri piccoli.=20 Nel PP-W07 BABYWATCH sono presenti diverse funzionalita' tra cui una FOTOCA= MERA con la quale e' possibile giocare ma anche utile per voi per comunicar= e visivamente direttamente da APP con i vostri bambini. Tramite la micro-SI= M installata sul dispositivo e' inoltre possibile GEOLOCALIZAZZARE il BABYW= ATCH ovunque sia, tenendo anche traccia dello storico dei movimenti. Non so= lo, una funzionalia' che certamente attirera' la vostra attenzione e' la fu= nzionalia' chiamata "ZONA DI SICUREZZA". In che cosa consiste=3F Semplice e= ' possibile impostare un raggio variabile all'interno in cui i vostri bambi= ni possono spostarsi senza problema ma basta oltrepassare il confine e il v= ostro telefono tramite APP vi avvisera' dell'allontanamento. Immaginate di = essere al parco o i spiaggia e volervi godere un po' di relax senza stare t= utto il tempo a controllare i movimenti dei vostri piccoli, quale funzional= ia' migliore potete immaginare=3F=20 From owner-zfs-devel@freebsd.org Mon Aug 9 19:25:52 2021 Return-Path: Delivered-To: zfs-devel@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id A676B65AAD7 for ; Mon, 9 Aug 2021 19:25:52 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: from mail-qk1-x733.google.com (mail-qk1-x733.google.com [IPv6:2607:f8b0:4864:20::733]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Gk5f75GMhz3LLZ for ; Mon, 9 Aug 2021 19:25:51 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: by mail-qk1-x733.google.com with SMTP id a19so19653770qkg.2 for ; Mon, 09 Aug 2021 12:25:51 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=sTfygUslUVkI/O96tu50g33A10xbumfcN4BuJdCUQVE=; b=RGZJNkA/30O0S7VQCbW+JPeTfGMiH3/jD2Bve6tmOnd3Q3F1uMVBjCqSfWb4IhvATL FQQi1Nvyp9ba0KVzSUjQy9pkwI0BDncDpEs+bjqw0AT45YIDWq49TWgAncqxEYtZU7M4 WNbkCcCIuHrxeRx6jGHUZEB69rB3fFZ/QebmKOI4zwIl/vtoMznrB8W8BXFsqwIaz5ev fKP6fIjC/2M2FB1Iis/7/VKRBAVrHHtBRZZKB/jdLEjX8LQacR/OiZj6P1Ph/APN5fhN t6cUYxDdeMTjI76fzVpmERz2U9scXe9cAGxfxKa66+Pv2apiBYl5DaRSiSNaiN2OpsEQ GyvQ== X-Gm-Message-State: AOAM530r/bMV7XZQshASwTrX7dwGqUeXz7yeMMM04yYSh7/tJGvC52eY 8UEJ+7cmJ80JLn91nfRRLgU/zTXtsZriz843uzXQxg== X-Google-Smtp-Source: ABdhPJwJaRx+xjeFScYlnbf33A8GhFbCjwOrhSyy1YcKTuOJ6tDwiUl1O4b8weqpWzFz5SztUl5HZkugwas+abJ+baY= X-Received: by 2002:a37:9e11:: with SMTP id h17mr23255904qke.193.1628537151015; Mon, 09 Aug 2021 12:25:51 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Matthew Ahrens Date: Mon, 9 Aug 2021 12:25:39 -0700 Message-ID: Subject: OpenZFS Developer Summit 2021 - announcement & CFP To: developer , illumos-zfs , Devel , zfs-devel@freebsd.org, freebsd-fs , zfs-discuss X-Rspamd-Queue-Id: 4Gk5f75GMhz3LLZ X-Spamd-Bar: -- X-Spamd-Result: default: False [-2.70 / 15.00]; RCVD_TLS_ALL(0.00)[]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[delphix.com:s=google]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[zfs-devel@freebsd.org]; URI_COUNT_ODD(1.00)[1]; RCPT_COUNT_FIVE(0.00)[6]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[delphix.com:+]; DMARC_POLICY_ALLOW(-0.50)[delphix.com,reject]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::733:from]; NEURAL_HAM_SHORT(-1.00)[-1.000]; FORGED_SENDER(0.30)[mahrens@delphix.com,matthew.ahrens@delphix.com]; MIME_TRACE(0.00)[0:+,1:+,2:~]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[mahrens@delphix.com,matthew.ahrens@delphix.com]; MAILMAN_DEST(0.00)[zfs-devel]; RCVD_COUNT_TWO(0.00)[2] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.34 X-BeenThere: zfs-devel@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: ZFS file system development for FreeBSD discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 09 Aug 2021 19:25:52 -0000 The ninth annual OpenZFS Developer Summit will be held online, *November 8-9, 2021*. All OpenZFS developers are invited to participate, and (free!) registration is now open. See http://www.open-zfs.org/wiki/OpenZFS_Developer_Summit_2021 for all of the details, including slides and videos from previous years' conferences. The goal of the event is to foster cross-community discussions of OpenZFS work and to make progress on some of the projects we have proposed. The first day of the conference will be set aside for presentations. *If you would like to give a talk, submit a proposal via email to matt@mahrens.org , including a 1-2 paragraph abstract.* Talks may be recorded prior to the event, and speakers' (virtual) attendance the day of the event will allow for everyone to experience the talks together, and for Q&A with attendees. We welcome proposals for any kinds of talks related to ZFS, and we are particularly interested in fostering a diverse group of speakers. In the past, the most popular talks have featured internal details of new or upcoming features in OpenZFS, and explanations of how existing subsystems work. Mentorship is available to help make your presentation the best it can be. The deadlines are as follows: *September 3, 2021 All abstracts/proposals submitted* to matt@mahrens.org September 10, 2021 Proposal submitters notified November 8-9, 2021 OpenZFS Developer Summit The second day of the event will be a hackathon, time set aside to work with OpenZFS developers, with the goal of having some progress to report at the end of the day. This event is only possible because of the efforts and contributions of the community of developers and companies that sponsor the event. Although our costs are lower this year with an online conference, sponsorship opportunities are available. Please see the website for details and send email to victoria@vgfevents.com if you have any questions. --matt > >