From nobody Sat Apr 22 18:34:35 2023 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Q3g6d5V9Lz4674Z; Sat, 22 Apr 2023 18:34:49 +0000 (UTC) (envelope-from aryeh.friedman@gmail.com) Received: from mail-ed1-x52f.google.com (mail-ed1-x52f.google.com [IPv6:2a00:1450:4864:20::52f]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Q3g6d1f08z3rV5; Sat, 22 Apr 2023 18:34:49 +0000 (UTC) (envelope-from aryeh.friedman@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-ed1-x52f.google.com with SMTP id 4fb4d7f45d1cf-50506ac462bso4201860a12.3; Sat, 22 Apr 2023 11:34:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1682188487; x=1684780487; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=4ezYZ+DTAfqiWOwLpHzGH9mR3wuwYlATkifkpUMIZgU=; b=UNkpVyGo3qH9g3gOvRSH/RO4wapDPKceFu3mzu+VFsifMEEj0F5dP9IQW5ZKia9+PJ /Dmr1Jlekd3YypjFH03RbU2KP2++Mj+reb5M+gJZCqYMYMnwsW0vjot8OgYufYpfQ5r3 5Z6QIEJQZ4X9dxy4NPpO/Vo3QUEv1rb04rtYa0U2P8Uk7RTfFH1onLSItL7/4vTgyFfn E6CqMQtIpqQMydr0X/8sau4LeADzfK+/6vGhFnwbF0eKDKpKSgziNz8SACACosOG2lYl 9e5mOtm4Vh9GkwZsepON3aNWXQen4CS4LWM0hAa8VHDDh5AWIL/TNlwTxF5y0BtLiShK TH6Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1682188487; x=1684780487; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=4ezYZ+DTAfqiWOwLpHzGH9mR3wuwYlATkifkpUMIZgU=; b=GYZxk2FzeE2NZhd60uVkCiexuZE5OebQ4Bf+KeefwzvIFpNHKcNUFfhMnMgctK4jOl 6C1ycDFgff0cfh+j2LGmny6rfN0ZXjBd7qC6U1u5sA/p/Ds8cMZ6Uu9VWCNLXm+P+A9w WFSxY//jpCz9HTr81yAx33A/u1S5jhjr4pg6Q+9GTHFKd9WWjLrEXf+c3edPUDkNwOs2 Eb6MRY87DUg6gieChnDdTpZ5CxJlyjzZ21f1Zus6aJt/THuBcUi2JrKyhSu4iUZBnwQn PW5vdsvkU0kdUcZfNJ3klG86f9KGzgs3ogSL65bApS7LJBK5FO1m78DmQXTQRvPC4VDK 3W6w== X-Gm-Message-State: AAQBX9diCKPNAdYOJxc87gI/x8zf6UVdDH/GGYLRg4VhJBo3UzIW2H9i gv2nA861VU1S76cgnNhzfhN7m/gBcRwXoklxAQzDEr/W X-Google-Smtp-Source: AKy350b8IE2WiBgJV2I1v2rTQb43On8dD1yNinmhtBQYPfm0IdSfvVasP+YokxYforGNWbvAtjvu6BkOrABrIpHgHb4= X-Received: by 2002:aa7:c14c:0:b0:506:83e7:8c6c with SMTP id r12-20020aa7c14c000000b0050683e78c6cmr7538236edp.10.1682188487553; Sat, 22 Apr 2023 11:34:47 -0700 (PDT) List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 References: In-Reply-To: From: Aryeh Friedman Date: Sat, 22 Apr 2023 14:34:35 -0400 Message-ID: Subject: Re: Installing openAI's GPT-2 Ada AI Language Model To: Mario Marietto Cc: freebsd-hackers , Yuri Victorovich , FreeBSD Mailing List , Odhiambo Washington Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: 4Q3g6d1f08z3rV5 X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; TAGGED_FROM(0.00)[] X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-ThisMailContainsUnwantedMimeParts: N On Sat, Apr 22, 2023 at 2:14=E2=80=AFPM Mario Marietto wrote: > > I don't know. This should be evaluated by you. I'm not involved so much i= n the technicalities : > > https://github.com/lm-sys/FastChat > > Let me understand what the Ada (117M) model is,if you want. I want to lea= rn. It is basically the smallest conversational model offered by the GPT-2/openAI team. The reason is I see babySpock as being an "corporate AI" (in that it mixes and matches models to get the best results). The primary problem I see with chatGPT (except for the cost for using it at the API level, ran up $25 bill in 2 days of just testing and developing babySpock against their API... this is financially unsustainable so I have to move it in house) is that due to its inability to mix and match context(s) [and the web ui to chatGPT having total context length limits] in order to give it a broad perspective of how I work and think (i.e. what "irrelevent" context to filter out but still get a reasonable reply)... I am planning to use the Ada model as a "cognitive CPU" in the production version babySpock and have a "OS tape" constantly looping through it.. the reason of course is the models are one shot affairs and are stateless between calls (i.e. needs external context) and thus if I was to have a cognitive layer for doing the context assembly I would need a stateful "cognitive OS" to do it on.... I have some semi-FOSS (BSD licensed but not 100% free) business ideas on how to scale this but the business philosophy here is not in the scope of a technical discussion unless you want to know and I will send some stuff privately. --=20 Aryeh M. Friedman, Lead Developer, http://www.PetiteCloud.org