InfiniBand in 10.0?
Dennis Glatting
freebsd at pki2.com
Wed Dec 25 02:01:09 UTC 2013
On Tue, 2013-12-24 at 19:42 -0600, iamatt wrote:
> And then you need to see what mpi are you going to use. I'm curious
> which one you plan on using and how well it works on FreeBSD.
>
I've used MPIch2 under FreeBSD with John The Ripper about a year ago. It
eventually worked but it was a struggle to get there. I have not been
able to get OpenMPI to work at my former job (also FreeBSD). I've twice
came close but in each case the project was turned off before I could
finish.
I'm running oclHascat under CentOS 6.5 in my home lab where I have >50T
(uncompressed) of words. Most of my servers are FreeBSD of various
versions.
>
> On Dec 24, 2013 4:16 PM, "Dennis Glatting" <freebsd at pki2.com> wrote:
> On Tue, 2013-12-24 at 15:46 -0600, iamatt wrote:
> > A good batch of mellanox cables. Melanic had a bad run of
> FDR early
> > in the year. Most FDR centers have been plagued with them.
> Even the
> > back end of our DDN lustres has had issues. They typically
> do not
> > just fail but downgrade to QDR rate.
> >
>
> Ah. I though they were stripping the technology InfinBand.
>
>
>
> > On Dec 24, 2013 3:24 PM, "Dennis Glatting"
> <freebsd at pki2.com> wrote:
> > On Tue, 2013-12-24 at 14:43 -0600, iamatt wrote:
> > > Not sure if I would even bother with running IB on
> BSD. We
> > have enough
> > > problems just getting things stable with Linux and
> mellanox
> > ofed . Add all
> > > of the documented build problems with the mellanox
> FDR
> > cables and recalls
> > > and you are guaranteed a nightmare. TACC, navy,
> and almost
> > every center I
> > > am aware of has replaced all or most or some of
> their FDR,
> > including us.
> > >
> >
> > Replaced them with what?
> >
> >
> > > As far as cards go it's all mellanox. Qlogic lost
> the game,
> > still dealing
> > > with a 12800 qlogic switch which is a total
> nightmare.
> > Stick with
> > > mellanox. Different vendors used to make their
> own fabric
> > managers in the
> > > early time of IB but now its best to stick with
> open ibd.
> > >
> > > FreeBSD and IB gives me the chills just thinking
> about it.
> > > On Dec 24, 2013 1:59 PM, "Dennis Glatting"
> > <freebsd at pki2.com> wrote:
> > >
> > > >
> > > > This site appears dated:
> > > >
> > > > https://wiki.freebsd.org/InfiniBand
> > > >
> > > > I have a couple of questions.
> > > >
> > > > 1) Under 10.0-RC2, is InfiniBand a part of
> GENERIC or did
> > I still have
> > > > to modify the conf as specified in the wiki? I
> don't see
> > any of the
> > > > options from the wiki in 10.0-RC2 GENERIC.
> > > >
> > > > 2) Any recommendation on cards? What works, what
> doesn't?
> > > >
> > > > 3) What things to watch out for?
> > > >
> > > >
> > > > Thanks.
> > > >
> > > >
> > > >
> > > >
> > > > _______________________________________________
> > > > freebsd-questions at freebsd.org mailing list
> > > >
> >
> http://lists.freebsd.org/mailman/listinfo/freebsd-questions
> > > > To unsubscribe, send any mail to "
> > > > freebsd-questions-unsubscribe at freebsd.org"
> > > >
> > > _______________________________________________
> > > freebsd-questions at freebsd.org mailing list
> > >
> http://lists.freebsd.org/mailman/listinfo/freebsd-questions
> > > To unsubscribe, send any mail to
> > "freebsd-questions-unsubscribe at freebsd.org"
> >
> >
>
>
More information about the freebsd-questions
mailing list