svn commit: r213049 - stable/8/sbin/hastd
Pawel Jakub Dawidek
pjd at FreeBSD.org
Thu Sep 23 09:02:10 UTC 2010
Author: pjd
Date: Thu Sep 23 09:02:10 2010
New Revision: 213049
URL: http://svn.freebsd.org/changeset/base/213049
Log:
MFC r208028,r210368,r210702,r210869,r210870,r210872,r210873,r210875,r210876,
r210879,r210880,r210881,r210882,r210883,r210886,r210892,r211397,r211407,
r211452,r211875,r211876,r211877,r211878,r211879,r211880,r211881,r211882,
r211883,r211884,r211885,r211886,r211887,r211895,r211896,r211897,r211898,
r211899,r211975,r211976,r211977,r211978,r211979,r211981,r211982,r211983,
r211984,r212033,r212034,r212036,r212037,r212038,r212046,r212049,r212051,
r212052,r212899,r213003,r213004,r213006,r213007,r213008,r213009:
r208028:
mdoc: move remaining sections into consistent order
This pertains mostly to FILES, HISTORY, EXIT STATUS and AUTHORS sections.
Found by: mdocml lint run
Reviewed by: ru
r210368:
Actually, only the fullsync mode is implemented, not memsync mode.
Correct manual page.
r210702:
Spelling fixes.
r210869:
Add an argument to the proto_register() function which allows protocol to
declare it is the default and be placed at the end of the queue so it is
checked last.
r210870:
Now that TCP will be checked last we don't need any knowledge about other
protocols.
r210872:
Mark two more places that we won't reach.
r210873:
Keep $FreeBSD$ in __FBSDID() only for C files.
r210875:
Problem with assertion is that it logs on stderr. Add two macros:
PJDLOG_ASSERT() and PJDLOG_VERIFY() that will check the given condition
and log the problem where appropriate. The difference between those
two is that PJDLOG_VERIFY() always work and PJDLOG_ASSERT() can be
turned off by defining NDEBUG.
r210876:
Assert that various buffers we are large enough.
r210879:
- Use pjdlog_exitx() to log errors and exit instead of errx().
- Use 'unable to' (instead of 'cannot') consistently.
r210880:
Reset signal handlers after fork().
r210881:
Allow to use 'none' keywork as remote address in case second cluster node
is not setup yet.
r210882:
Make control_set_role() more public. We will need it soon.
r210883:
Prepare configuration parsing code to be called multiple times:
- Don't exit on errors if not requested.
- Don't keep configuration in global variable, but allocate memory for
configuration.
- Call yyrestart() before yyparse() so that on error in configuration file
we will start from the begining next time and not from the place we left of.
r210886:
Implement configuration reload on SIGHUP. This includes:
- Load added resources.
- Stop and forget removed resources.
- Update modified resources in least intrusive way, ie. don't touch
/dev/hast/<name> unless path to local component or provider name were
modified.
Obtained from: Wheel Systems Sp. z o.o. http://www.wheelsystems.com
r210892:
Document 'none' value for remote.
Reviewed by: dougb
r211397:
Fix typos, spelling, formatting and mdoc mistakes found by Nobuyuki while
translating these manual pages. Minor corrections by me.
Submitted by: Nobuyuki Koganemaru <n-kogane at syd.odn.ne.jp>
r211407:
The 'size' variable is there to limit how many bytes we want to copy from
'addr'. It is very likely that size of 'addr' is larger than 'size', so checking
strlcpy() return value is bogus.
r211452:
For some setups sending data in 128kB chunks makes communication very slow. No
idea why. 32kB on the other hand seems to work properly everywhere.
Reported by: Thomas Steen Rasmussen <thomas at gibfest.dk>
r211875:
Make comment more readable.
Obtained from: Wheel Systems Sp. z o.o. http://www.wheelsystems.com
r211876:
Add mtx_owned() implementation.
Obtained from: Wheel Systems Sp. z o.o. http://www.wheelsystems.com
r211877:
Add QUEUE_INSERT() and QUEUE_TAKE() macros that simplify the code a bit.
Obtained from: Wheel Systems Sp. z o.o. http://www.wheelsystems.com
r211878:
We have sync_start() function to start synchronization, introduce sync_stop()
function to stop it.
Obtained from: Wheel Systems Sp. z o.o. http://www.wheelsystems.com
r211879:
Log that synchronization was interrupted in a proper place.
Obtained from: Wheel Systems Sp. z o.o. http://www.wheelsystems.com
r211880:
Don't increase number synchronized bytes in case of an error.
Obtained from: Wheel Systems Sp. z o.o. http://www.wheelsystems.com
r211881:
- Remove redundant and incorrect 'old' word from debug message.
- Log disconnects as warnings.
Obtained from: Wheel Systems Sp. z o.o. http://www.wheelsystems.com
r211882:
Implement keepalive mechanism inside HAST protocol so we can detect secondary
node failures quickly for HAST resources that are rarely modified.
Remove XXX from a comment now that the guard thread never sleeps infinitely.
Obtained from: Wheel Systems Sp. z o.o. http://www.wheelsystems.com
r211883:
Reduce indent where possible.
Obtained from: Wheel Systems Sp. z o.o. http://www.wheelsystems.com
r211884:
When logging to stdout/stderr don't close those descriptors after fork().
Obtained from: Wheel Systems Sp. z o.o. http://www.wheelsystems.com
r211885:
- Run hooks in background - don't block waiting for them to finish.
- Keep all hooks we're running in a global list, so we can report when
they finish and also report when they are running for too long.
Obtained from: Wheel Systems Sp. z o.o. http://www.wheelsystems.com
r211886:
Allow to execute specified program on various HAST events.
Obtained from: Wheel Systems Sp. z o.o. http://www.wheelsystems.com
r211887:
Document new 'exec' parameter.
Obtained from: Wheel Systems Sp. z o.o. http://www.wheelsystems.com
r211895:
Add hooks execution.
Obtained from: Wheel Systems Sp. z o.o. http://www.wheelsystems.com
r211896:
Check if no signals were delivered just before going to sleep.
Obtained from: Wheel Systems Sp. z o.o. http://www.wheelsystems.com
r211897:
Correct when we log interrupted synchronization.
Obtained from: Wheel Systems Sp. z o.o. http://www.wheelsystems.com
r211898:
When logging to stdout/stderr, flush after each log.
Obtained from: Wheel Systems Sp. z o.o. http://www.wheelsystems.com
r211899:
When SIGTERM or SIGINT is received, terminate worker processes.
Obtained from: Wheel Systems Sp. z o.o. http://www.wheelsystems.com
r211975:
Implement mtx_destroy() and rw_destroy().
Obtained from: Wheel Systems Sp. z o.o. http://www.wheelsystems.com
r211976:
- Add hook_fini() which should be called after fork() from the main hastd
process, once it start to use hooks.
- Add hook_check_one() in case the caller expects different child processes
and once it can recognize it, it will pass pid and status to hook_check_one().
Obtained from: Wheel Systems Sp. z o.o. http://www.wheelsystems.com
r211977:
Allow to run hooks from the main hastd process.
Obtained from: Wheel Systems Sp. z o.o. http://www.wheelsystems.com
r211978:
- Call hook on role change.
- Document new event.
Obtained from: Wheel Systems Sp. z o.o. http://www.wheelsystems.com
r211979:
Disconnect after logging errors.
Obtained from: Wheel Systems Sp. z o.o. http://www.wheelsystems.com
r211981:
- Move functionality responsible for checking one connection to separate
function to make code more readable.
- Be sure not to reconnect too often in case of signal delivery, etc.
Obtained from: Wheel Systems Sp. z o.o. http://www.wheelsystems.com
r211982:
Use sigtimedwait(2) for signals handling in primary process.
This fixes various races and eliminates use of pthread* API in signal handler.
Pointed out by: kib
With help from: jilles
Obtained from: Wheel Systems Sp. z o.o. http://www.wheelsystems.com
r211983:
Execute hook when split-brain is detected.
Obtained from: Wheel Systems Sp. z o.o. http://www.wheelsystems.com
r211984:
Execute hook when connection between the nodes is established or lost.
Obtained from: Wheel Systems Sp. z o.o. http://www.wheelsystems.com
r212033:
Constify arguments we can constify.
Obtained from: Wheel Systems Sp. z o.o. http://www.wheelsystems.com
r212034:
Use pjdlog_exit() before fork().
Obtained from: Wheel Systems Sp. z o.o. http://www.wheelsystems.com
r212036:
When someone gives NULL as data, assume this is because he want to declare
connection side only.
Obtained from: Wheel Systems Sp. z o.o. http://www.wheelsystems.com
r212037:
We only want to know if descriptors are ready for reading.
Obtained from: Wheel Systems Sp. z o.o. http://www.wheelsystems.com
r212038:
Because it is very hard to make fork(2) from threaded process safe (we are
limited to async-signal safe functions in the child process), move all hooks
execution to the main (non-threaded) process.
Do it by maintaining connection (socketpair) between child and parent
and sending events from the child to parent, so it can execute the hook.
This is step in right direction for others reasons too. For example there is
one less problem to drop privs in worker processes.
Obtained from: Wheel Systems Sp. z o.o. http://www.wheelsystems.com
r212046:
Mask only those signals that we want to handle.
Suggested by: jilles
Obtained from: Wheel Systems Sp. z o.o. http://www.wheelsystems.com
r212049:
Forgot to add event.c and event.h in r212038.
Pointed out by: pluknet <pluknet at gmail.com>
Obtained from: Wheel Systems Sp. z o.o. http://www.wheelsystems.com
r212051:
Correct error message.
Submitted by: Mikolaj Golub <to.my.trociny at gmail.com>
r212052:
Include process PID in log messages.
Submitted by: Mikolaj Golub <to.my.trociny at gmail.com>
r212899:
Add __dead2 to functions that we know they are going to exit.
r213003:
Sort includes.
r213004:
If we are unable to receive control message is most likely because the main
process died. Instead of entering infinite loop, terminate.
r213006:
Fix descriptor leaks: when child exits, we have to close control and event
socket pairs. We did that only in one case out of three.
r213007:
Fix possible deadlock where worker process sends an event to the main process
while the main process sends control message to the worker process, but worker
process hasn't started control thread yet, because it waits for reply from the
main process.
The fix is to start the control thread before sending any events.
Reported and fix suggested by: Mikolaj Golub <to.my.trociny at gmail.com>
r213008:
Assert that descriptor numbers are sane.
r213009:
Switch to sigprocmask(2) API also in the main process and secondary process.
This way the primary process inherits signal mask from the main process,
which fixes a race where signal is delivered to the primary process before
configuring signal mask.
Reported by: Mikolaj Golub <to.my.trociny at gmail.com>
Added:
stable/8/sbin/hastd/event.c
- copied unchanged from r212049, head/sbin/hastd/event.c
stable/8/sbin/hastd/event.h
- copied unchanged from r212049, head/sbin/hastd/event.h
Modified:
stable/8/sbin/hastd/Makefile
stable/8/sbin/hastd/control.c
stable/8/sbin/hastd/control.h
stable/8/sbin/hastd/hast.conf.5
stable/8/sbin/hastd/hast.h
stable/8/sbin/hastd/hast_proto.c
stable/8/sbin/hastd/hast_proto.h
stable/8/sbin/hastd/hastd.8
stable/8/sbin/hastd/hastd.c
stable/8/sbin/hastd/hastd.h
stable/8/sbin/hastd/hooks.c
stable/8/sbin/hastd/hooks.h
stable/8/sbin/hastd/parse.y
stable/8/sbin/hastd/pjdlog.c
stable/8/sbin/hastd/pjdlog.h
stable/8/sbin/hastd/primary.c
stable/8/sbin/hastd/proto.c
stable/8/sbin/hastd/proto.h
stable/8/sbin/hastd/proto_common.c
stable/8/sbin/hastd/proto_impl.h
stable/8/sbin/hastd/proto_socketpair.c
stable/8/sbin/hastd/proto_tcp4.c
stable/8/sbin/hastd/proto_uds.c
stable/8/sbin/hastd/secondary.c
stable/8/sbin/hastd/synch.h
stable/8/sbin/hastd/token.l
Directory Properties:
stable/8/sbin/hastd/ (props changed)
Modified: stable/8/sbin/hastd/Makefile
==============================================================================
--- stable/8/sbin/hastd/Makefile Thu Sep 23 05:24:50 2010 (r213048)
+++ stable/8/sbin/hastd/Makefile Thu Sep 23 09:02:10 2010 (r213049)
@@ -5,7 +5,7 @@
PROG= hastd
SRCS= activemap.c
SRCS+= control.c
-SRCS+= ebuf.c
+SRCS+= ebuf.c event.c
SRCS+= hast_proto.c hastd.c hooks.c
SRCS+= metadata.c
SRCS+= nv.c
Modified: stable/8/sbin/hastd/control.c
==============================================================================
--- stable/8/sbin/hastd/control.c Thu Sep 23 05:24:50 2010 (r213048)
+++ stable/8/sbin/hastd/control.c Thu Sep 23 09:02:10 2010 (r213049)
@@ -32,17 +32,19 @@ __FBSDID("$FreeBSD$");
#include <sys/types.h>
#include <sys/wait.h>
-#include <signal.h>
#include <assert.h>
#include <errno.h>
#include <pthread.h>
+#include <signal.h>
#include <stdio.h>
#include <string.h>
+#include <unistd.h>
#include "hast.h"
#include "hastd.h"
#include "hast_proto.h"
+#include "hooks.h"
#include "nv.h"
#include "pjdlog.h"
#include "proto.h"
@@ -50,19 +52,31 @@ __FBSDID("$FreeBSD$");
#include "control.h"
-static void
-control_set_role(struct hastd_config *cfg, struct nv *nvout, uint8_t role,
- struct hast_resource *res, const char *name, unsigned int no)
+void
+child_cleanup(struct hast_resource *res)
{
- assert(cfg != NULL);
- assert(nvout != NULL);
- assert(name != NULL);
+ proto_close(res->hr_ctrl);
+ res->hr_ctrl = NULL;
+ proto_close(res->hr_event);
+ res->hr_event = NULL;
+ res->hr_workerpid = 0;
+}
+
+static void
+control_set_role_common(struct hastd_config *cfg, struct nv *nvout,
+ uint8_t role, struct hast_resource *res, const char *name, unsigned int no)
+{
+ int oldrole;
/* Name is always needed. */
- nv_add_string(nvout, name, "resource%u", no);
+ if (name != NULL)
+ nv_add_string(nvout, name, "resource%u", no);
if (res == NULL) {
+ assert(cfg != NULL);
+ assert(name != NULL);
+
TAILQ_FOREACH(res, &cfg->hc_resources, hr_next) {
if (strcmp(res->hr_name, name) == 0)
break;
@@ -85,6 +99,7 @@ control_set_role(struct hastd_config *cf
pjdlog_info("Role changed to %s.", role2str(role));
/* Change role to the new one. */
+ oldrole = res->hr_role;
res->hr_role = role;
pjdlog_prefix_set("[%s] (%s) ", res->hr_name, role2str(res->hr_role));
@@ -106,13 +121,22 @@ control_set_role(struct hastd_config *cf
pjdlog_debug(1, "Worker process %u stopped.",
(unsigned int)res->hr_workerpid);
}
- res->hr_workerpid = 0;
+ child_cleanup(res);
}
/* Start worker process if we are changing to primary. */
if (role == HAST_ROLE_PRIMARY)
hastd_primary(res);
pjdlog_prefix_set("%s", "");
+ hook_exec(res->hr_exec, "role", res->hr_name, role2str(oldrole),
+ role2str(res->hr_role), NULL);
+}
+
+void
+control_set_role(struct hast_resource *res, uint8_t role)
+{
+
+ control_set_role_common(NULL, NULL, role, res, NULL, 0);
}
static void
@@ -306,7 +330,7 @@ control_handle(struct hastd_config *cfg)
TAILQ_FOREACH(res, &cfg->hc_resources, hr_next) {
switch (cmd) {
case HASTCTL_SET_ROLE:
- control_set_role(cfg, nvout, role, res,
+ control_set_role_common(cfg, nvout, role, res,
res->hr_name, ii++);
break;
case HASTCTL_STATUS:
@@ -329,8 +353,8 @@ control_handle(struct hastd_config *cfg)
break;
switch (cmd) {
case HASTCTL_SET_ROLE:
- control_set_role(cfg, nvout, role, NULL, str,
- ii);
+ control_set_role_common(cfg, nvout, role, NULL,
+ str, ii);
break;
case HASTCTL_STATUS:
control_status(cfg, nvout, NULL, str, ii);
@@ -375,7 +399,8 @@ ctrl_thread(void *arg)
pthread_exit(NULL);
pjdlog_errno(LOG_ERR,
"Unable to receive control message");
- continue;
+ kill(getpid(), SIGTERM);
+ pthread_exit(NULL);
}
cmd = nv_get_uint8(nvin, "cmd");
if (cmd == 0) {
Modified: stable/8/sbin/hastd/control.h
==============================================================================
--- stable/8/sbin/hastd/control.h Thu Sep 23 05:24:50 2010 (r213048)
+++ stable/8/sbin/hastd/control.h Thu Sep 23 09:02:10 2010 (r213049)
@@ -36,6 +36,11 @@
#define HASTCTL_STATUS 2
struct hastd_config;
+struct hast_resource;
+
+void child_cleanup(struct hast_resource *res);
+
+void control_set_role(struct hast_resource *res, uint8_t role);
void control_handle(struct hastd_config *cfg);
Copied: stable/8/sbin/hastd/event.c (from r212049, head/sbin/hastd/event.c)
==============================================================================
--- /dev/null 00:00:00 1970 (empty, because file is newly added)
+++ stable/8/sbin/hastd/event.c Thu Sep 23 09:02:10 2010 (r213049, copy of r212049, head/sbin/hastd/event.c)
@@ -0,0 +1,162 @@
+/*-
+ * Copyright (c) 2010 Pawel Jakub Dawidek <pjd at FreeBSD.org>
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE AUTHORS AND CONTRIBUTORS ``AS IS'' AND
+ * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+ * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHORS OR CONTRIBUTORS BE LIABLE
+ * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+ * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
+ * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
+ * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
+ * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
+ * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
+ * SUCH DAMAGE.
+ */
+
+#include <sys/cdefs.h>
+__FBSDID("$FreeBSD$");
+
+#include <assert.h>
+#include <errno.h>
+
+#include "hast.h"
+#include "hast_proto.h"
+#include "hooks.h"
+#include "nv.h"
+#include "pjdlog.h"
+#include "proto.h"
+#include "subr.h"
+
+#include "event.h"
+
+void
+event_send(const struct hast_resource *res, int event)
+{
+ struct nv *nvin, *nvout;
+ int error;
+
+ assert(res != NULL);
+ assert(event >= EVENT_MIN && event <= EVENT_MAX);
+
+ nvin = nvout = NULL;
+
+ /*
+ * Prepare and send event to parent process.
+ */
+ nvout = nv_alloc();
+ nv_add_uint8(nvout, (uint8_t)event, "event");
+ error = nv_error(nvout);
+ if (error != 0) {
+ pjdlog_common(LOG_ERR, 0, error,
+ "Unable to prepare event header");
+ goto done;
+ }
+ if (hast_proto_send(res, res->hr_event, nvout, NULL, 0) < 0) {
+ pjdlog_errno(LOG_ERR, "Unable to send event header");
+ goto done;
+ }
+ if (hast_proto_recv_hdr(res->hr_event, &nvin) < 0) {
+ pjdlog_errno(LOG_ERR, "Unable to receive event header");
+ goto done;
+ }
+ /*
+ * Do nothing with the answer. We only wait for it to be sure not
+ * to exit too quickly after sending an event and exiting immediately.
+ */
+done:
+ if (nvin != NULL)
+ nv_free(nvin);
+ if (nvout != NULL)
+ nv_free(nvout);
+}
+
+int
+event_recv(const struct hast_resource *res)
+{
+ struct nv *nvin, *nvout;
+ const char *evstr;
+ uint8_t event;
+ int error;
+
+ assert(res != NULL);
+
+ nvin = nvout = NULL;
+
+ if (hast_proto_recv_hdr(res->hr_event, &nvin) < 0) {
+ /*
+ * First error log as debug. This is because worker process
+ * most likely exited.
+ */
+ pjdlog_common(LOG_DEBUG, 1, errno,
+ "Unable to receive event header");
+ goto fail;
+ }
+
+ event = nv_get_uint8(nvin, "event");
+ if (event == EVENT_NONE) {
+ pjdlog_error("Event header is missing 'event' field.");
+ goto fail;
+ }
+
+ switch (event) {
+ case EVENT_CONNECT:
+ evstr = "connect";
+ break;
+ case EVENT_DISCONNECT:
+ evstr = "disconnect";
+ break;
+ case EVENT_SYNCSTART:
+ evstr = "syncstart";
+ break;
+ case EVENT_SYNCDONE:
+ evstr = "syncdone";
+ break;
+ case EVENT_SYNCINTR:
+ evstr = "syncintr";
+ break;
+ case EVENT_SPLITBRAIN:
+ evstr = "split-brain";
+ break;
+ default:
+ pjdlog_error("Event header contain invalid event number (%hhu).",
+ event);
+ goto fail;
+ }
+
+ pjdlog_prefix_set("[%s] (%s) ", res->hr_name, role2str(res->hr_role));
+ hook_exec(res->hr_exec, evstr, res->hr_name, NULL);
+ pjdlog_prefix_set("%s", "");
+
+ nvout = nv_alloc();
+ nv_add_int16(nvout, 0, "error");
+ error = nv_error(nvout);
+ if (error != 0) {
+ pjdlog_common(LOG_ERR, 0, error,
+ "Unable to prepare event header");
+ goto fail;
+ }
+ if (hast_proto_send(res, res->hr_event, nvout, NULL, 0) < 0) {
+ pjdlog_errno(LOG_ERR, "Unable to send event header");
+ goto fail;
+ }
+ nv_free(nvin);
+ nv_free(nvout);
+ return (0);
+fail:
+ if (nvin != NULL)
+ nv_free(nvin);
+ if (nvout != NULL)
+ nv_free(nvout);
+ return (-1);
+}
Copied: stable/8/sbin/hastd/event.h (from r212049, head/sbin/hastd/event.h)
==============================================================================
--- /dev/null 00:00:00 1970 (empty, because file is newly added)
+++ stable/8/sbin/hastd/event.h Thu Sep 23 09:02:10 2010 (r213049, copy of r212049, head/sbin/hastd/event.h)
@@ -0,0 +1,46 @@
+/*-
+ * Copyright (c) 2010 Pawel Jakub Dawidek <pjd at FreeBSD.org>
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE AUTHORS AND CONTRIBUTORS ``AS IS'' AND
+ * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+ * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHORS OR CONTRIBUTORS BE LIABLE
+ * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+ * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
+ * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
+ * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
+ * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
+ * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
+ * SUCH DAMAGE.
+ *
+ * $FreeBSD$
+ */
+
+#ifndef _EVENT_H_
+#define _EVENT_H_
+
+#define EVENT_NONE 0
+#define EVENT_CONNECT 1
+#define EVENT_DISCONNECT 2
+#define EVENT_SYNCSTART 3
+#define EVENT_SYNCDONE 4
+#define EVENT_SYNCINTR 5
+#define EVENT_SPLITBRAIN 6
+
+#define EVENT_MIN EVENT_CONNECT
+#define EVENT_MAX EVENT_SPLITBRAIN
+
+void event_send(const struct hast_resource *res, int event);
+int event_recv(const struct hast_resource *res);
+
+#endif /* !_EVENT_H_ */
Modified: stable/8/sbin/hastd/hast.conf.5
==============================================================================
--- stable/8/sbin/hastd/hast.conf.5 Thu Sep 23 05:24:50 2010 (r213048)
+++ stable/8/sbin/hastd/hast.conf.5 Thu Sep 23 09:02:10 2010 (r213049)
@@ -1,4 +1,5 @@
.\" Copyright (c) 2010 The FreeBSD Foundation
+.\" Copyright (c) 2010 Pawel Jakub Dawidek <pjd at FreeBSD.org>
.\" All rights reserved.
.\"
.\" This software was developed by Pawel Jakub Dawidek under sponsorship from
@@ -27,14 +28,14 @@
.\"
.\" $FreeBSD$
.\"
-.Dd February 1, 2010
+.Dd August 30, 2010
.Dt HAST.CONF 5
.Os
.Sh NAME
.Nm hast.conf
.Nd configuration file for the
.Xr hastd 8
-deamon and the
+daemon and the
.Xr hastctl 8
utility.
.Sh DESCRIPTION
@@ -59,6 +60,7 @@ control <addr>
listen <addr>
replication <mode>
timeout <seconds>
+exec <path>
on <node> {
# Node section
@@ -78,6 +80,7 @@ resource <name> {
name <name>
local <path>
timeout <seconds>
+ exec <path>
on <node> {
# Resource-node section
@@ -169,14 +172,16 @@ The only situation where some small amou
the data is stored on primary node and sent to the secondary.
Secondary node then acknowledges data receipt and primary reports
success to an application.
-However, it may happen that the seconderay goes down before the received
+However, it may happen that the secondary goes down before the received
data is really stored locally.
Before secondary node returns, primary node dies entirely.
When the secondary node comes back to life it becomes the new primary.
Unfortunately some small amount of data which was confirmed to be stored
to the application was lost.
-The risk of such a situation is very small, which is the reason for this
-mode to be the default.
+The risk of such a situation is very small.
+The
+.Ic memsync
+replication mode is currently not implemented.
.It Ic fullsync
.Pp
Mark the write operation as completed when local as well as remote
@@ -184,7 +189,7 @@ write completes.
This is the safest and the slowest replication mode.
The
.Ic fullsync
-replication mode is currently not implemented.
+replication mode is the default.
.It Ic async
.Pp
The write operation is reported as complete right after the local write
@@ -201,6 +206,76 @@ replication mode is currently not implem
Connection timeout in seconds.
The default value is
.Va 5 .
+.It Ic exec Aq path
+.Pp
+Execute the given program on various HAST events.
+Below is the list of currently implemented events and arguments the given
+program is executed with:
+.Bl -tag -width ".Ic xxxx"
+.It Ic "<path> role <resource> <oldrole> <newrole>"
+.Pp
+Executed on both primary and secondary nodes when resource role is changed.
+.Pp
+.It Ic "<path> connect <resource>"
+.Pp
+Executed on both primary and secondary nodes when connection for the given
+resource between the nodes is established.
+.Pp
+.It Ic "<path> disconnect <resource>"
+.Pp
+Executed on both primary and secondary nodes when connection for the given
+resource between the nodes is lost.
+.Pp
+.It Ic "<path> syncstart <resource>"
+.Pp
+Executed on primary node when synchronization process of secondary node is
+started.
+.Pp
+.It Ic "<path> syncdone <resource>"
+.Pp
+Executed on primary node when synchronization process of secondary node is
+completed successfully.
+.Pp
+.It Ic "<path> syncintr <resource>"
+.Pp
+Executed on primary node when synchronization process of secondary node is
+interrupted, most likely due to secondary node outage or connection failure
+between the nodes.
+.Pp
+.It Ic "<path> split-brain <resource>"
+.Pp
+Executed on both primary and secondary nodes when split-brain condition is
+detected.
+.Pp
+.El
+The
+.Aq path
+argument should contain full path to executable program.
+If the given program exits with code different than
+.Va 0 ,
+.Nm hastd
+will log it as an error.
+.Pp
+The
+.Aq resource
+argument is resource name from the configuration file.
+.Pp
+The
+.Aq oldrole
+argument is previous resource role (before the change).
+It can be one of:
+.Ar init ,
+.Ar secondary ,
+.Ar primary .
+.Pp
+The
+.Aq newrole
+argument is current resource role (after the change).
+It can be one of:
+.Ar init ,
+.Ar secondary ,
+.Ar primary .
+.Pp
.It Ic name Aq name
.Pp
GEOM provider name that will appear as
@@ -223,6 +298,24 @@ When operating as a primary node this ad
the secondary node.
When operating as a secondary node only connections from this address
will be accepted.
+.Pp
+A special value of
+.Va none
+can be used when the remote address is not yet known (eg. the other node is not
+set up yet).
+.El
+.Sh FILES
+.Bl -tag -width ".Pa /var/run/hastctl" -compact
+.It Pa /etc/hast.conf
+The default
+.Nm
+configuration file.
+.It Pa /var/run/hastctl
+Control socket used by the
+.Xr hastctl 8
+control utility to communicate with the
+.Xr hastd 8
+daemon.
.El
.Sh EXAMPLES
The example configuration file can look as follows:
@@ -248,19 +341,6 @@ resource tank {
}
}
.Ed
-.Sh FILES
-.Bl -tag -width ".Pa /var/run/hastctl" -compact
-.It Pa /etc/hast.conf
-The default
-.Nm
-configuration file.
-.It Pa /var/run/hastctl
-Control socket used by the
-.Xr hastctl 8
-control utility to communicate with the
-.Xr hastd 8
-daemon.
-.El
.Sh SEE ALSO
.Xr gethostname 3 ,
.Xr geom 4 ,
Modified: stable/8/sbin/hastd/hast.h
==============================================================================
--- stable/8/sbin/hastd/hast.h Thu Sep 23 05:24:50 2010 (r213048)
+++ stable/8/sbin/hastd/hast.h Thu Sep 23 09:02:10 2010 (r213049)
@@ -48,7 +48,12 @@
#include "proto.h"
-#define HAST_PROTO_VERSION 0
+/*
+ * Version history:
+ * 0 - initial version
+ * 1 - HIO_KEEPALIVE added
+ */
+#define HAST_PROTO_VERSION 1
#define EHAST_OK 0
#define EHAST_NOENTRY 1
@@ -74,6 +79,7 @@
#define HIO_WRITE 2
#define HIO_DELETE 3
#define HIO_FLUSH 4
+#define HIO_KEEPALIVE 5
#define HAST_TIMEOUT 5
#define HAST_CONFIG "/etc/hast.conf"
@@ -121,6 +127,8 @@ struct hast_resource {
int hr_extentsize;
/* Maximum number of extents that are kept dirty. */
int hr_keepdirty;
+ /* Path to a program to execute on various events. */
+ char hr_exec[PATH_MAX];
/* Path to local component. */
char hr_localpath[PATH_MAX];
@@ -173,6 +181,8 @@ struct hast_resource {
pid_t hr_workerpid;
/* Control connection between parent and child. */
struct proto_conn *hr_ctrl;
+ /* Events from child to parent. */
+ struct proto_conn *hr_event;
/* Activemap structure. */
struct activemap *hr_amp;
@@ -183,7 +193,7 @@ struct hast_resource {
TAILQ_ENTRY(hast_resource) hr_next;
};
-struct hastd_config *yy_config_parse(const char *config);
+struct hastd_config *yy_config_parse(const char *config, bool exitonerror);
void yy_config_free(struct hastd_config *config);
void yyerror(const char *);
Modified: stable/8/sbin/hastd/hast_proto.c
==============================================================================
--- stable/8/sbin/hastd/hast_proto.c Thu Sep 23 05:24:50 2010 (r213048)
+++ stable/8/sbin/hastd/hast_proto.c Thu Sep 23 09:02:10 2010 (r213049)
@@ -56,8 +56,10 @@ struct hast_main_header {
uint32_t size;
} __packed;
-typedef int hps_send_t(struct hast_resource *, struct nv *nv, void **, size_t *, bool *);
-typedef int hps_recv_t(struct hast_resource *, struct nv *nv, void **, size_t *, bool *);
+typedef int hps_send_t(const struct hast_resource *, struct nv *nv, void **,
+ size_t *, bool *);
+typedef int hps_recv_t(const struct hast_resource *, struct nv *nv, void **,
+ size_t *, bool *);
struct hast_pipe_stage {
const char *hps_name;
@@ -65,14 +67,14 @@ struct hast_pipe_stage {
hps_recv_t *hps_recv;
};
-static int compression_send(struct hast_resource *res, struct nv *nv,
+static int compression_send(const struct hast_resource *res, struct nv *nv,
void **datap, size_t *sizep, bool *freedatap);
-static int compression_recv(struct hast_resource *res, struct nv *nv,
+static int compression_recv(const struct hast_resource *res, struct nv *nv,
void **datap, size_t *sizep, bool *freedatap);
#ifdef HAVE_CRYPTO
-static int checksum_send(struct hast_resource *res, struct nv *nv,
+static int checksum_send(const struct hast_resource *res, struct nv *nv,
void **datap, size_t *sizep, bool *freedatap);
-static int checksum_recv(struct hast_resource *res, struct nv *nv,
+static int checksum_recv(const struct hast_resource *res, struct nv *nv,
void **datap, size_t *sizep, bool *freedatap);
#endif
@@ -84,7 +86,7 @@ static struct hast_pipe_stage pipeline[]
};
static int
-compression_send(struct hast_resource *res, struct nv *nv, void **datap,
+compression_send(const struct hast_resource *res, struct nv *nv, void **datap,
size_t *sizep, bool *freedatap)
{
unsigned char *newbuf;
@@ -132,7 +134,7 @@ compression_send(struct hast_resource *r
}
static int
-compression_recv(struct hast_resource *res, struct nv *nv, void **datap,
+compression_recv(const struct hast_resource *res, struct nv *nv, void **datap,
size_t *sizep, bool *freedatap)
{
unsigned char *newbuf;
@@ -169,7 +171,7 @@ compression_recv(struct hast_resource *r
#ifdef HAVE_CRYPTO
static int
-checksum_send(struct hast_resource *res, struct nv *nv, void **datap,
+checksum_send(const struct hast_resource *res, struct nv *nv, void **datap,
size_t *sizep, bool *freedatap __unused)
{
unsigned char hash[SHA256_DIGEST_LENGTH];
@@ -188,7 +190,7 @@ checksum_send(struct hast_resource *res,
}
static int
-checksum_recv(struct hast_resource *res, struct nv *nv, void **datap,
+checksum_recv(const struct hast_resource *res, struct nv *nv, void **datap,
size_t *sizep, bool *freedatap __unused)
{
unsigned char chash[SHA256_DIGEST_LENGTH];
@@ -236,7 +238,7 @@ checksum_recv(struct hast_resource *res,
* There can be no data at all (data is NULL then).
*/
int
-hast_proto_send(struct hast_resource *res, struct proto_conn *conn,
+hast_proto_send(const struct hast_resource *res, struct proto_conn *conn,
struct nv *nv, const void *data, size_t size)
{
struct hast_main_header hdr;
@@ -293,7 +295,7 @@ end:
}
int
-hast_proto_recv_hdr(struct proto_conn *conn, struct nv **nvp)
+hast_proto_recv_hdr(const struct proto_conn *conn, struct nv **nvp)
{
struct hast_main_header hdr;
struct nv *nv;
@@ -335,7 +337,7 @@ fail:
}
int
-hast_proto_recv_data(struct hast_resource *res, struct proto_conn *conn,
+hast_proto_recv_data(const struct hast_resource *res, struct proto_conn *conn,
struct nv *nv, void *data, size_t size)
{
unsigned int ii;
@@ -384,7 +386,7 @@ if (ret < 0) printf("%s:%u %s\n", __func
}
int
-hast_proto_recv(struct hast_resource *res, struct proto_conn *conn,
+hast_proto_recv(const struct hast_resource *res, struct proto_conn *conn,
struct nv **nvp, void *data, size_t size)
{
struct nv *nv;
Modified: stable/8/sbin/hastd/hast_proto.h
==============================================================================
--- stable/8/sbin/hastd/hast_proto.h Thu Sep 23 05:24:50 2010 (r213048)
+++ stable/8/sbin/hastd/hast_proto.h Thu Sep 23 09:02:10 2010 (r213049)
@@ -37,12 +37,12 @@
#include <nv.h>
#include <proto.h>
-int hast_proto_send(struct hast_resource *res, struct proto_conn *conn,
+int hast_proto_send(const struct hast_resource *res, struct proto_conn *conn,
struct nv *nv, const void *data, size_t size);
-int hast_proto_recv(struct hast_resource *res, struct proto_conn *conn,
+int hast_proto_recv(const struct hast_resource *res, struct proto_conn *conn,
struct nv **nvp, void *data, size_t size);
-int hast_proto_recv_hdr(struct proto_conn *conn, struct nv **nvp);
-int hast_proto_recv_data(struct hast_resource *res, struct proto_conn *conn,
- struct nv *nv, void *data, size_t size);
+int hast_proto_recv_hdr(const struct proto_conn *conn, struct nv **nvp);
+int hast_proto_recv_data(const struct hast_resource *res,
+ struct proto_conn *conn, struct nv *nv, void *data, size_t size);
#endif /* !_HAST_PROTO_H_ */
Modified: stable/8/sbin/hastd/hastd.8
==============================================================================
--- stable/8/sbin/hastd/hastd.8 Thu Sep 23 05:24:50 2010 (r213048)
+++ stable/8/sbin/hastd/hastd.8 Thu Sep 23 09:02:10 2010 (r213049)
@@ -170,6 +170,23 @@ stored.
The default location is
.Pa /var/run/hastd.pid .
.El
+.Sh FILES
+.Bl -tag -width ".Pa /var/run/hastctl" -compact
+.It Pa /etc/hast.conf
+The configuration file for
+.Nm
+and
+.Xr hastctl 8 .
+.It Pa /var/run/hastctl
+Control socket used by the
+.Xr hastctl 8
+control utility to communicate with
+.Nm .
+.It Pa /var/run/hastd.pid
+The default location of the
+.Nm
+PID file.
+.El
.Sh EXIT STATUS
Exit status is 0 on success, or one of the values described in
.Xr sysexits 3
@@ -196,23 +213,6 @@ nodeA# hastctl role primary shared
nodeA# newfs -U /dev/hast/shared
nodeA# mount -o noatime /dev/hast/shared /shared
.Ed
-.Sh FILES
-.Bl -tag -width ".Pa /var/run/hastctl" -compact
-.It Pa /etc/hast.conf
-The configuration file for
-.Nm
-and
-.Xr hastctl 8 .
-.It Pa /var/run/hastctl
-Control socket used by the
-.Xr hastctl 8
-control utility to communicate with
-.Nm .
-.It Pa /var/run/hastd.pid
-The default location of the
-.Nm
-PID file.
-.El
.Sh SEE ALSO
.Xr sysexits 3 ,
.Xr geom 4 ,
Modified: stable/8/sbin/hastd/hastd.c
==============================================================================
--- stable/8/sbin/hastd/hastd.c Thu Sep 23 05:24:50 2010 (r213048)
+++ stable/8/sbin/hastd/hastd.c Thu Sep 23 09:02:10 2010 (r213049)
@@ -1,5 +1,6 @@
/*-
* Copyright (c) 2009-2010 The FreeBSD Foundation
+ * Copyright (c) 2010 Pawel Jakub Dawidek <pjd at FreeBSD.org>
* All rights reserved.
*
* This software was developed by Pawel Jakub Dawidek under sponsorship from
@@ -51,24 +52,25 @@ __FBSDID("$FreeBSD$");
#include <pjdlog.h>
#include "control.h"
+#include "event.h"
#include "hast.h"
#include "hast_proto.h"
#include "hastd.h"
+#include "hooks.h"
#include "subr.h"
/* Path to configuration file. */
-static const char *cfgpath = HAST_CONFIG;
+const char *cfgpath = HAST_CONFIG;
/* Hastd configuration. */
static struct hastd_config *cfg;
-/* Was SIGCHLD signal received? */
-static bool sigchld_received = false;
-/* Was SIGHUP signal received? */
-static bool sighup_received = false;
/* Was SIGINT or SIGTERM signal received? */
bool sigexit_received = false;
/* PID file handle. */
struct pidfh *pfh;
+/* How often check for hooks running for too long. */
+#define REPORT_INTERVAL 10
+
static void
usage(void)
{
@@ -77,22 +79,6 @@ usage(void)
}
static void
-sighandler(int sig)
-{
-
- switch (sig) {
- case SIGCHLD:
- sigchld_received = true;
- break;
- case SIGHUP:
- sighup_received = true;
- break;
- default:
- assert(!"invalid condition");
- }
-}
-
-static void
g_gate_load(void)
{
@@ -139,15 +125,16 @@ child_exit(void)
if (res == NULL) {
/*
* This can happen when new connection arrives and we
- * cancel child responsible for the old one.
+ * cancel child responsible for the old one or if this
+ * was hook which we executed.
*/
+ hook_check_one(pid, status);
continue;
}
pjdlog_prefix_set("[%s] (%s) ", res->hr_name,
role2str(res->hr_role));
child_exit_log(pid, status);
- proto_close(res->hr_ctrl);
- res->hr_workerpid = 0;
+ child_cleanup(res);
if (res->hr_role == HAST_ROLE_PRIMARY) {
/*
* Restart child process if it was killed by signal
@@ -169,12 +156,226 @@ child_exit(void)
}
}
+static bool
+resource_needs_restart(const struct hast_resource *res0,
+ const struct hast_resource *res1)
+{
+
+ assert(strcmp(res0->hr_name, res1->hr_name) == 0);
+
+ if (strcmp(res0->hr_provname, res1->hr_provname) != 0)
+ return (true);
+ if (strcmp(res0->hr_localpath, res1->hr_localpath) != 0)
+ return (true);
+ if (res0->hr_role == HAST_ROLE_INIT ||
+ res0->hr_role == HAST_ROLE_SECONDARY) {
+ if (strcmp(res0->hr_remoteaddr, res1->hr_remoteaddr) != 0)
+ return (true);
+ if (res0->hr_replication != res1->hr_replication)
+ return (true);
+ if (res0->hr_timeout != res1->hr_timeout)
+ return (true);
+ if (strcmp(res0->hr_exec, res1->hr_exec) != 0)
+ return (true);
+ }
+ return (false);
+}
+
+static bool
+resource_needs_reload(const struct hast_resource *res0,
+ const struct hast_resource *res1)
+{
+
+ assert(strcmp(res0->hr_name, res1->hr_name) == 0);
+ assert(strcmp(res0->hr_provname, res1->hr_provname) == 0);
+ assert(strcmp(res0->hr_localpath, res1->hr_localpath) == 0);
+
+ if (res0->hr_role != HAST_ROLE_PRIMARY)
+ return (false);
+
+ if (strcmp(res0->hr_remoteaddr, res1->hr_remoteaddr) != 0)
+ return (true);
+ if (res0->hr_replication != res1->hr_replication)
+ return (true);
+ if (res0->hr_timeout != res1->hr_timeout)
+ return (true);
+ if (strcmp(res0->hr_exec, res1->hr_exec) != 0)
+ return (true);
+ return (false);
+}
+
static void
hastd_reload(void)
{
+ struct hastd_config *newcfg;
+ struct hast_resource *nres, *cres, *tres;
+ uint8_t role;
+
+ pjdlog_info("Reloading configuration...");
+
+ newcfg = yy_config_parse(cfgpath, false);
+ if (newcfg == NULL)
+ goto failed;
+
+ /*
+ * Check if control address has changed.
+ */
+ if (strcmp(cfg->hc_controladdr, newcfg->hc_controladdr) != 0) {
+ if (proto_server(newcfg->hc_controladdr,
+ &newcfg->hc_controlconn) < 0) {
+ pjdlog_errno(LOG_ERR,
+ "Unable to listen on control address %s",
+ newcfg->hc_controladdr);
+ goto failed;
+ }
+ }
+ /*
+ * Check if listen address has changed.
+ */
+ if (strcmp(cfg->hc_listenaddr, newcfg->hc_listenaddr) != 0) {
+ if (proto_server(newcfg->hc_listenaddr,
+ &newcfg->hc_listenconn) < 0) {
+ pjdlog_errno(LOG_ERR, "Unable to listen on address %s",
+ newcfg->hc_listenaddr);
+ goto failed;
+ }
+ }
+ /*
+ * Only when both control and listen sockets are successfully
+ * initialized switch them to new configuration.
+ */
+ if (newcfg->hc_controlconn != NULL) {
+ pjdlog_info("Control socket changed from %s to %s.",
+ cfg->hc_controladdr, newcfg->hc_controladdr);
+ proto_close(cfg->hc_controlconn);
+ cfg->hc_controlconn = newcfg->hc_controlconn;
+ newcfg->hc_controlconn = NULL;
+ strlcpy(cfg->hc_controladdr, newcfg->hc_controladdr,
+ sizeof(cfg->hc_controladdr));
+ }
+ if (newcfg->hc_listenconn != NULL) {
+ pjdlog_info("Listen socket changed from %s to %s.",
+ cfg->hc_listenaddr, newcfg->hc_listenaddr);
+ proto_close(cfg->hc_listenconn);
+ cfg->hc_listenconn = newcfg->hc_listenconn;
+ newcfg->hc_listenconn = NULL;
+ strlcpy(cfg->hc_listenaddr, newcfg->hc_listenaddr,
+ sizeof(cfg->hc_listenaddr));
+ }
- /* TODO */
- pjdlog_warning("Configuration reload is not implemented.");
+ /*
+ * Stop and remove resources that were removed from the configuration.
+ */
+ TAILQ_FOREACH_SAFE(cres, &cfg->hc_resources, hr_next, tres) {
+ TAILQ_FOREACH(nres, &newcfg->hc_resources, hr_next) {
+ if (strcmp(cres->hr_name, nres->hr_name) == 0)
+ break;
+ }
+ if (nres == NULL) {
+ control_set_role(cres, HAST_ROLE_INIT);
+ TAILQ_REMOVE(&cfg->hc_resources, cres, hr_next);
+ pjdlog_info("Resource %s removed.", cres->hr_name);
+ free(cres);
+ }
+ }
+ /*
+ * Move new resources to the current configuration.
+ */
+ TAILQ_FOREACH_SAFE(nres, &newcfg->hc_resources, hr_next, tres) {
+ TAILQ_FOREACH(cres, &cfg->hc_resources, hr_next) {
+ if (strcmp(cres->hr_name, nres->hr_name) == 0)
+ break;
*** DIFF OUTPUT TRUNCATED AT 1000 LINES ***
More information about the svn-src-all
mailing list